AI RESEARCH
Functional Component Ablation Reveals Specialization Patterns in Hybrid Language Model Architectures
arXiv CS.AI
•
ArXi:2603.22473v1 Announce Type: cross Hybrid language models combining attention with state space models (SSMs) or linear attention offer improved efficiency, but whether both components are genuinely utilized remains unclear. We present a functional component ablation framework applied to two sub-1B hybrid models -- Qwen3.5-0.8B (sequential: Gated DeltaNet + softmax attention) and Falcon-H1-0.5B (parallel: Mamba-2 + attention) -- with a pure Transformer control (Qwen2.5-0.5B.