Interpretable-by-Design Transformers via Architectural Stream Independence

ArXi:2603.07482v1 Announce Type: new While transformers achieve strong performance, their internal decision-making processes remain opaque. We investigate whether architectural constraints can enforce interpretability by design through architectural stream independence: maintaining a token stream (carrying symbolic structure) and contextual semantics in separated streams that remain independently observable throughout processing, with integration delayed until output.