AI RESEARCH
Do Language Models Encode Knowledge of Linguistic Constraint Violations?
arXiv CS.CL
•
ArXi:2605.12055v1 Announce Type: new Large Language Models (LLMs) achieve strong linguistic performance, yet their internal mechanisms for producing these predictions remain unclear. We investigate the hypothesis that LLMs encode representations of linguistic constraint violations within their parameters, which are selectively activated when processing ungrammatical sentences. To test this, we use sparse autoencoders to decompose polysemantic activations into sparse, monosemantic features and recover candidates for violation-related features.