AI RESEARCH
Implicit Representations of Grammaticality in Language Models
arXiv CS.CL
•
ArXi:2605.05197v1 Announce Type: new Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitted to maximize corpus likelihood, generate grammatically well-formed text and discriminate well between grammatical and ungrammatical sentences in tightly controlled minimal pairs. However, their string probabilities do not sharply discriminate between grammatical and ungrammatical sentences overall.