Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossible

ArXi:2510.07178v2 Announce Type: replace Are large language models (LLMs) sensitive to the distinction between humanly possible and impossible languages? This question was recently used in a broader debate on whether LLMs and humans share the same innate learning biases. Previous work has answered it in the positive by comparing LLM learning curves on existing language datasets and on "impossible" datasets derived from them via various perturbation functions. Using the same methodology, we examine this claim on a wider set of languages and impossible perturbations.