AI RESEARCH
CharBench: Evaluating the Role of Tokenization in Character-Level Tasks
arXiv CS.CL
•
ArXi:2508.02591v3 Announce Type: replace Tasks that require character-level reasoning, such as counting or locating characters within words, remain challenging for contemporary language models. A common conjecture is that language models' reliance on subword units, rather than characters, contributes to their struggles with character-level tasks, yet recent studies offer conflicting