AI RESEARCH
HeceTokenizer: A Syllable-Based Tokenization Approach for Turkish Retrieval
arXiv CS.CL
•
ArXi:2604.10665v1 Announce Type: new HeceTokenizer is a syllable-based tokenizer for Turkish that exploits the deterministic six-pattern phonological structure of the language to construct a closed, out-of-vocabulary (OOV)-free vocabulary of approximately 8,000 unique syllable types. A BERT-tiny encoder (1.5M parameters) is trained from scratch on a subset of Turkish Wikipedia using a masked language modeling objective and evaluated on the TQuAD retrieval benchmark using Recall.