AI RESEARCH
GiLT: Augmenting Transformer Language Models with Dependency Graphs
arXiv CS.CL
•
ArXi:2605.15562v1 Announce Type: new Augmenting Transformers with linguistic structures effectively enhances the syntactic generalization performance of language models. Previous work in this direction focuses on syntactic tree structures of languages, in particular constituency tree structures. We propose Graph-Infused Layers Transformer Language Model (GiLT) which leverages dependency graphs for augmenting Transformer language models.