Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures

ArXi:2305.18915v1 Announce Type: cross In this work we build upon negative results from an attempt at language modeling with predicted semantic structure, in order to establish empirical lower bounds on what could have made the attempt successful. specifically, we design a concise binary vector representation of semantic structure at the lexical level and evaluate in-depth how good an incremental tagger needs to be in order to achieve better-than-baseline performance with an end-to-end semantic-bootstrapping language model.