Enhancing Hyperspace Analogue to Language (HAL) Representations via Attention-Based Pooling for Text Classification

ArXi:2603.20149v1 Announce Type: cross The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all tokens, thereby diluting the impact of contextually salient words with uninformative structural tokens.