E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning

ArXi:2409.06679v3 Announce Type: replace Processing long contexts is increasingly important for Large Language Models (LLMs) in tasks like multi-turn dialogues, code generation, and document summarization. This paper addresses the challenges of achieving high long-context performance, low computational complexity, and compatibility with pretrained models -- collectively termed the ``impossible triangle''. We