LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations

ArXi:2509.12539v2 Announce Type: replace-cross We present LEAF ("Lightweight Embedding Alignment Framework"), a knowledge distillation framework for text embedding models. A key distinguishing feature is that our distilled leaf models are aligned to their teacher. In the context of information retrieval, this allows for flexible asymmetric architectures where documents are encoded with the larger teacher model, while queries can be served with the smaller leaf models.