AI RESEARCH
Representation Without Reward: A JEPA Audit for LLM Fine-Tuning
arXiv CS.AI
•
ArXi:2605.15394v1 Announce Type: cross Joint-embedding predictive architectures (JEPAs) propose that a model should learn useful abstractions when trained to predict latent representations rather than observed outputs. For autoregressive language-model fine-tuning the principle entails a stricter requirement: the induced hidden-state geometry must reach the language-model head \emph{and} improve the decoded task metric. We test that requirement under a fixed Llama-3.2-1B-Instruct LoRA harness on natural-language-to-regex generation, comparing twenty-two.