AI RESEARCH
LatentQA: Teaching LLMs to Decode Activations Into Natural Language
arXiv CS.LG
•
ArXi:2412.08686v2 Announce Type: replace-cross Top-down transparency typically analyzes language model activations using probes with scalar or single-token outputs, limiting the range of behaviors that can be captured. To alleviate this issue, we develop a expressive probe that can directly output natural language, performing LatentQA: the task of answering open-ended questions about activations. A key difficulty in developing such a probe is collecting a dataset mapping activations to natural-language descriptions.