[R] Doc-to-LoRA: Learning to Instantly Internalize Contexts from Sakana AI

This is cool paper! Creating loras from docs on the fly using a hypernetwork. "Long input sequences are central to in-context learning, document understanding, and multi-step reasoning of Large Language Models (LLMs). However, the quadratic attention cost of Transformers makes inference memory-intensive and slow. While context distillation (CD) can transfer information into model parameters, per-prompt distillation is impractical due to