AI RESEARCH
CALYREX: Cross-Attention LaYeR EXtended Transformers for System Prompt Anchoring
arXiv CS.LG
•
ArXi:2605.09737v1 Announce Type: new Modern large language models (LLMs) rely on system prompts to establish behavioral constraints and safety rules. Standard causal self-attention treats privileged instructions and untrusted user content with equal structural priority -- a mismatch that leaves models vulnerable to prompt injection and instruction erosion over extended contexts. We propose CALYREX (Cross-Attention LaYeR EXtended transformers), which utilizes cross-attention between input and system prompt to structurally isolate and anchor the rule.