AI RESEARCH
Open-source single-GPU reproductions of Cartridges and STILL for neural KV-cache compaction [P]
r/MachineLearning
•
I implemented two recent ideas for long-context inference / KV-cache compaction and open-sourced both reproductions: Cartridges: STILL: The goal was to make the ideas easy to inspect and run, with benchmark code and readable implementations instead of just paper/blog summaries.