Composable CFG grammars for llama.cpp (pygbnf)
r/LocalLLaMA
•
Generative AI
Open Source AI
It was becoming increasingly painful for me to get a constrained generation library working reliably on my Mac for local experiments. Guidance is great, but I kept running into version mismatches with llama-cpp-python. In practice it made it hard to experiment locally with anything beyond structured JSON outputs. So I ended up writing a small library called pygbnf. (available via pip) It lets you define context-free grammars in Python in a fairly lightweight way (inspired by Guidance’s style) and use them for constrained generation.