Composable CFG grammars for llama.cpp (pygbnf)

r/LocalLLaMA
Generative AI Open Source AI

It was becoming increasingly painful for me to get a constrained generation library working reliably on my Mac for local experiments. Guidance is great, but I kept running into version mismatches with llama-cpp-python. In practice it made it hard to experiment locally with anything beyond structured JSON outputs. So I ended up writing a small library called pygbnf. (available via pip) It lets you define context-free grammars in Python in a fairly lightweight way (inspired by Guidance’s style) and use them for constrained generation.