I stumbled on a Gemma 4 chat template bug for tools and fixed it
r/LocalLLaMA
•
Generative AI
Open Source AI
TLDR: tool parameters using the common JSON Schema pattern `anyOf: [$ref, null]` are rendered into the prompt as empty `type` fields. This strips the useful schema information before the model sees it. -- Long, rambling version: Gemma 4 was having issues with calling my custom MCP tool on >3 inference engines, while Qwen3.5 and gpt-oss-20b were doing fine. I guessed it was either a chat template issue or inference library issue on an edge case, and thought time would sort it out, since many people were happy with Gemma 4 as an agent.