SLM to controll NPC in a game world

Hello everybody, I am working on a project where the player gives commands to a creature in a structured game world and the creature shall react to the player's prompt in a sensible way. The world is described as JSON with distances, directions, object type, unique id The prompt examples are: - Get the closest stone - Go to the tree in the north - Attack the wolf - Get any stone but avoid the wolf And the output is (grammar enforced) JSON with action (move, attack, idle, etc) and the target plus a reasoning for debugging. I tried Qwen 1.5B instruct and reasoning models it works semi well.