Show HN: Caliper – Auto Instrumented LLM Observability with Custom Metadata

Caliper is designed to auto instrument LLM calls within Python, it monkey patches the OpenAI and Anthropic SDKs (Got plans to add LiteLLM so you can use any provider you want to) so it's almost completely invisible to you as the developer and for basic metrics can slot in as a single init at start. It can also gather custom metadata about a call, this can be any KV pairs you want, both pre and post request.