llm-all-models-async 0.1

Simon Willison Blog
Generative AI

Release: llm-all-models-async 0.1 LLM plugins can define new models in both sync and async varieties. The async variants are most common for API-backed models - sync variants tend to be things that run the model directly within the plugin. My llm-mrchatterbox plugin is sync only. I wanted to try it out with various Datasette LLM features (specifically datasette-enrichments-llm ) but Datasette can only use async models. So. I had Claude spin up this plugin that turns sync models into async models using a thread pool.