AI API Token Pricing Explained — A Buyer's Guide

What is a token and how tokenization works Before you can understand pricing, you need to understand what you are paying for. A **token** is the smallest unit of text an LLM processes. Most providers use sub-word tokenizers (BPE or SentencePiece) that split text into pieces roughly 3-4 characters long. The word "tokenization" becomes three or four tokens; a short JSON payload may use tokens than the same data in plain English. Tokenizer choice varies by provider and model family. That means the same prompt can cost different amounts depending on which API you call.