Before LLMs Could Predict, They Had to Count

Every language model, from a 1990s bigram counter to GPT-4, does the same job: given some words, figure out what word comes next.