AI SAFETY & ETHICS
Implications Of Predicting The Next Token
LessWrong AI
•
I find that a lot of people have trouble with this concept of predicting the next token. And by trouble, I mean that they struggle to understand what it actually means to predict the next token. It seems simpler than it is. Because when you say "predict the next token," I think what most people think of is the Marko chain intuition that you have a big table of statistics, and then you look at what word is the next most likely, and then you pick that as the word. The thing about this is that if you have ever used a Marko chain, you would know that Marko chain text is complete gibberish.