Large Language Models: How Do They Pick Every Word?

Quick game. Finish this for me: peanut butter and… You said jelly, didn't you? A few of you went with banana. Some of you may have even said "Jelly Time" but Nobody said carburetor. In half a second, your brain ranked a pile of possible next words and grabbed a likely one. Remember that feeling, because that little guessing game is the engine inside every chatbot you've ever talked to.

Large language models, LLMs for short, are enormous math functions with a single job. Hand one some text, and it scores every token that could come next. Tokens are the model's building blocks: usually a whole word, sometimes a piece of one. In English, a hundred tokens works out to roughly seventy-five words, and a rare word gets chopped into a few chunks.

So how does a next-token guesser hold a conversation? It wears a costume. Behind the chat window, the app quietly writes a little document: some instructions, then the word User and your message, then the word Assistant and a blank. The model keeps extending that document, and every token it picks gets glued on before it guesses again. The reply on your screen is just that document growing.

Here's what a single guess really is. The model doesn't hand back a word. It hands back a score for every token in its vocabulary. For GPT-3, that's fifty thousand two hundred fifty-seven scores. A function called softmax turns them into probabilities: raise e to the power of each score, then divide by the total, so every value is positive and they all add up to one.

Now it has to choose. Always grabbing the favorite sounds safe, but it produces stiff, repetitive writing, so chatbots usually roll weighted dice instead. The knob for those dice is called temperature: divide every score by it before the softmax. Turn it down, and the favorite wins almost every time. Turn it up, and long shots get a real chance. The probabilities come out the same on every run. The dice roll is where the variety comes from.

Where do the scores come from? From parameters: numbers the model multiplies its inputs by, arranged in huge grids called matrices. A list of numbers goes in, gets multiplied by a grid of weights, gets added up, and the result feeds the next step, over and over. GPT-3 has a hundred seventy-five billion parameters. Meta's Llama three point one has four hundred five billion. And nobody types any of them in. They're learned.

Learning means playing fill-in-the-blank on a mountain of real text. Hide the next token, let the model guess, then check the answer. How wrong it was gets measured by the loss: the negative log of the probability it gave the correct token. Put ninety percent on the right answer, and the loss is tiny. Put one percent on it, and the loss shoots up.

After each check, every parameter gets a tiny nudge in whichever direction would have shrunk the loss. Picture the loss as a landscape, with one direction for every parameter, and training as a ball rolling downhill in small steps. That's gradient descent. Working out which way is downhill for billions of parameters at once is the job of backpropagation, which is really just the chain rule from calculus, run backward through the whole network.

Now for the scale, and it's ridiculous. GPT-3 trained on about three hundred billion tokens, and the total arithmetic came to roughly three point one four times ten to the twenty-third operations. At one operation per second, you'd need about seven hundred thousand times the age of the universe. Even one of today's best AI chips, running flat out, would need about ten years. So the work gets split across thousands of chips running in parallel.

And parallel is the magic word. Older language models took in text the way we do, one token after another, and that's hard to split across chips. In twenty seventeen, a team at Google published the transformer, a design that takes in every token of a passage at once. It fit the hardware perfectly, and it's the T in GPT.

Step one inside a transformer: each token becomes a vector, basically a very long row of numbers. In GPT-3, every vector holds twelve thousand two hundred eighty-eight of them. Think of them as coordinates in a space of meaning, where related ideas end up near each other. Salsa tends to land nearer to nachos than to motherboard. We can only draw two of those dimensions, but the idea holds up.

The trouble is, one token on its own can be ambiguous. Take chip. In "I dunked a chip in the salsa," it's a snack. In "the robot needs a faster chip," it's hardware. Attention is how context gets in. Each token sends out a query, each token offers a key, and the model scores every match at once. The best matches pass along their values, and chip slides toward the snack side or the hardware side of that space.

Once the tokens have compared notes, each vector gets a private upgrade: a small neural network called a multilayer perceptron, where research suggests a lot of a model's stored knowledge is kept. Attention, then perceptron, then do it again. GPT-3 stacks that pair ninety-six times, and by the top of the stack, each vector carries a rich summary of everything that came before it.

The very last vector in line is the one that makes the call. One final multiplication turns it into those fifty thousand two hundred fifty-seven scores, softmax turns the scores into probabilities, the dice pick a token, and that token gets glued onto the text. Then the whole machine runs again for the next one. Every chatbot reply you've ever seen was built exactly like this, one token at a time.

There's one more stage before it feels like an assistant. A model trained only on web text is brilliant at continuing documents, but it isn't automatically helpful. So it gets extra training. First on example conversations that show good answers. Then on human feedback: people compare pairs of answers and pick the better one, those picks train a reward model, and reinforcement learning nudges the parameters toward answers people prefer. That recipe comes from OpenAI's InstructGPT work.

And here's the part that still gets me. People build the architecture and choose the training recipe, but nobody writes rules for grammar, or facts, or jokes. Those emerge from billions of tiny nudges, which is why figuring out exactly why a model said something is a whole research field of its own, called interpretability. So next time a chatbot answers you, picture a machine that has soaked up more writing than any person ever could, rolling weighted dice, one token at a time. Peanut butter and… you know the rest.