How Large Language Models Work: From Tokens to ChatGPT, Step by Step

The Whole Idea in One Chain

Here is the full journey. Do not worry if it reads like a riddle; the rest of the article unpacks each link.

Break a sentence into pieces and you get tokens. Turn tokens into lists of numbers and you get embeddings. Let every token ask a question and you get a query; let the others answer and you get keys and values. Compare queries with keys and you get attention scores. Normalize them with softmax and you get attention weights. Use the weights to mix the values and you get context. Do this for every token and you get self-attention. Add a feed-forward network, residual connections and normalization and you get a transformer block. Stack the blocks and you get a transformer. Train it to predict the next token and you get a language model. Teach it to follow instructions and learn from human feedback and you get something close to ChatGPT.

Step 1: Break a Sentence Into Pieces and You Get Tokens

A computer cannot read words the way you do. The first thing a language model does is cut text into small pieces called tokens. A token can be a whole word (“interview”), part of a word (“inter” + “view”), a punctuation mark, or even a space attached to a word.

Why not whole words? There are too many, and new ones appear every day. Word pieces let a model handle any text, including rare names and typos, with a fixed vocabulary of tens of thousands of tokens. Each token in that vocabulary gets an ID number. The sentence “I aced the interview” might become something like [40, 1734, 279, 7274].

One token is roughly three quarters of an English word, so a “128,000-token context window” means about 95,000 words the model can consider at once.

Step 2: Turn Tokens Into Vectors and You Get Embeddings

A token ID like 7274 is just a label. It says nothing about meaning. So the model looks up each ID in a large table and replaces it with a long list of numbers, often hundreds or thousands of them. That list is called a vector, and the vector that represents a token is called an embedding.

Nobody writes these numbers by hand. They start out random and are adjusted during training until they capture what each token means and how it is used.

Step 3: Plot the Embeddings in Space and Meaning Becomes Geometry

Treat each embedding as the coordinates of a point and every token becomes a location in a huge, many-dimensional space. You cannot picture a thousand dimensions, but the idea works in two or three.

In that space, tokens with similar meanings end up close together. “Salary” sits near “pay” and “compensation.” “Interview” sits near “meeting” and “hiring.” Even directions can carry meaning: the famous example is that the step from “man” to “woman” points in roughly the same direction as the step from “king” to “queen.”

Embeddings alone do not say where a token sits, and order changes meaning: “the candidate interviewed the manager” is not the same as “the manager interviewed the candidate.” So models also add positional information to each vector.

Step 4: Every Token Asks and Answers Questions: Queries, Keys and Values

Words change meaning with their neighbors: “bank” means something different in “river bank” and “bank account.” So each token needs to look at the others. This is the most important idea in modern AI.

From every token’s embedding, the model creates three new vectors by multiplying it with three learned tables of numbers:

• Query: the question this token is asking. Think of the word “it” asking, “Which noun earlier in the sentence am I referring to?”
• Key: what this token offers, like a label on a filing cabinet. A noun’s key might effectively say, “I am a thing that can be referred to.”
• Value: the actual information this token will hand over if another token decides it is relevant.

Think of a library: the query is your search, the keys are the titles on the shelves, and the values are what is inside the books.

Step 5: Compare a Query With the Keys and You Get Attention Scores

Each token compares its query with the key of every token it can see, using a dot product: multiply the two vectors number by number and add up the results. Similar directions give a large result; unrelated ones give a small or negative one.

The result is a set of attention scores, one for each pair of tokens. A high score means “this other token looks relevant to my question.” In the sentence “The recruiter called the candidate because she liked her portfolio,” the token “her” should produce a high score with “candidate” and a lower one with “recruiter.”

Step 6: Normalize the Scores With Softmax and You Get Attention Weights

Raw scores can be any size, positive or negative. A function called softmax turns them into positive values that add up to exactly 1, giving the biggest scores a much bigger share.

These shares are the attention weights, like percentages of focus. A simplified example for “her”:

So “her” will pay about 84% of its attention to “candidate.”

Step 7: Use the Weights to Combine the Values and You Get Context

Now each token builds a new vector by taking a weighted mix of the values: 84% of the value from “candidate,” 11% from “recruiter,” and so on. The result is a new representation of “her” that carries information about who “her” actually is.

That is what context means inside a model. The word is no longer just “her” in the abstract; it is “her, meaning the candidate, in a sentence about a portfolio.”

Step 8: Let Every Token Do This and You Get Self-Attention

When every token runs this process at the same time, it is called self-attention: the sentence paying attention to itself.

Two refinements make it powerful:

• Only the tokens it can see. In a model that writes text, each token may look only at the tokens before it, never ahead. This causal mask exists because, while generating, future words do not exist yet.
• Many heads at once. Models run several attention processes in parallel, called attention heads, each with its own query, key and value tables. One might track grammar, another who refers to whom. This is multi-head attention.

Step 9: Add Feed-Forward, Residuals and Normalization and You Get a Transformer Block

Attention shares information; the model also needs to process it. Three more parts complete the unit:

• Feed-forward network: after attention, each token’s vector passes on its own through a small neural network. If attention is a meeting where everyone shares notes, this is each person processing them at their desk.
• Residual connections: instead of replacing a token’s vector with the new result, the model adds the result to the original. This keeps earlier information intact and makes deep networks easier to train.
• Normalization: a step that rescales the numbers to a steady range so they do not grow out of control or shrink to nothing as they pass through many layers.

Attention plus feed-forward, wrapped with residual connections and normalization, is one transformer block.

Step 10: Stack the Blocks and You Get a Transformer

One block gives a modest understanding. Stack dozens of them, the output of one feeding the next, and you get a transformer, the architecture introduced by Google researchers in the 2017 paper “Attention Is All You Need.” It is the “T” in GPT, which stands for Generative Pre-trained Transformer.

Each layer refines the picture: early layers tend to capture grammar, later ones more abstract ideas such as tone and intent. The weights inside all those tables, the numbers the model learns, are what people mean by parameters. Large models have billions of them.

Step 11: Train It to Predict the Next Token and You Get a Language Model

So far we have a machine that turns text into context-aware vectors. Now we give it a job. At the end of the stack, the model turns the vector of the last token into a score for every token in its vocabulary, then applies softmax again to get a probability for each one. In other words: “Given everything so far, what comes next?”

During training, the model reads enormous amounts of text: books, articles, code, websites. At every position, it guesses the next token, compares its guess with the real one, and nudges all its parameters slightly in the direction that would have made the right answer more likely. Repeat that trillions of times and the result is a language model.

This simple task teaches a lot: to predict the next word in a textbook or a contract, a model has to pick up grammar, facts and reasoning patterns.

Generating text is just this prediction on repeat. The model picks a next token, usually by sampling from the probabilities (a setting called temperature controls how adventurous the pick is), adds it to the text, and predicts again, one token at a time.

Step 12: Teach It to Follow Instructions and Learn From Feedback and You Get ChatGPT

A model trained only on next-token prediction is a brilliant autocomplete, not an assistant. Ask it a question and it might reply with three more questions. Two more training stages fix that:

1. Instruction tuning: the model is trained on many examples of a request followed by a good, helpful answer. It learns the pattern of “someone asks, I respond usefully.”
2. Learning from human feedback: people compare pairs of model answers and pick the better one. Those preferences are used to steer the model further toward answers that are helpful, accurate and safe. This is known as reinforcement learning from human feedback, or RLHF.

Add a chat interface and you get something close to ChatGPT, Claude or Gemini: the same transformer, shaped to act as an assistant.

Putting It All Together: One Sentence’s Journey

Imagine you type, “The hardest interview question is…” In a fraction of a second:

1. The text is split into tokens and each becomes an embedding with position information.
2. In every transformer block, each token forms a query, key and value, scores the earlier tokens, turns the scores into weights and mixes in their values. “Hardest” links to “question,” and the phrase is understood as an interview topic.
3. Feed-forward layers process each updated vector; residuals and normalization keep everything stable; dozens of blocks refine it further.
4. The final vector becomes probabilities over the vocabulary. A token like “tell” scores high; the model picks it, appends it, and starts again.

Quick Glossary

Why This Matters for Your Career

If you are interviewing for a role that touches AI, explaining tokens or attention in plain words is a real advantage. It also makes you a smarter user of AI tools: you will see why they can state wrong facts confidently (they predict likely text, not verified truth) and why a clear, specific prompt gets better answers. For practical uses, see our guides to using AI tools to practice job interviews and what AI interview copilots actually change.

Frequently Asked Questions

What is the difference between a token and a word?

A token is the unit a language model reads. Common words are often a single token, while longer or rarer words are split into several pieces. In English, one token is roughly three quarters of a word on average.

Does a language model understand what it says?

It builds rich internal representations of meaning and context, but it is trained to predict likely text, not to check facts. That is why its answers can sound confident and still be wrong, so important claims should always be verified.

Is ChatGPT just a transformer?

At its core, yes: a large transformer trained to predict the next token. What makes it a helpful assistant is the extra training on instructions and human feedback, plus the product built around it.