What Are Diffusion Language Models, and Why Are They Different From GPT?

In this article, you will learn how diffusion language models generate text by iteratively refining a masked sequence, and how this differs from the token-by-token generation of GPT-style models.

Topics we will cover include:

• How autoregressive models and masked diffusion models differ in the way they generate text.
• How forward and reverse diffusion work, and how masking adapts diffusion to discrete text tokens.
• The open challenges diffusion language models still face, from token dependencies to key-value caching.

Diffusion itself is not a new idea. It is well known from the image generators that transform random noise into meaningful images, and this work was presented in the Denoising Diffusion Probabilistic Models paper. However, the problem is that this idea of starting with something noisy and then refining it is easier to understand with images, which consist of continuous values. Applying this idea to text is usually hard, as text is made of discrete token IDs, and adding a little Gaussian noise to a token ID does not produce a meaningful nearby word.

However, recent work, including LLaDA, Mercury, and Gemini Diffusion in 2025, and then Google’s DiffusionGemma in 2026, has drawn the community’s attention to diffusion models because of their faster inference. A traditional large language model (LLM) predicts the next token, adds it to the sequence, and then repeats this. But diffusion models predict multiple positions simultaneously and refine their predictions before producing an answer. This raises a major research question in the community:

Do language models even need to generate one token at a time?

I will not pass a verdict on it. Also, this topic is kind of complex, so I don’t expect you to understand it fully, and I have kept the math as minimal as possible. But I do want you to take away the intuition of what is happening. In this article, we will study:

1. How GPT and diffusion models generate text.
2. How diffusion models actually work.
3. Challenges that still exist in diffusion models, and some resources for you to explore more.

So, let’s get started.

Autoregressive vs. Diffusion

GPT models are autoregressive in nature, which means that they estimate the next token using the prefix that has already been generated. So, if you consider a sequence \( x = (x_1, \dots, x_N) \), then its probability factorization is:

\[
p(x) = \prod_{i=1}^{N} p(x_i \mid x_1, \dots, x_{i-1})
\]

Now, during training, you can process many token positions in parallel using teacher forcing. But when it comes to inference, you must wait for the previously generated token before choosing the next. This creates a serial dependency problem.

On the other hand, a masked diffusion model starts with an output canvas of unknown positions and progressively reveals tokens. It can choose several tokens in a single step, potentially in a non-left-to-right order. Let me show you with a table that will help you understand it better:

So, you can see that although both of them use transformer architectures, their objectives and inference procedures differ. Autoregressive models extend a sequence, whereas diffusion models refine a sequence.

Now, we will go a bit deeper to see how diffusion models work.

How Diffusion Models Actually Work

1. Forward and Reverse Diffusion

Let’s start with image diffusion. There are two processes: forward and reverse. In the forward process, you basically corrupt the clean data by progressively adding noise, like this:

Forward diffusion process corrupting an image with noise.

Then you train the neural network to reverse this process, commonly using a mean squared error objective. Sampling then reverses this corruption in many steps. A standard expression for the forward (noising) process is:

\[
x_t = \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1-\bar{\alpha}_t}\, \epsilon
\]

Here, \( x_0 \) is clean data, \( x_t \) is its corrupted state at noise level \( t \), and \( \epsilon \) is Gaussian noise.

2. Masked Diffusion for Text

As I mentioned above, token IDs are different from the continuous-valued pixels in images. Therefore, for language, corruption is commonly done by replacing tokens with [MASK]. For example, if you have a sentence:

Machine learning models can generate text.

It might look like this after masking:

Machine [MASK] models [MASK] generate [MASK].

Now, the model tries to predict these original tokens from this corrupted sequence. You can set the corruption level \( t \) between 0 and 1 and then replace each token independently with [MASK] with probability \( t \). So at \( t = 0 \) your sequence is clean, and at \( t = 1 \), everything is masked. During training, the model learns to recover the missing tokens at different noise levels. By the way, masking is one of the most common approaches, but there are other options too.

Now that the training is done, what happens at inference?

3. Denoising at Inference

At inference time, we have a sampler that (1) initializes masked output positions, which is the starting point, (2) then predicts the vocabulary distribution at every unresolved position, and (3) then decides which predictions to commit and which to leave masked for subsequent rounds. This is usually decided through some confidence-based metric, but it’s just an implementation-level choice. (4) The process repeats until all positions are filled. Also, some samplers can remask previously committed tokens based on how the sentence is forming, which is often called revision, but this is not a property of every sampler.

There are two more important things that I want to highlight before moving forward:

1. GPT-style causal attention allows each position to attend only to earlier positions. But many diffusion models use bidirectional attention inside the active denoising sequence. This means that a masked word in “The scientist presented [MASK] at the conference” can use both the preceding and the following context. This is useful for infilling, editing, and structured generation. It does not mean the model sees a correct future answer: surrounding positions may still be masks or wrong guesses. There are also some hybrid designs that use causal attention between completed blocks and bidirectional attention within the current block.
2. Secondly, diffusion can win on wall-clock latency when GPUs efficiently process parallel token positions. But results depend on lots of other factors. Also, quality matters, and autoregressive models remain stronger on some reasoning and coding benchmarks. You can see the DiffusionGemma 26B vs. Gemma 26B results in the model card.

Challenges With Diffusion Models

I do want to highlight some of the important issues that are still under discussion for diffusion models, and then I will share some further resources if you are interested in exploring this topic further:

1. Token dependencies: You may already get this from our discussion above, but if two positions are sampled in parallel, each may look plausible on its own while being inconsistent with the other. More denoising rounds, remasking, and coupled sampling might be needed to preserve coherence.
2. Sampling budget: Aggressive low-step sampling improves latency but risks quality loss, and more passes may erode the speed advantage. So you need to find a good balance between the two.
3. Variable output length: A fixed masked canvas does not naturally know how long the answer should be. But there are some blockwise and more flexible editing approaches that address this.
4. Key-value (KV) caching: Bidirectional refinement can alter representations across a block, making conventional prefix caching harder. Blockwise decoding helps by freezing completed blocks.

Final Thoughts and References

I personally think that diffusion is best understood as another way to allocate computation during generation, not as an automatic replacement for GPT-style models. Each of them has its own tradeoffs, and hybrid techniques might be a better route toward production deployment. As promised, here are some of the best resources, or at least good starting points, that you can explore further at your own pace:

• Denoising Diffusion Probabilistic Models — foundational paper.
• How to Build a Diffusion Language Model — my personal favorite.
• LLaDA: Large Language Diffusion Models
• DiffusionGemma: The Developer Guide
• Dream 7B, a large diffusion language model — it is open source, with both code and model checkpoints available.
• Breaking the Factorization Barrier — research on dependencies between parallel predictions.