How AI image generators work: from noise to picture
Every AI picture starts as pure static. We ran one prompt at 1 to 50 steps to show what the model does at each one.
On this page
Type “a lighthouse at sunset” into an AI image generator and, a few seconds later, a lighthouse appears. It is tempting to picture the AI searching a giant library of photos and pasting pieces together. That is not what happens. The picture is not found anywhere. It is grown from a screen of random static, one clean-up pass at a time.
That one fact explains most of what confuses people about these tools. An image generator starts from pure noise and removes a little of it at each step, steered by your prompt. It is why the same prompt gives a new picture each time, why a seed can bring one back, why fast models cost less, and why hands and lettering used to come out mangled. We ran one prompt through FLUX.1 dev at seven step counts and five guidance settings to show each stage. The whole test cost $0.04.
Every AI picture starts as random static
Picture an old television tuned between channels: a field of random grey speckles. A diffusion model, the kind behind FLUX, Stable Diffusion and most video generators, starts with a field like that. Then it asks one question, over and over: which part of this image looks like noise, and what would it look like with that noise removed?
Each answer is one step. After the first step the static has a hint of structure, like a pink top half and a dark bottom half. After a few more, shapes appear. After a couple of dozen, there is a lighthouse with a lamp, a cottage and gulls. The hero at the top of this page shows exactly that: six runs of the same prompt and the same starting noise, stopped after 1, 2, 4, 8, 16 and 28 steps.
The starting static is where the randomness lives. A number called the seed decides which static you get, so a new seed means a new layout. In our earlier post on why the same prompt gives a different answer, two FLUX runs with the same seed returned byte-for-byte identical files. Here, too, the 28-step image and the guidance 3.5 image came back as the same file, because they were the same request.
The model learned by watching pictures dissolve
How does a model learn to remove noise? By practising the opposite first. Training takes millions of real images, adds a little random noise, then a little more, until each one is pure static. The model is shown every stage and asked to guess the noise that was just added. Do that billions of times and it gets very good at the guess.
The idea goes back to a 2015 paper by Jascha Sohl-Dickstein and colleagues, who described it as slowly destroying the structure in data and learning a reverse process that restores it. It became practical in 2020, when Jonathan Ho, Ajay Jain and Pieter Abbeel published denoising diffusion probabilistic models, or DDPM. Their images scored an FID of 3.17 on the CIFAR10 benchmark, a state-of-the-art result at the time.
The catch was speed. DDPM used 1,000 steps for every picture. Sampling a batch of 256 tiny 32×32 images took 17 seconds, and a batch of 128 images at 256×256 took 300 seconds, according to the paper’s appendix. Nearly everything since has been about getting the same quality with less work.
One point matters for anyone worried about copying. The model does not store its training images and stitch them back together. It stores a learned sense of what images tend to look like, and uses it to guess noise. That is also why it can draw a lighthouse that never existed.
It paints on a small hidden canvas, not on pixels
The first big saving came from not working on pixels at all. In 2021, Robin Rombach and colleagues proposed running diffusion in the “latent space of powerful pretrained autoencoders”. That paper became Stable Diffusion. An autoencoder is a pair of networks: one squeezes an image into a much smaller grid of numbers, and the other expands such a grid back into a full image.
The diffusion model does all of its denoising on that small grid. Only at the very end does the decoder turn it into the pixels you see. The authors tested shrinking factors from 1 to 32 and found 4 and 8 gave the best balance of speed and detail. Stable Diffusion 3’s paper settled on 16 numbers per spot because more channels kept finer detail.
The small canvas also explains an old weakness. Each spot on the grid stands for an 8×8 patch of pixels. A fingertip or a letter only 16 pixels tall gets about two spots to describe it, which leaves little room for getting it exactly right. Newer models do better because of richer grids and much stronger text understanding: the Stable Diffusion 3 paper credits its design with gains in “typography” and human preference ratings.
Your prompt steers every step, and guidance sets how hard
So far the model only knows how to make an image less noisy. Your prompt decides what it should become. First, a text encoder turns the prompt into a list of numbers that capture its meaning, much like the embeddings behind search. Stable Diffusion 3 was trained with three text encoders at once. At every step, the denoiser looks at those numbers when it decides what noise to remove.
A setting called guidance controls how hard the prompt pulls. The standard technique, from Jonathan Ho and Tim Salimans in 2022, trains one model to denoise both with and without a prompt. At each step it makes both guesses and pushes further in the direction the prompt adds. The paper describes the result as a trade-off between quality and variety.
Our test matches that. At guidance 1, FLUX.1 dev gave a hazy scene with poles and bushes nobody asked for. The default of 3.5, the value Black Forest Labs uses in its own example code, gave the balanced picture. At 15 the sky went smooth and cloudless and the scene lost detail. FLUX.1 dev uses a distilled form of guidance that needs only one guess per step, and every guidance setting we tried cost the same $0.0037.
Fewer steps is where the speed and the price come from
Every step is one full run of a large network, so the step count drives both waiting time and cost. In our test, 1 to 8 steps of FLUX.1 dev each cost $0.0012 on Runware, 16 steps cost $0.0025, and 28 steps cost $0.0037. A 50-step run also cost $0.0037 and did not look better than 28 steps. It came out different: the sky clouded over and one outbuilding disappeared.
Researchers have cut the step count in three main ways. Better samplers came first: in 2020, DDIM produced good samples “10× to 50× faster” than DDPM by taking bigger, smarter jumps. Then came straighter paths. Rectified flow, used by Stable Diffusion 3 and FLUX, “connects data and noise in a straight line”, so the model can travel from static to picture in fewer, longer strides.
The third way is distillation: train a fast student model to copy a slow teacher’s finished result in one or a few jumps. Adversarial diffusion distillation reached the quality of SDXL in four steps in 2023. FLUX.1 schnell applies a version of it, and its model card promises high-quality images “in only 1 to 4 steps”, under an Apache 2.0 license.
The difference is stark. With one step, FLUX.1 dev made a smear, and FLUX.1 schnell made a finished lighthouse. Both models have 12 billion parameters; schnell was simply trained to skip the walk. The trade is some range and fine detail, which is why fast models make great drafts. Our post on what one AI image actually costs breaks down cloud image prices in more detail.
Video is the same trick stretched across time
AI video generators use the same recipe with one more dimension. Instead of a single noisy picture, the model starts from a noisy block of frames and cleans up all of them together. Denoising every frame at once is what keeps a face the same face from the first second to the last.
The hidden canvas matters even more here. Wan, an open video model family from Alibaba, describes itself as “built upon the mainstream diffusion transformer paradigm” in its technical report. Its autoencoder squeezes video “by 4×8×8 times”: four frames into one and each 8×8 patch into one spot, with 16 numbers per spot. That is about 48 times fewer numbers than the raw frames, as the table above shows.
Even so, video is heavy. The Wan report says sampling usually takes around 50 steps, each one over a block of frames. The smaller 1.3B model needs 8.19 GB of graphics memory, which fits many gaming PCs. This is why video tools now offer cheaper, faster preview modes before the full render; our guide to draft mode for AI video covers when to use them.
What this means when you press Generate
Once you see an image generator as a noise remover with a steering wheel, its settings stop being mysterious. Each one maps to a part of the process described above.
Three habits follow from it. Save the seed of any picture you like, so you can rerun it with a changed word and keep the composition; our guide to keeping a character consistent builds on this. Leave steps and guidance at the model’s defaults until you have a reason to move them, because more steps cost more and, as our 50-step run showed, often change the picture without improving it. And explore on a fast distilled model, then switch to the full model for the version you keep.
The lighthouse at the top of this page went from static to a finished scene in 28 passes, for less than half a cent. Nothing in it was copied from a photo, and none of it existed before the seed was picked. Every AI picture you see is a guess about what is hiding in the noise, and your prompt is the hint.
Test prompt, used for every FLUX image in this post: “A white lighthouse on a rocky headland at sunset, waves breaking on dark rocks below, a small keeper's cottage with a red roof beside it, a few gulls in a pink and orange sky, photographic”.
