How much RAM and VRAM you actually need to run local AI

Here is the question behind almost every “can I run local AI” search: you have a laptop with some amount of memory in it, you have a model you want to run, and you want to know if the first is big enough for the second. That is the entire problem. It is a subtraction, and once you can do it in your head you never have to guess again.

The trouble is that most answers online are either a hand-wavy “you need a good GPU” or a wrong oversimplification. You’ll see the rule “take the parameter count and double it” passed around as gospel. It ignores the two things that actually move the number: how much you shrink the model, and how much text you feed it. Get those wrong and you either buy hardware you don’t need or download a model that won’t load.

The memory a local model needs is a short formula: the model’s weights, plus a tax for your context window, plus a slice for the system. Add them up and compare against your memory. That comparison decides everything. This post builds that formula in plain English, hands you a table you can read your own machine off of, and explains the one trap that makes a Mac and a PC count their memory differently.

Memory is the wall, not speed

First, a correction to a common instinct. People assume the bottleneck for running AI at home is raw speed: a faster chip, more cores. Speed decides how quickly words come out. Memory decides whether the model runs at all. A model that doesn’t fit in memory does not run slowly. It does not run.

The reason is simple. To generate each word, the machine has to have every one of the model’s weights available to read. Those weights live in memory. If the model is 40 GB and your usable memory is 16 GB, there is nowhere to put it. This is why the memory number, not the gigahertz number, is the gate. Everything below is about calculating that memory number and reading it off your machine.

A model’s size is params times bytes

A language model is a giant list of numbers called weights. An “8B” model has 8 billion of them. The memory those weights take is nothing more than how many there are multiplied by how many bytes you spend storing each one.

That second number is set by precision. Models are trained at 16-bit precision, which is 2 bytes per weight. So the raw, full-quality 8B model is 8 billion times 2 bytes, or about 16 GB. A 70B model at full precision is 140 GB. That is the source of the “double the parameters” rule, and for the versions labs first release, it is roughly right.

Almost nobody runs those versions at home. Instead they run a quantized copy: the same model with each weight rounded to fewer bits. Drop to 8-bit and you halve the size to 1 byte per weight. Drop to 4-bit, the popular default, and each weight costs about half a byte. The 16 GB model becomes a 5 GB one, and in ordinary use you cannot tell them apart. The precision you pick, not the parameter count alone, sets the bill.

About 0.6 GB per billion at 4-bit

Here is the rule of thumb worth memorizing. At 4-bit, the format most people run, a model’s download is close to 0.6 GB per billion parameters. An 8B model is a 4.9 GB file on Ollama; a 70B model is 43 GB. Both land right on that 0.6 line, because the common 4-bit format spends a little over 4 bits per weight, not exactly four.

Why 0.6 and not the 0.5 the raw math suggests? The good 4-bit builds keep a few weights at higher precision where it matters, so the real file runs slightly heavier than the theoretical minimum. Rounding the rule up to 0.6 gets you an honest number instead of a hopeful one. At 8-bit, use about 1 GB per billion; at full 16-bit, about 2 GB per billion.

One wrinkle for the newer models. A “mixture of experts” model like some of the 120B-class releases stores all its weights in memory but only uses a fraction for each word. That means it costs memory like its full size (a 120B MoE still needs roughly 65 GB at 4-bit) while running at the speed of a much smaller one. For the memory question, count the total parameters, not the active ones.

Your context window is a second bill

The weights are only the first line of the bill. The second is the part the “double the parameters” crowd forgets entirely: the model also has to remember the conversation so far, and that memory grows with every word. It is called the KV cache, and for long inputs it can cost more than the model itself.

The size depends on the model’s internal shape, but the behavior is the thing to remember: the cache grows in a straight line with how much text is in play. Double the context, double this cost. At the short prompts of everyday chat, it is a rounding error. At the giant context windows the marketing brags about, it is enormous.

Look at what happens to the 8B model. At a normal chat length of a few thousand tokens, its cache is well under a gigabyte, easy to ignore. Fill its full 128K window and the cache balloons to 16 GB, more than three times the size of the 5 GB model. This is the quiet reason a model that loaded fine yesterday runs out of memory when you paste in a long document today.

It is also why the advertised context window and the usable one diverge, a gap worth understanding on its own (your million-token window is lying to you). Two practical levers help: run a shorter context if you don’t need the whole window, and use an 8-bit KV cache, which halves every number in that figure for a barely perceptible quality cost.

On a Mac, RAM is VRAM minus a quarter

Now the part that trips everyone up: your machine’s memory number and its usable memory number are not the same, and how they differ depends on what kind of computer you have.

Apple Silicon Macs use unified memory, one shared pool the processor and the graphics chip both draw from. There is no separate “graphics memory.” Your RAM is your video memory. That is a gift: a 64 GB MacBook can devote most of that 64 GB to a model, something no consumer graphics card comes close to.

The catch is a reservation. macOS caps how much of the pool the graphics chip can claim, through a Metal setting called recommendedMaxWorkingSetSize. The practical numbers: about 75% of RAM is available for the model on machines with 64 GB or more, and closer to two-thirds on smaller ones, with the rest held back for the operating system. So a 16 GB Mac gives you roughly 12 GB to work with, a 32 GB Mac about 22 GB, a 64 GB Mac about 48 GB, and a 128 GB Mac about 96 GB.

Two consequences follow. First, a 32 GB Mac behaves like a 24 GB graphics card for this purpose, not a 32 GB one, because the OS takes its cut off the top. Second, power users can raise the cap. The reservation is a soft limit set by a kernel parameter, and a one-line sudo sysctl iogpu.wired_limit_mb command can push it higher on a machine with memory to spare. Leave the OS a few gigabytes and you can reclaim most of that reserved quarter.

On a PC, only the GPU number counts

A Windows or Linux desktop with a discrete graphics card works the opposite way, and it is stricter. The card has its own dedicated memory, called VRAM, and that is the only pool that runs a model at full speed. Your 64 GB of system RAM does not help. A model has to fit in the VRAM on the card, and the number that matters is the one printed on the box: 8 GB, 12 GB, 16 GB, 24 GB.

You canspill the overflow into system RAM. Tools will let a model straddle both. But the moment part of the model lives outside VRAM, speed collapses, because the graphics chip has to reach across a slow bus for those weights on every single word. A model that half-fits your card doesn’t half-run. It grinds. For a PC, the honest rule is: pick a model that fits inside VRAM, with headroom for the context tax on top.

This is why, on the PC side, VRAM is the single spec that matters more than any other, and why an older 24 GB card can beat a newer 12 GB one for AI even if the newer card is faster at games. A 24 GB card such as an RTX 3090 or 4090 holds a 32B model at 4-bit; a 12 GB card tops out around an 8B. The generational speed bump is nice. The extra memory is what lets you run the bigger model at all.

What your machine actually runs

Put it together and you can read your own machine off a single table. Work out your usable memory first (about 75% of RAM on a Mac, the VRAM figure on a PC), then find your row.

The pattern is clean once the two-column trap is behind you. Eight gigabytes of usable memory runs a genuinely useful small model. Sixteen is the sweet spot for the 7-8B models most people should start with. Twenty-four opens up the mid-sized 14B tier and, at a squeeze, the 32B one. Getting to a 70B model means 48 GB or more of usable memory, which in practice means a high-memory Mac or a multi-card PC build. For most readers, the honest ceiling is a 14B or 32B model, and that is already very good.

If you are shopping rather than checking, the buying lesson falls straight out of the formula: buy memory, not clock speed. On a Mac, configure more unified RAM than you think you need, because it is soldered and you cannot add it later. On a PC, spend on the card with the most VRAM you can afford before you spend on anything else.

The subtraction to remember

Strip it to a sentence. Take your model’s parameter count, multiply by 0.6 for a 4-bit copy, add a gigabyte or two for context and system, and check it against your usable memory: 75% of RAM on a Mac, the VRAM number on a PC. If your memory is bigger, it runs. If it isn’t, drop to a smaller model or a heavier quant until it fits.

That is the whole discipline. Memory is the wall, weights and context are the two things stacked against it, and the quant level is the dial you turn to make them fit. With that, you can look at any model page and any spec sheet and know, before you download a single byte, whether the two will meet. When they do, the payoff is a capable model running entirely on your own machine, with no bill and no data leaving your desk.