A deep read: the full picture, with the receipts.
The short answer, at the quantization most people actually run (Q4_K_M) and a typical 8K context: 8GB comfortably holds Gemma 2 9B (5.8 GB), 12GB holds Qwen3 14B (9.6 GB), 24GB holds Gemma 4 31B (20.1 GB), and 32GB holds Qwen3 32B (21.3 GB).
And a warning that surprises people: 16GB buys you nothing over 12GB here. Both top out at Qwen3 14B (9.6 GB). That is not a rounding artifact: it is a hole in the model landscape. The popular dense open models cluster at 14B and then jump to 24B, and a 24B at Q4_K_M needs about 15.6 GB, which is tight on a 16GB card rather than comfortable. If you are choosing a card specifically to run a bigger model, 16GB is the tier to skip.
Everything below is computed the same way our local LLM GPU checker computes it, from the same table of models and quantization constants, so if you change the context length or the quantization there, the numbers move together instead of disagreeing with this page.
How the estimate works#
Three things decide whether a model fits, and only the first is the one people quote:
Parameters. A "7B" model has roughly 7 billion weights. That is a count, not a size.
Bytes per weight. Quantization decides how much space each weight takes. At Q4_K_M a weight costs about 0.55 bytes, so a 7B model is roughly 7 x 0.55 = 3.9 GB of weights.
Context. The KV cache grows with how much context you keep, and it comes out of the same memory. We apply a multiplier to the weight size:
So the estimate is parameters x bytes-per-weight x context multiplier. A model is comfortable when it needs 85% or less of your VRAM, and tight up to 100%: the last 15% is where the operating system, the display, and fragmentation live. Tight will often load and then fail on a long prompt.
What fits in 8GB#
The tier most people have, and the one that most often disappoints. Gemma 2 9B (5.8 GB) is the comfortable ceiling.
What fits in 12GB#
What fits in 16GB#
The disappointing tier. The comfortable ceiling is the same 14B-class model as 12GB: the extra 4GB only moves a 24B from impossible to tight, which is not the same as usable.
What fits in 24GB#
The tier a 3090/4090-class card gives you, and the one where most people stop needing to think about quantization.
What fits in 32GB#
What this does not cover#
These are dense models. Mixture-of-Experts models, the kind that activate a fraction of their weights per token, behave differently: they still need memory to hold every parameter, but run far faster than a dense model of the same total size. A dense 70B and a large MoE are not comparable on this table, and the table does not try.
Estimates, not measurements. Bytes-per-weight are the standard GGUF K-quant values and the context multipliers are KV-cache-aware rules of thumb. Different runtimes, batch sizes and cache settings will move the real number. Treat "tight" as "test it before you rely on it".
Unified memory is a different question. On an Apple or Strix Halo machine the model shares one pool with everything else, and the useful ceiling is not the same as the total. That has its own calculator: unified memory.
Frequently asked questions
Is 8GB of VRAM enough to run a local LLM in 2026?
Yes, for small models. At Q4KM and 8K context, 8GB comfortably holds Gemma 2 9B (5.8 GB). What 8GB will not do is run a 14B-class model without dropping to a harsher quantization or a shorter context, and it will not run anything 24B or larger at all.
Which is better, a bigger model or a better quantization?
Usually the bigger model. Dropping from Q8 to Q4KM roughly halves the memory for a modest quality cost, which is what makes a larger model fit in the first place. Going below Q4 buys much less and costs much more: the dataset marks Q3KM as "smallest usable" for a reason.
Why does my model need more memory than this table says?
Longer context is the usual answer: the multiplier here is for 8K, and 128K costs 1.7x the weight size rather than 1.18x. After that it is the runtime: some allocate the full KV cache up front, some grow it.
Does this apply to a Mac?
Partly. The model sizes are the same, but Apple's unified memory is shared with the OS and everything else running, so the number to compare against is not the machine's total. Use the unified memory calculator for that, and see our guide to the best Macs for local AI.
What if I want to fine-tune rather than just run a model?
Fine-tuning needs several times the memory of inference, because gradients and optimiser state have to live alongside the weights. That has a separate calculator: fine-tuning VRAM.
Sources
- Google — Gemma 4 model overview and inference memory requirementsai.google.dev
- llama.cpp — quantize tool documentationgithub.com
- Hugging Face — GGUF format on the Hubhuggingface.co
Discussion