Skip to content

Free tool

What GPU Do I Need?

Pick the open LLM you want to run and we’ll show the minimum GPU or Mac that runs it comfortably, plus a ranked list of every card that fits, cheapest-accessible first.

Needs about 15.6 GB of VRAM. Runs on RTX 4060 Ti 16GB, tight. Comfortable from AMD RX 7900 XT (20 GB).. 17 of 20 cards listed will run it.

Needs ≈ 15.6 GB VRAM

Runs on RTX 4060 Ti 16GB (tight) · comfortable from AMD RX 7900 XT (20 GB).

  • Apple unified 16GB11 GB usable of 16 GB
    Too small
  • RTX 3060 12GB12 GB · budget
    Too small
  • RTX 4070 Ti (12 GB)12 GB · mid
    Too small
  • RTX 4060 Ti 16GB16 GB · budget (slow bus)
    Tight
  • RTX 4080 / Super (16 GB)16 GB · high
    Tight
  • RTX 5070 Ti (16 GB)16 GB · mid
    Tight
  • RTX 5080 (16 GB)16 GB · high
    Tight
  • Apple unified 24GB17 GB usable of 24 GB
    Tight
  • AMD RX 7900 XT (20 GB)20 GB · ROCm/Vulkan
    Runs well
  • AMD RX 7900 XTX (24 GB)24 GB · ROCm/Vulkan
    Runs well
  • RTX 3090 / Ti (24 GB)24 GB · used-value 24GB
    Runs well
  • RTX 4090 (24 GB)24 GB · flagship
    Runs well
  • Apple unified 36GB26 GB usable of 36 GB
    Runs well
  • RTX 5090 (32 GB)32 GB · flagship
    Runs well
  • Apple unified 48GB35 GB usable of 48 GB
    Runs well
  • Apple unified 64GB48 GB usable of 64 GB
    Runs well
  • Apple unified 128GB96 GB usable of 128 GB
    Runs well
  • RTX PRO 6000 Blackwell (96 GB)96 GB · workstation
    Runs well
  • Apple unified 256GB192 GB usable of 256 GB
    Runs well
  • Apple unified 512GB384 GB usable of 512 GB
    Runs well
See our GPU picks →What can my GPU run?

“Comfortable” leaves ~15% headroom over the estimated need (params × bytes-per-weight × a context factor), anchored to real GGUF sizes. Apple usable memory ≈ 72% of total. Bandwidth matters for speed: a 4060 Ti 16GB fits the same models as a 4080 but runs them slower. Verify current prices in the guide before buying.

Frequently asked

What GPU do I need to run Llama 3.3 70B locally?

At 4-bit (Q4_K_M) Llama 3.3 70B needs about 45.4 GB including a typical chat context, so no single consumer card runs it; you'll want two 24 GB cards (dual RTX 3090/4090) or an Apple machine with 64 GB+ unified memory (~48 GB usable, which the tool calls tight for this model). For a single card, drop to a 32B model (Qwen2.5 32B or DeepSeek-R1 32B) at 20.8 GB, which is tight on a 24 GB 4090.

What's the best value GPU for running local LLMs?

For the most VRAM per dollar, a used RTX 3090 (24 GB) is the enthusiast favorite; a 32B model at 4-bit is tight on it and two of them clear a 70B. On a budget, an 8B model comes to 5.2 GB, which is comfortable on an RTX 3060 12GB. Apple Silicon is excellent value if you already have a Mac with lots of unified memory, since the GPU shares the whole pool.

Is more VRAM or more speed more important?

VRAM decides whether a model runs at all; memory bandwidth decides how fast. Get enough VRAM for the model + context first, then prioritize bandwidth. A 4060 Ti 16GB fits the same models as a 4080 but has a much narrower memory bus, so it generates noticeably slower on larger models.

Can I run local LLMs on an AMD GPU or a Mac?

Yes. AMD RX 7900 XT/XTX (20–24 GB) run local LLMs via ROCm or Vulkan, though the software ecosystem is less mature than NVIDIA's CUDA. Apple Silicon runs them very well via MLX/llama.cpp, and its large unified memory lets even a laptop hold models that need multiple discrete GPUs. Just budget ~72% of total memory as usable.

How to size a GPU for an LLM

The single number that decides whether a model runs on your GPU is VRAM, the memory on the card. If the model plus its working data does not fit in VRAM, it either will not load or spills to system RAM and slows to a crawl.

The rough VRAM math

A model's weights take about (parameters times bytes per parameter) of memory. At 16-bit precision that is roughly 2 GB per billion parameters, so a 7B model needs around 14 GB just for weights. Quantizing to 4-bit roughly quarters that: the formats people actually use land near 0.55 GB per billion, so a 7B model is about 3.9 GB of weights, and this tool puts it at 4 to 5 GB once an ordinary chat context is counted.

Leave room for context

Weights are not the whole story. The KV cache, which holds the running context, grows with the context length and can add several gigabytes on long prompts. Size for the weights plus a comfortable margin, not the bare minimum.

Bandwidth sets the speed

Once a model fits, memory bandwidth rather than raw compute usually sets how fast it generates tokens. That is why cards with faster memory feel quicker for local inference even at a similar price.

Where this answer comes from

Last checked on Sep 10, 2026, and reviewed on a 45-day schedule. Figures come from the company that publishes them, never from another site’s summary. How we check this.

Related reading

Go deeper on how this works and what to pick.

Newsletter

Liked the tool? Get the signal.

One weekly email on the AI + hardware that actually matters, from the people who build these calculators.

Free · unsubscribe anytime · no spam.