What GPU Do I Need?

At 4-bit (Q4_K_M) Llama 3.3 70B needs about 45.4 GB including a typical chat context, so no single consumer card runs it; you'll want two 24 GB cards (dual RTX 3090/4090) or an Apple machine with 64 GB+ unified memory (~48 GB usable, which the tool calls tight for this model). For a single card, drop to a 32B model (Qwen2.5 32B or DeepSeek-R1 32B) at 20.8 GB, which is tight on a 24 GB 4090.

This tool holds no prices, so it can't rank cards by value; check live listings for that. What it can tell you is what each amount of memory runs. A 24 GB card such as an RTX 3090 or 4090 holds a 32B model at 4-bit, which the tool calls tight, and two of them hold a 70B, which the tool calls tight. On a smaller card, an 8B model comes to 5.2 GB, which is comfortable on an RTX 3060 12GB. If you already own a Mac with lots of unified memory, try it before buying a card, since its GPU shares the whole pool.

Add up what each one needs, because every model you keep loaded holds its own weights and context cache in memory at once. At 4-bit with a typical chat context, an 8B model (5.2 GB) plus a 32B model (20.8 GB) comes to about 26 GB. In this tool's list the smallest option that holds both is Apple unified 36GB (about 26 GB of it usable by the GPU), which the tool calls tight, and RTX 5090 (32 GB) holds them comfortably. With two cards you can put one model on each instead. If the models only need to take turns, loading one at a time needs just the largest one's memory, at the cost of a reload each time you switch.

VRAM decides whether a model runs at all; memory bandwidth decides how fast. Get enough VRAM for the model + context first, then prioritize bandwidth. A 4060 Ti 16GB fits the same models as a 4080 but has a much narrower memory bus, so it generates noticeably slower on larger models.

Yes. AMD RX 7900 XT/XTX (20–24 GB) run local LLMs via ROCm or Vulkan, though the software ecosystem is less mature than NVIDIA's CUDA. Apple Silicon runs them very well via MLX/llama.cpp, and its large unified memory lets even a laptop hold models that need multiple discrete GPUs. Just budget ~72% of total memory as usable.