Tag

#local-llm

Every story tagged local-llm, newest first.

A GPU graphics card circuit board lying on a wooden workbench with tools and electronics visible in the blurred background

Which local LLMs fit in 8, 12, 16, 24 or 32GB of VRAM

At Q4_K_M and 8K context, 8GB comfortably holds Gemma 2 9B (5.8 GB) and 24GB holds Gemma 3 27B (17.5 GB). Full tables for every common VRAM tier, computed the same way our GPU checker computes them.

Ahmad J · Sep 6, 2026 · 8 min read

Wooden and black electronic device with lens housing on wooden floor in home office

Tutorial · software

How to run a local LLM on your own machine with Ollama

Install Ollama, pull a model, and chat with it offline in about ten minutes. No cloud account, no API key, and nothing leaves your machine: the model runs entirely on your own hardware.

Ahmad J · Aug 13, 2026 · 4 min read

Interior of server rack cabinet with black metal frame, cable management visible, and perforated center panel, shot from directly above

Article · aiDeep read

Local vs. Cloud AI Processing: The Real Trade-Offs

Where you run AI, on the device or in a datacenter, decides its latency, cost, privacy, and control. A precise breakdown of the real trade-offs, why memory (not compute) limits local models, and why most 2026 systems route between both.

Ahmad J · Aug 12, 2026 · 4 min read

Magnifying lamp illuminates a green circuit board on a workbench surrounded by electronics tools and components

Tutorial · ai

How to fine-tune a small language model with LoRA

Adapt a small open model to your task with LoRA and QLoRA: choose the right base, build a clean chat-format dataset, train lightweight adapters on a single GPU, read the loss honestly, and know when fine-tuning is the wrong tool entirely.

Ahmad J · Aug 12, 2026 · 4 min read

Two RAM modules on a wooden surface, with a larger white heatspreader unit on the left and a smaller circuit board module on the right

Article · aiDeep read

Memory Bandwidth, Not Compute, Limits Local LLM Speed

Why TFLOPS don't matter for inference: token generation is memory-bound, and the bottleneck is how fast weights reach your chip.

Ahmad J · Aug 11, 2026 · 5 min read

Which local LLMs fit in 8, 12, 16, 24 or 32GB of VRAM

How to run a local LLM on your own machine with Ollama

Local vs. Cloud AI Processing: The Real Trade-Offs

How to fine-tune a small language model with LoRA

Memory Bandwidth, Not Compute, Limits Local LLM Speed

More stories