Tag

#inference

Every story tagged inference, newest first.

Two RAM modules with gold connectors lying on a gray surface, the larger one on the left and a smaller one on the right

AWS's new G7 instances won its own benchmark with half the GPUs

AWS benchmarked its Blackwell-based G7 instances against G5, G6 and G6e on 30B mixture-of-experts models. The two-GPU box beat the four-GPU boxes, the cheapest configuration and the fastest one turned out to be different machines, and the same model cost 2.5 times more per token on retrieval traffic than on chat.

Ahmad J · Sep 8, 2026 · 6 min read

A small square microchip sits on a concrete ledge in a blurred modern interior with glass partitions and red lighting

Guide · aiDeep read

Choosing the right model size for your task

Bigger is not automatically better. A decision framework for matching model size to the job, the latency budget, and the hardware you actually have.

Ahmad J · Sep 4, 2026 · 4 min read

Five small robotic arms with orange joints mounted on a wooden platform in a gallery setting

News · ai

Gemini can now skim a video instead of watching every frame

Google replaced fixed-rate video ingest with an agentic loop that fetches only the moments it needs, and reports up to 88 percent fewer tokens. Every headline figure is an upper bound, the gains concentrate on long video, and the cost saving is just the token saving passed through.

Ahmad J · Sep 2, 2026 · 4 min read

A small square computer chip sits on a gray cube displayed in a data center with black server racks and fluorescent ceiling lights

News · aiQuick

Small Models Are Quietly Taking Over the Easy Work

Most production AI tasks are routine, and a new class of small models handles them at a fraction of the cost. The frontier models are becoming the exception, not the default.

Ahmad J · Sep 2, 2026 · 3 min read

A white circuit board with black components mounted on concrete ground against a gray wall, photographed at an angle in shallow focus

Article · ai

Small AI Models Are Quietly Winning in Production

Frontier models get the headlines, but inside real companies smaller, cheaper, faster models do the actual work. Here's how they win, where they don't, and what it costs to ignore them.

Ahmad J · Aug 14, 2026 · 4 min read

AWS's new G7 instances won its own benchmark with half the GPUs

Choosing the right model size for your task

Gemini can now skim a video instead of watching every frame

Small Models Are Quietly Taking Over the Easy Work

Small AI Models Are Quietly Winning in Production

More stories