Local AI build choices ranked by measured effect
What you control when you pick a build of one model
One model name comes in many builds and can be run with many settings, and those choices are yours. Each bar is the largest measured effect of one choice, as a percentage. A plus is a gain and a minus is a cost or a reduction. Dashed lines are the tier edges at 5%, 30% and 100%. Each number comes from a different source and test, so the bars rank the size of the effect, not the choices against each other.
Details and sources
Every choice with its tier, options, what it changes, the measured effect and where the number comes from, one row per choice, ordered by tier.
| Tier | Choice | Options | What it changes | Effect | Source |
|---|---|---|---|---|---|
| Loading data… | |||||
How to use this
The choices that change the result most are the model itself (size tier, dense or mixture of experts), the bits, thinking and a draft model. How the build was made, the harness, caching and parallel requests are mid-sized. Which runtime you use matters least. No choice falls under 5%.
- Bits are the first choice. 4-bit is the usual sweet spot: about 70% less memory and more than twice the speed of 16-bit for a few percent of quality at most. 8-bit costs twice the memory of 4-bit for a gain that is hard to measure. Below 4 bits the size keeps falling but no source read here measures the quality loss, so it is not ranked.
- Prefer a QAT build when the developer offers one. It has the same size as any other 4-bit build and loses about half as much.
- If memory allows, a bigger model at 4-bit beats a smaller one at 16-bit. Size tier changes scores by tens of percent, bits by a few percent. This is LLMeter's reading of separate sources, not a single measurement.
- A mixture-of-experts variant buys speed, not memory. All its weights must be loaded, but only the active part computes each token.
- Thinking is a budget decision. It helps a capable model on hard tasks and can hurt a small one that runs out of tokens or time.
- Runtime, parallel requests and context length change speed and memory, not the answers. The same build scored the same in two runtimes.
From the article What makes LLMs better, faster and cheaper: the techniques, ranked by measured effect. Data file: llm-user-choices-2026-11-19.csv.