LLM techniques ranked by measured effect on speed and quality

Techniques that change speed, ranked by measured effect

Each bar is the largest measured effect of one technique on generation speed or waiting time, as a percentage. A 2x speedup is written as +100% and 3x as +200%. The prompt caching bar is a shorter wait, and the thinking bar is a slowdown, placed by its size. The scale is logarithmic, from 1% to 1000%: each step along it is the same ratio, so a +900% bar is not nine times a +100% bar. Dashed lines are the tier edges at 5%, 30% and 100%. Each number comes from a different source, model and test, so the bars rank the size of the effect, not the techniques against each other.

Details and sources

Every technique with its tier, the measured effect, what the percentage measures, the baseline and the result, the kind of evidence and where the number comes from, one row per technique, ordered by tier.

Tier Technique Effect What the % measures Baseline to result Evidence Source
Loading data…

What this shows

  • Predicting several tokens at once is worth 2x to 3.6x. In speculative decoding a small draft model, or extra prediction heads, guesses several tokens that the large model then checks in one pass. The original method gives identical outputs.
  • Diffusion models are the outlier. The 10x figure is the developer's own comparison against other companies' models, so quality is not held equal.
  • Attention and memory engineering give 2x to 6x each. FlashAttention, paged KV-cache memory with continuous batching and latent attention all attack the same bottleneck: memory traffic, not arithmetic.
  • Quantization speedups depend on hardware. The 2022 result was 3.25x to 4.5x. A 2024 study on current GPUs found 1.08x to 3.09x, largest on the biggest model.
  • These gains stack only partly. They compete for the same headroom, and no source read here measures the full stack, so the percentages must not be multiplied together.

From the article What makes LLMs better, faster and cheaper: the techniques, ranked by measured effect. Data file: llm-techniques-2026-11-19.csv.