Running every intermediate branching decision through a twenty-seven-billion parameter model is one of the most wasteful patterns in modern agent architectures.
Most agent steps do not require deep contextual synthesis or prose generation. They simply require choosing between discrete paths, such as deciding which tool to call or verifying whether an output matches expected invariants. Forcing a large model to handle these binary gates consumes substantial time and memory.
Jeff demonstrates that pairing a tiny zero-point-eight billion parameter base model with task-specific LoRA adapters can offload these routing decisions entirely. In benchmarks running alongside Qwen 3.8-27B across more than twelve hundred coding tasks, delegating decision steps to the smaller model reduced decision time from 8.1 seconds down to 0.25 seconds. That is a thirty-five times speedup while maintaining an equivalent overall task completion rate of roughly sixty-two percent.
The small model acts as a calibrated fast path, passing execution back to the large model only when its confidence falls below a configured threshold. It fits easily in memory alongside the primary model with only a two-gigabyte footprint.
Fast agents do not need larger weights; they need better tiering between fast decisions and deep reasoning.
