Intermediate · Systems

Hill Climbing (Agent Improvement)

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. The ongoing work of incrementally improving an agent and its harness once both are already running in production.

Technical Definition

The ongoing work of incrementally improving an agent and its harness once both are already running in production.

How it works

Borrowed from the classic optimization term, hill climbing in the agentic sense means treating a deployed agent as a system you tune forever: each week you find the biggest failure class in production traces, fix it in the harness — a better tool description, a tighter permission, a new eval — and ship the improvement. GitHub's own engineering reports framed their agent work this way: the model barely changed while measured task success climbed, because the harness kept climbing the hill. The practice needs continuous evals to know which direction is uphill, and discipline to fix the system rather than patching individual outputs.

Related Concepts

• Continuous Evals — Running evaluation suites continuously in production — on live traffic samples and every model or prompt change — instead of as a one-time pre-launch gate.
• Agent Harness — Everything surrounding a model that makes it useful in real workflows — the tools, permissions, memory, context, and orchestration around the model itself.
• Loop Engineering — The practice of designing repeatable systems around agents, instead of manually prompting them for one task at a time.