Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT

benchmarksMETR BlogJul 21, 2026

We propose a measure of an AI agent’s optimization ability with an “expenditure horizon.” We give an empirical illustration from the NanoGPT speedrun. One difficulty in measuring AI’s ability to accelerate AI R&D is accounting for token cost, experiment com...

Impact: LowConfidence: MediumNovelty: Material
benchmarkssource-feed-1bdee3f1-6eca-4e71-a9df-4f6430ba5051agentscodingopen-sourceresearch

See the whole story

Read the original on METR Blog

Sens.ai aggregates and assesses published reporting. The assessment above is machine generated; the original source is authoritative.