Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
benchmarksMETR BlogJul 21, 2026
We propose a measure of an AI agent’s optimization ability with an “expenditure horizon.” We give an empirical illustration from the NanoGPT speedrun. One difficulty in measuring AI’s ability to accelerate AI R&D is accounting for token cost, experiment com...
Impact: LowConfidence: MediumNovelty: Material
benchmarkssource-feed-1bdee3f1-6eca-4e71-a9df-4f6430ba5051agentscodingopen-sourceresearch