Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?
Ankit Sonthalia, Haritz Puerto, Alexander Rubinstein, Martin Gubri, Seong Joon Oh
arXiv:2610.08775v1Today’s standout paper is Agent in a Bottle. It tackles a very practical question: if an LLM can solve a task well, can it turn that ability into something cheaper and reusable when the same task has to be done thousands or millions of times? The authors call this bottling. Instead of answering each request one by one with an expensive model call, an agent gets an entire unlabeled workload and a budget, then decides whether to write code, train a small model, or build another reusable artifact. To study this, they created BOTTLED, a benchmark with fixed compute, time, and API limits. The surprising result is that strong zero-shot performance does not reliably predict good bottling. But when it works, the savings are huge: in one case, a frontier model kept most of its quality at hundreds of times lower cost. That makes this paper important for anyone thinking about deploying AI at scale.
Also spotted that day
Previous daily papers