Daily AI Paper

Long-horizon agents are exciting because they point toward something much bigger than today’s short coding tasks. The idea is to build systems that can work autonomously for hours or even days on economically meaningful problems: scientific discovery, machine learning research, security work, infrastructure operations, and other tasks where progress comes in many small steps rather than one quick answer. That matters not just for productivity, but because long-horizon competence is one of the ingredients people discuss in automated discovery and recursive self-improvement. If a model can reliably keep track of goals, recover from mistakes, and make steady progress over a long workflow, that is a very different capability from solving a short benchmark prompt. The problem, as this paper frames it, is that open-source agent training has mostly scaled the wrong kind of data. We have many corpora for short and mid-horizon tasks, like terminal sessions or pull-request-sized fixes, but those tasks usually end with a simple pass or fail signal. By contrast, the truly long-horizon benchmarks that resemble real work are mostly human-authored evaluation sets. They are small, expensive to create, and not designed as training corpora. So the field has ended up with a gap: lots of large-scale training data for short tasks, and only tiny evaluation sets for long tasks. That makes it hard to train agents for the very behaviors we care about most. There is a second, closely related issue: reward sparsity. In long-horizon tasks, the agent often gets feedback only at the end. If the task takes dozens of steps and the model is weak early in training, most rollouts fail completely, which means the training signal can vanish. The paper points out that this is especially painful for group-relative policy optimization, or GRPO, where rewards are normalized within a group of sampled rollouts. If every rollout in a group fails, then there is no useful advantage signal at all, and that group contributes nothing to learning. In long-horizon settings, that is exactly the kind of group you get most often at the beginning. The core idea in Tev is to solve both the data problem and the reward problem together. The first part is a new dataset called Tev-Gym, which contains 1,170 verified long-horizon environments across seven domains: software engineering, data engineering, AI and computer science research, mathematics and operations research, security and forensics, infrastructure administration, and quantum computing. The second part is a training recipe called progress-potential reward shaping, or PPRS, which turns intermediate progress into a dense per-step reward. To understand the dataset, it helps to understand the paper’s synthesis strategy, which they call latent-world-first. Instead of asking a model to invent a task, invent the inputs, and invent the checker all at once, they start from a hidden but complete and correct specification of the task domain. They call that hidden specification the latent world. Think of it as the full answer key plus the full underlying state of the problem. From there, they build the task backward. Concretely, each environment is generated from a seeded program. First, the generator creates the latent world: the complete state of the domain, such as all the records in a warehouse, the shards in a distributed checkpoint, or the files in a structured research workflow. They validate that world against invariants so that it is internally consistent. Then they compile the workflow into a directed acyclic graph, or DAG, of stages with explicit dependencies and some allowed parallelism. That DAG defines the task horizon: how many dependent stages there are and how much work must be done before the final answer is correct. Next, they render the public input files from the latent world. These can be CSVs, TSVs, JSON, SQLite databases, safetensors shards, stripped binaries, or infrastructure manifests. Importantly, the rendering is designed to round-trip exactly, meaning the latent world can be recovered from the rendered files without ambiguity. After that, they apply a controlled corruption plan: schema drift, missing or duplicate records, scrambled indices, stale caches, and similar defects. Those corrupted files are what the agent actually sees. But because the corruption is declared and invertible, the hidden world still fully determines the task. That means the correct answer is computable from the agent’s view, and the verifier can check the output against the known ground truth exactly. This setup matters because it avoids a common failure mode in synthetic benchmark generation, where the prompt, the files, and the checker are each generated separately and may not truly agree. Here, everything is anchored to the same latent world: the prompt, the corrupted input tree, the strict verifier, and the progress metric. The authors argue that this prevents “artifact drift,” where a task looks valid on the surface but is not actually coherent end to end. The paper also emphasizes that the environments are not just correct, but calibrated and diverse. Each environment is labeled along many axes, including subdomain, scale, workflow stage, tool capability, artifact role, relation type, and corruption class. Two of the main controls are scale tier and workflow structure. Increasing the scale tier grows the world and lengthens the amount of reading, joining, and writing required. Changing the DAG and the number of corruption classes changes the dependency depth and the order in which the agent must repair things. So the framework gives a controllable difficulty ladder, which is important if you want to train agents progressively rather than throwing them into a fixed set of hard tasks. Before an environment is admitted, it must pass a set of qualification gates. These are meant to filter out exploitable, invalid, or poorly calibrated tasks. Among other things, they check that gaming probes fail, that the progress score behaves properly on partially correct states, and that the scorer, verifier, and reference solvers are independent. The result is a dataset designed not only to be hard, but to be trustworthy. The second major contribution is the dense reward. The latent world that can grade the final submission can also grade intermediate states. So instead of waiting until the end, Tev defines a progress potential, a score between zero and one that measures how close the agent’s current workspace is to the known-correct intermediate states. As the agent completes more of the workflow, the potential rises; if it regresses, the potential falls. This gives the training process a shaped reward at every step. In other words, the model gets feedback for partial progress, not just for final success. That is the key training insight. Under terminal-only reward, a long-horizon task gives almost no signal until the very end, and most early rollouts are useless. With progress-potential shaping, even an incomplete rollout can still show whether the agent moved the state forward. That creates within-group variance from the first turn, which is exactly what group-relative methods need to learn. The paper trains two main models with this recipe: Tev-9B and Tev-27B. Tev-9B is trained on the unit tier, while Tev-27B is trained on the full Tev-Gym corpus. They also apply the same recipe to a larger base model, producing Tev-320B. The evaluation focuses heavily on Agents’ Last Exam, especially the computing and math sciences split, which they call ALE-CM, along with Terminal-Bench subsets. The headline result is that Tev-27B makes large gains on ALE-CM. The paper says Tev-9B improves Pass@5 by 83% over Qwen3.5-9B, and Tev-27B improves Pass@5 by 65% over Qwen3.8-27B to 0.28, matching GPT-6 Astra and Claude Opus 5 on that benchmark. That is a strong claim because it suggests the training recipe is not just making the model better at the synthetic environments, but at a real long-horizon exam-style benchmark. Even more interestingly, the model generalizes beyond the data it was trained on. Although training is only on ALE-related data, Tev-27B transfers to Terminal-Bench 3.0 and 4.0, improving Pass@1 by 56% and 150% respectively, and surpassing GLM 5.2 and Gemini 3.7 Flash. On the mid-horizon Terminal-Bench 2.1, Tev-9B improves zero-shot Pass@1 by 28%, and Tev-27B reaches 0.794 Pass@1, which the paper describes as Claude Opus 4.8-level performance. The larger Tev-320B, obtained by applying the same recipe to GLM-5.3-Flash, also improves TB 3.0 and 4.0 and trails only a small set of top systems on TB 4.0. What do these results actually show? First, that long-horizon training data can be generated in a way that is both verifiable and scalable enough to support model training. Second, that dense progress-based shaping can make hard long-horizon tasks trainable when terminal reward alone would be too sparse. Third, that skills learned on these ALE-related long-horizon environments transfer to other terminal and agentic benchmarks, which is a sign the model is learning more general workflow competence rather than memorizing a narrow task format. At the same time, there are important limits to keep in mind. The paper is explicit that the training data focuses on ALE-CM due to resource constraints, even though the synthesis approach is broader. So the breadth of domains in Tev-Gym is promising, but the training and evaluation emphasis is still concentrated in a particular slice of long-horizon work. Also, while the latent-world approach gives exact verification and dense progress signals, it depends on tasks where a hidden complete specification can be constructed and where intermediate states can be meaningfully scored. That may not be possible for every real-world problem, especially open-ended research tasks with ambiguous success criteria. Another thing to watch is whether the progress signal remains aligned with true task quality as tasks become more open-ended. Dense shaping is powerful, but it can also create incentives to optimize the metric rather than the real goal if the metric is imperfect. The paper addresses this with qualification gates and hidden checks, but this remains a general concern for any shaped-reward system. Overall, the contribution here is a fairly coherent recipe: build long-horizon environments backward from a known-correct latent world, use that world to verify both final answers and intermediate progress, and then train with a dense shaped reward that makes long workflows learnable. The paper’s main claim is that this combination closes a major gap in agent training: not just more data, but the right kind of data and the right kind of reward for tasks where progress is incremental, sparse, and hard to measure. If that holds up more broadly, it is a meaningful step toward agents that can actually sustain effort over long, complex workflows.