Daily AI Paper

Earth observation is full of important tasks, but the way we build models for them is still mostly artisanal. When a new sensor appears, or a new region needs mapping, or a disaster response team suddenly needs a model in a few days, the usual process is that experts choose a backbone, design a training recipe, tune hyperparameters, and iterate against a benchmark. That works, but it is slow, it depends on scarce expertise, and it does not scale to the long tail of EO problems. The paper argues that this whole loop can be handed to an agent. The key idea is simple but powerful: if a task can be verified automatically, then we can separate it into a harness and a verifier. The harness defines what the agent is allowed to use, such as labeled data, unlabeled data, pretrained checkpoints, software, and compute. The verifier defines what success means, by scoring the agent’s model on data the agent never sees. In other words, instead of asking humans to hand-design the model, we ask them to define the task environment and the scoring rule. Then a general-purpose coding agent can search for the model on its own. This matters especially in Earth observation because EO already has many natural ways to verify performance. The most familiar is the standard benchmark: train on one split, score on a held-out split. But EO also has independent sensors, in-situ measurements, physical models, and repeated observations of the same place over time. Those all create ways to check whether a prediction is right without exposing the answer to the agent. So the paper’s broader claim is that many EO settings that we usually treat as different research problems are really just different choices of harness and verifier. Few-label learning, label-budget search, and domain adaptation all fit into that framework. The authors call this setup RSI4EO, for recursive self-improvement for Earth observation. The recursive part is important. The agent does not just try one model and stop. It writes training code, trains a model, asks the verifier for a score, studies the result, and then tries a better pipeline. It can even train on its own outputs. The paper describes a setting with only six labeled images where the agent runs multiple generations of self-training, each one taught by the best pipeline found so far, and eventually automates the whole loop with its own scripts. The point is that the model improves recursively, but the recursion itself is designed by the agent, not by humans. To understand the landscape, it helps to compare this to prior work. In machine learning engineering, there has been a long history of automating parts of the pipeline: hyperparameter search, neural architecture search, and self-configuring systems like nnU-Net. But those methods still search within spaces that people define. More recently, coding agents built on large language models can write and debug arbitrary training code, which widens the search space a lot. In parallel, self-improvement systems in other domains have shown that when a strong verifier exists, progress can accelerate quickly. The paper’s contribution is to bring that verifier-driven, agentic loop into Earth observation as a general framework. Methodologically, the task is defined as a pair: harness plus verifier. The agent receives the harness, which includes the allowed data and compute, and it produces a pipeline. A pipeline here means a self-contained directory of code and trained weights that maps input imagery to predictions. The verifier runs that pipeline on held-out validation data and returns a score. The agent can submit many pipelines during its budget, see only the scores, and decide what to try next. At the end, the best pipeline is evaluated once on a hidden test set. A very important design choice is that the verifier is sealed. The agent never sees the validation images or labels. It only gets the score back. The verifier runs the whole pipeline exactly as final evaluation will, so it checks not just the core model but also preprocessing, ensembling, and test-time augmentation. That means the agent is optimizing the actual end-to-end system, not just a proxy. It also means the setup is robust against accidental leakage and more realistic for deployment. The paper emphasizes that this is especially natural in EO, because the field already has many sources of verifiable signal. The harness is what makes the task flexible. By changing the amount of labeled data, the amount of unlabeled data, the available checkpoints, or the compute budget, the same framework can represent very different research problems. If you want few-shot learning, you restrict labels. If you want domain adaptation, you place the verifier on a target domain. If you want to know the minimum annotation budget needed to reach a target score, that becomes another harness choice. So the framework does not just automate model building; it also unifies several subfields under one task definition. The experiments cover four EO benchmarks with published results, spanning optical, SAR, optical-SAR, and multispectral time-series imagery. The tasks include land-cover mapping, building damage assessment from optical and SAR imagery, SAR ship instance segmentation, and crop-type mapping from Sentinel-2 time series. The paper also mentions a surface-material classifier from hyperspectral pixels as an additional demonstration. The agent runs offline in containerized environments and is given a fixed compute budget. The reported main results are obtained within about 12 hours on four GB200 GPUs. The results are striking. On all four benchmarks, agent-built models surpass the published state of the art. That is already notable, but the more interesting result is what happens when labels are scarce. With only about 0.1 to 1.4 percent of the training labels, the models keep most of their full-data accuracy. The paper reports that across these low-label settings, they retain about 69 to 84 percent of the accuracy they would get with more labels. On OpenEarthMap specifically, six labeled tiles are enough to reach 60.0 test mIoU, and 300 tiles selected without labels are enough to beat every published method trained on all labels. That is a very strong claim: a tiny labeled subset plus the agent’s search procedure can outperform fully supervised published baselines. The paper also notes that two agents, which build their models in very different ways, both beat the published state of the art on OpenEarthMap. That matters because it suggests the gains are not a fluke of one particular prompt or one brittle search strategy. The framework seems to be robust enough that different agent behaviors can still find strong solutions, as long as the verifier is informative and the harness is well designed. In the few-label BRIGHT setting, the paper frames a realistic disaster-response use case: one labeled image pair per event could be enough for the agent to produce a useful damage model within a day. What do these results actually show? First, they show that a coding agent can take over a large part of EO model development, not just assist with it. Second, they show that recursive self-improvement can work in a scientific vision task when the task is made verifiable. Third, they suggest that in EO, the main bottleneck may shift from model engineering to task specification: deciding what the verifier should measure and what the harness should include. In that sense, the paper is not just about better models; it is about changing who does the modeling. There are still limitations and open questions. The framework depends on having a good verifier. If the verifier does not faithfully capture the real task, the agent may optimize the wrong thing. That is a general issue in any automatic scoring setup. The paper also uses labeled held-out sets for its verifiers in the experiments, even though it argues that EO offers other verification sources too. So one next step is to test the same approach with richer verifiers based on physical consistency, multi-sensor agreement, or future observations. Another question is how well this scales beyond the benchmarks studied here, especially to tasks where labels are noisier, the metric is less direct, or the domain shift is more severe. There is also the practical question of compute. These results come from agent search over many candidate pipelines, so the approach trades human engineering time for agent and GPU time. That may still be a great trade in many settings, but it is not free. And while the paper argues that the agent can work offline in sealed containers, real deployments will need careful attention to security, reproducibility, and the fidelity of the harness. Still, the central message is clear. In Earth observation, we may be moving from a world where humans build each model by hand to a world where humans define the task, the verifier checks success, and the agent builds and improves the model recursively. If that holds up, then EO models will no longer improve only when experts have time to engineer them. They will improve as the agents themselves improve.