Archived
2026-09-23SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving
Jennifer Williams, Dave Farris, Jeff Farris, Jiantao Jiao
arXiv:2609.26777v1Today’s paper introduces SWE-Serve, a benchmark for a problem the field has struggled to measure well: can AI agents actually ship production inference features, not just hack together code that passes local tests? The authors build 53 repository-grounded tasks from real changes in the SGLang serving stack, covering model support, runtime behavior, APIs, and performance constraints. Each task is evaluated with hidden functional and regression tests, and where possible, end-to-end serving checks and calibrated performance gates. That matters because production inference work is full of cross-cutting details that ordinary software benchmarks miss. In their experiments, the best of 11 models and 31 effort settings reaches 75 percent pass at one, but the benchmark also exposes a big gap between “looks correct” and “works in production.” In particular, end-to-end tests reject many patches that would otherwise pass. SWE-Serve gives the community a much sharper way to track whether coding agents are becoming genuinely deployment-ready.
Also spotted that day
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMsA Decentralized Partially Observable Team Decision Methodology with Delayed Information SharingAgensh: Scaling Organizational Intelligence to 1,024 AgentsSpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party DialogueCliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding AgentsA2M: Trace-Optimized Agent Hijacking in the MCP EcosystemGrow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist AgentsType-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to ItFleXray: Universal Clinical X-ray SegmentationEquivSVA: A Formally Verified Dataset of Behavioral Assertions Across Equivalent RTL ImplementationsMetrics Failure in LLM-Based Code Vulnerability Repair: An Empirical Study and a Change-Aware ScreenAutomatic depth-based local center clustering via $β$-integrated local depth and adaptive groupingDiffusion-Induced Spatial Attention Overlapping Community DetectionDoes AI Save Time on Product Design? A Randomized Controlled Experiment of AI Prompt-to-Design Workflows
Previous daily papers
2026-10-15Latent Compaction with Test-Time Training2026-10-14Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards2026-10-14The Last Remote Sensing Model Built by Humans? Casting Any Earth Observation Task as Recursive Self-Improvement2026-10-14Hybrid Reasoning Agents2026-10-13AI Models Can Provably Hide Arbitrary Capabilities2026-10-13RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics2026-10-13Designing Protein Structures from Natural Language2026-10-12RSI-Anything: Unlocking Recursive Self-Improvement with World Models2026-10-11A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control2026-10-11FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems2026-10-11VioLA: Learning Generalist Humanoid Control Policies from Human Data2026-10-10FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?2026-10-10Ecology of AI Agents: Collaboration Creates a Population Threshold for Takeoff2026-10-08Long-WAM: Scaling the Context of World-Action Models2026-10-08SciExam for ENSO: Can AI Agents Build Climate Models?2026-10-08RoboJEPA: Scaling Robotic Latent World Models2026-10-08RECAST: Learning to Compute the Right Context through Adaptive Evidence Routing2026-10-07WorldSonus: Bringing Sound to Worlds2026-10-07IdeaAnchor: Teaching LLMs to Turn Literature into Research Ideas2026-10-07Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?2026-10-06CLIFT: Conformal Self-Verification for Web Agent Training and Test-Time Scaling2026-10-06PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data2026-10-06Base Models Can Reason By Taking a Cue From Training Data2026-10-05LESSER: Post-Training Data Selection with Output-Layer Gradients2026-10-05Language Models that Play Chess and Explain Their Moves2026-10-05Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis2026-10-04One Basis to Animate Them All: Gaussian Blendshape Distillation for Real-Time Avatars2026-10-04The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models2026-10-04DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation2026-10-03VISTA: A Visual Harness for Reasoning in an Interactive World2026-10-01Looped Diffusion Transformer2026-10-01Scaling Laws for Looped Mixture of Experts2026-10-01EvoDuet: Bilevel Co-Evolution of Web Searching and Task Solving for Scientific Discovery2026-09-30Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies2026-09-30Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning2026-09-30Pretraining Latent Information Feedback Transformers with Teacher Supervision2026-09-29Unifying Distributional Training for One-Step Visual Generation2026-09-29Improving Test-Time Scaling with Adaptive Looped Transformers2026-09-29Telescopic Language Models2026-09-28Statistical attribute alignment for black-box generative AI via output post-processing2026-09-28User Model Extraction via Belief Self-Distillation2026-09-28Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency2026-09-27TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations2026-09-27LLM Agents Can Easily Tamper With Their Own Traces2026-09-27Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure2026-09-26PoEM: Predicting RL Outcomes from Existing Policies2026-09-24Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model2026-09-24StudentBench: AI and human tutoring yield equivalent GRE learning gains2026-09-24Agent-Editing World Model: Rethinking World Modeling for LLM Agents2026-09-23Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents2026-09-23FleXray: Universal Clinical X-ray Segmentation2026-09-22GameHorizon Suite: Multi-Horizon Data and Evaluation in Gameplay2026-09-22Emergent Collusion in Long-Horizon LLM Agent Interaction2026-09-22WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory2026-09-21Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw2026-09-21Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design2026-09-21CodeMidas: Scaling Agentic Coding RL Environments from Code Itself2026-09-20RetireOPD: Self-Retiring On-Policy Distillation for Agentic Reinforcement Learning2026-09-20Quantifying Overclaiming Propensity in Frontier LLM Agents