Daily AI Paper

When we think about video AI, it’s tempting to imagine that “seeing more frames” automatically solves the problem. But streaming video models live under a much harsher reality. They don’t get to rewind, they don’t get unlimited memory, and they can’t always afford to inspect every frame at high resolution. So the central problem this paper tackles is very practical: can streaming vision-language models actually perceive fast, high-dynamic events in real-world video streams, or do they miss the crucial details when the action gets too quick? That matters because many real applications are exactly like this. Sports, traffic, wildlife, gaming, and other live feeds are full of brief events where the important evidence can appear and disappear in a fraction of a second. A model that samples too sparsely may preserve long context, but it will miss the moment a tire touches a track feature, or the exact number of wing flaps, or some other fleeting event. On the other hand, if the model tries to sample everything densely, it can run out of context budget and lose the broader history it also needs to answer correctly. This is the core trade-off FastBench is built around: temporal granularity versus long-term memory under a bounded token budget. To understand why this benchmark is needed, it helps to place it in the broader landscape. Streaming video understanding has become an important direction beyond offline clip analysis. Earlier video-language models were often evaluated on trimmed clips, where the relevant content is already centered and bounded. Streaming benchmarks instead ask models to work on continuous input and answer questions with different temporal scopes: sometimes about the past, sometimes about the present moment, sometimes about the future. Existing benchmarks like OVO-Bench, OVBench, and PhoStream cover parts of this problem, especially temporal reasoning, response timing, and memory. But the authors argue that these benchmarks do not specifically target high-dynamic perception. In other words, they test whether a model can reason over time, but not necessarily whether it can notice very fast visual changes in the first place. That gap is what FastBench tries to fill. The benchmark is designed for high-dynamic real-world streams, and it focuses on whether a model can answer questions whose evidence depends on fine-grained temporal perception. The data come from untrimmed videos across domains like sports, gaming, nature, and traffic. The key point is that these are not synthetic motion puzzles. They are real streams, with all the messiness that implies. The construction pipeline is one of the more interesting parts of the paper. It starts by using a vision-language model to brainstorm search keywords for high-dynamic scenarios, then retrieving raw videos from platforms like YouTube and Bilibili. From there, metadata such as timestamped comments and danmaku help localize segments likely to contain fast-moving events. Each raw video is cut into short clips, up to five seconds long, and the authors use a multi-stage annotation process to generate question-answer pairs. First, a VLM looks at the native high-frame-rate clip and proposes a candidate question and answer. But not every visually dynamic question is truly high-dynamic in the sense the benchmark wants. Some questions can still be answered from a sparse 2 FPS version of the clip. So the next step is a low-FPS verification filter: the clip is downsampled to 2 FPS, shown back to the model, and if the question can still be answered correctly, it is treated as pseudo-dynamic and removed. This is important because it ensures the benchmark is testing genuine dependence on fine temporal detail, not just generic scene understanding. After that comes a more novel verification stage grounded in physical trajectories. The model first identifies the target temporal interval and selects an initial frame. It then generates a text query for the relevant object, which is passed to SAM3 for segmentation. CoTracker3 then tracks those points through the segment and produces trajectory coordinates. The VLM uses those trajectories to verify or correct the answer. The idea here is to reduce hallucination by grounding the QA pair in objective motion evidence rather than relying only on the model’s own visual interpretation. Any pair that cannot be verified or corrected is discarded. Finally, human experts inspect the remaining pairs. The paper says ten experts perform three rounds of review, checking answer correctness, clarity of the evidence, and whether the question really depends on high-dynamic visual information rather than being answerable at 2 FPS or without vision at all. They also annotate the evidence interval and the timestamps where high-FPS perception is required. So the benchmark is not just a list of questions; it comes with human-annotated temporal evidence. The resulting benchmark contains 306 question-answer pairs. It spans eight domains and six capability categories, and it includes backward, instant, and forward temporal scopes. That means some questions ask about what happened before the query, some about what is happening right then, and some about what will happen after. This is useful because streaming models have to handle all three if they are to be genuinely useful in live settings. Alongside the benchmark, the authors introduce ProactiveFrame, which is a training-free baseline. The motivation is simple: instead of uniformly sampling frames at one fixed rate, can a model adjust its temporal perception more intelligently as the stream unfolds? ProactiveFrame uses text prompts to adapt incoming frame rates, and it maintains a dual-tier sliding window. Recent observations can be kept at high frame rate so the model does not miss fleeting details, while older content is gradually compressed into a sparse historical context. So the system tries to preserve both immediate precision and longer-term memory in a single chronological stream. This is a pragmatic design. It does not require retraining a model or learning a new controller. But it also exposes a hard question: can the model itself tell when a moment is likely to require finer temporal sampling? The experiments suggest that this is still very difficult. On the evaluation side, the paper compares different sampling strategies and model settings on FastBench. One important result is that more dense sampling does help. For example, Qwen3-VL-8B improves from 32.9% at 2 FPS to 44.6% at 24 FPS. That is a substantial gain, and it confirms the benchmark is sensitive to temporal granularity. But the gains do not keep increasing indefinitely, because as history gets compressed the model still faces a trade-off between seeing the fast event and retaining enough surrounding context. The strongest reported model, Gemini-3.5-Flash, reaches only 50.7% on the benchmark. That is a striking number because it suggests that even strong modern streaming VLMs are far from robust on high-dynamic perception. In other words, this is not a problem that has already been solved by scale alone. ProactiveFrame does better than sparse uniform sampling, improving by 5.4 and 1.5 percentage points in the comparisons reported by the paper, but it still falls well short of oracle-guided focusing. That gap is important. It means the model can benefit from a smarter buffering strategy, but it still cannot reliably decide on its own, from the stream alone, when it should devote more attention to fine temporal detail. So the bottleneck is not just sampling efficiency; it is also event anticipation and uncertainty detection. What do these results actually show? First, FastBench is doing its job: it separates models that can reason over video from models that can truly perceive fast visual changes. Second, the performance ceiling is low enough to reveal a real weakness in current streaming VLMs. Third, adaptive sampling helps, but not enough to close the gap to an oracle. So the paper’s main conclusion is not merely that “more frames are better.” It is that current models still struggle to manage the perception-memory trade-off in a way that is responsive to the dynamics of the stream itself. There are also some limitations to keep in mind. FastBench is relatively small, with 306 QA pairs, so it is best thought of as a focused diagnostic benchmark rather than a massive general-purpose dataset. It also relies on a fairly elaborate annotation pipeline with model-assisted generation, trajectory verification, and human inspection, which is strong for quality but may limit scale. And because the benchmark centers on selected high-dynamic scenarios, it does not cover every possible streaming setting. The broader challenge of open-world streaming perception remains larger than this one benchmark. Still, the paper makes a valuable contribution. It provides a concrete way to test whether streaming VLMs can actually notice fast events in real video, not just summarize slow ones. And by pairing the benchmark with a simple adaptive baseline, it also shows how far current systems are from robust, self-directed high-dynamic perception. The next step, as this paper makes clear, is not just building bigger models, but building models that can decide when the stream deserves sharper eyes.