Remember the game.
A Scanner screens stitched frame windows and builds game-aware memory of the scene, entities, and local dynamics.
GAME-AWARE MEMORYACM Multimedia 2026
◉ Video Understanding⌘ Multi-Agent Planning
Great games.
Annoying glitches.
Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding
1 University of California, Davis 2 Virginia Tech
THE QUESTION
Can a model watch a video game clip, spot the glitch, and explain what happened and when? Can an agentic framework help it do better?
01
Benchmark
Meet VideoGlitchBench.
We introduce VideoGlitchBench, the first benchmark for open-ended video game glitch detection with temporal localization. It contains 5,238 gameplay videos from 120 games. Each video is annotated with detailed glitch descriptions and precise temporal spans. This enables unified evaluation of semantic understanding and temporal grounding.
First, an overview of VideoGlitchBench.
Glitches exhibit diverse temporal patterns. Figure 1 shows typical examples in VideoGlitchBench, including a single glitch, repeated occurrences of the same glitch, and multiple glitches within one video.
Then, let's have a brief look at the statistics.
5,238gameplay videos
120games
6 / 21genres / subgenres
19.11saverage video length60 s maximum
1.03average glitches per video6 maximum
35.94average description tokens164 maximum
Taxonomy of VideoGlitchBench.
Word cloud of all the bug descriptions.
VIDEO SOURCE Our dataset is built upon GamePhysics, a large-scale collection of gameplay videos from the GamePhysics subreddit.
Finally, let's dive into three examples.
Click a colored interval or timestamp to replay that occurrence.
02
Method
Less guessing. More investigation.
We propose GliDe, an agentic framework for open-ended video game glitch detection with temporal localization. Figure 2 illustrates its five-stage pipeline. A lightweight game-aware memory preserves global gameplay context. Debate-based verification distinguishes true glitches from unusual but valid behaviors. Event-level grounding merges fragmented evidence into complete glitch events. The final output contains natural-language descriptions and one or more temporal intervals for each glitch.
Challenge the hypothesis.
A Planner and Executor gather evidence with VQA, zoom-in, and object tracking. An Advocate, Skeptic, and Judge debate whether the event is truly a glitch.
DEBATE-BASED VERIFICATIONRecover the full event.
Semantic clustering links the same glitch across windows. Bidirectional propagation recovers complete intervals before generating a structured report.
EVENT-LEVEL GROUNDINGDEFAULT SETUP 4 FPS → 8 frames / window → 2 × 4 stitched grid → Description + temporal intervals
03
Results
A hard task. An effective framework.
First, learn how we evaluate.
Open-ended video game glitch detection requires evaluating both semantic correctness and temporal localization. For each prediction–ground-truth pair, an LLM judge scores the semantic similarity of their descriptions from 0 to 5. We combine this score with temporal IoU, then perform global one-to-one matching with the Hungarian algorithm.
We report three groups of metrics. Precision, recall, and F1 measure description quality. The mean IoU over matched pairs measures temporal grounding. The overall F1 × IoU score evaluates semantic correctness and temporal localization jointly.
Then, see the results.
Table 1 compares vanilla models and GliDe-enhanced models on VideoGlitchBench. We evaluate four proprietary models and six open-source backbones. Our GliDe framework comparison is conducted on open-source models, while proprietary models are included as reference baselines.
Table 1 · All metrics ↑. P / R / F1 and F1 × IoU are percentages. Green rows use GliDe; gains are absolute differences from the same vanilla backbone.
We present two examples to illustrate the two core challenges of VideoGlitchBench.
Figure 3 highlights the difficulty of distinguishing genuine glitches from visually unusual but valid in-game behaviors. In Human: Fall Flat, the vanilla model falsely identifies twisted limbs and brief apparent floating as glitches. It misses the actual anomaly in the final segment. In contrast, GliDe focuses on the true crane-related failure, producing a prediction that is much closer to the ground truth.
Figure 4 highlights the temporal challenge of fragmented and repeated glitches. In ARK: Survival Evolved, the same black model anomaly appears in two disjoint intervals. The vanilla model detects the two segments separately and fails to associate them as one glitch event. By contrast, GliDe links the repeated observations through semantically consistent evidence and produces a single coherent glitch report with multiple temporal spans.
Finally, what do we find?
Current models still struggle on open-ended video game glitch detection. The best proprietary baseline reaches only 26.01% F1, with an mIoU of 0.47. Across six vanilla open-source models, average F1 is only 14.47%, with an average mIoU of 0.28. VideoGlitchBench remains highly challenging for current multimodal models.
GliDe consistently improves both description generation and temporal grounding. All six open-source backbones show clear gains on all major metrics. Average F1 improves from 14.47% to 36.05%, while mIoU improves from 0.28 to 0.51. The overall F1 × IoU score rises from 4.37% to 17.05%. All GliDe-enhanced models outperform the proprietary baselines in Table 1 on F1, mIoU, and F1 × IoU.
04
Citation
Thanks for reading this far :)
If you find VideoGlitchBench or GliDe useful in your research, please consider citing our work.
BIBTEX
@article{zheng2026open,
title={Open-Ended Video Game Glitch Detection with Agentic Reasoning and Temporal Grounding},
author={Zheng, Muyang and Zhou, Tong and Wu, Geyang and Lin, Zihao and Wang, Haibo and Huang, Lifu},
journal={arXiv preprint arXiv:2604.07818},
year={2026}
}