Reviso Plans Video Focus Before Captioning

Reviso, an agentic video captioning model from Peking University, Kling AI Research, and six other universities, treats detailed video understanding as a planning problem. Instead of weighting every frame equally, the system builds a hierarchical Context-Focus-Points plan from a low-frame-rate preview, then selectively revisits segments with tools like image cropping and OCR. Officially released on November 3, 2026, Reviso lifts its Qwen3.5-9B backbone by 15.4 F1 points on the MCTS-VCB benchmark and improves downstream video-QA accuracy by 4.9 points on LVBench.

Hierarchical Planning Before Frame Selection

Reviso begins with a low-FPS global preview of the video, from which it constructs a three-tier plan: Context identifies the setting and scene type, Focus prioritizes key aspects worth attention, and Points spell out concrete observation targets. Each point includes the tools required to resolve it, such as frame cropping for spatial detail or OCR for text extraction. The model then revisits those segments at higher frame rates, interprets findings, and updates the plan iteratively.

The team argues that not all visual evidence deserves equal attention. "The importance of visual information depends on the content of each video," the paper states. By deferring high-resolution inspection until after planning, Reviso allocates its 128-frame budget to the segments that matter most for accurate captioning.

Training on Verified Trajectories

The training pipeline uses 10,000 supervised fine-tuning trajectories and 20,000 reinforcement learning samples, drawn from LLaVA-Video-178K, Vript, Ego4D, and YouTube. The team runs LoRA SFT for two epochs, followed by 200 steps of GRPO with three reward components:

• Key-Fact Reward: scores the presence of ground-truth details in the caption
• Plan Alignment Reward: measures how well observations match the stated plan
• Unproductive Tool-Call Penalty: adjusts observation-cost penalties based on caption quality

This verification-guided approach ensures that Reviso learns to plan efficiently and avoid wasting frames on irrelevant segments.

Benchmark Performance

With every evaluated model capped at 128 input frames, Reviso scores F1 of 39.5 on DREAM-1K and 60.9 on MCTS-VCB. It beats all evaluated open-source baselines across four captioning benchmarks and lifts downstream video-QA accuracy on LVBench by 4.9 points over the Qwen3.5-9B backbone. The authors note that among agentic workflows on DREAM-1K, "only GPT-6 Astra paired with Codex achieving a higher F1 score" than Reviso.

The model's gains come from selective attention, not brute-force frame sampling. By planning what to watch before committing frames, Reviso extracts more actionable information from the same computational budget.

Release and Open-Source Availability

Code is available at github.com/pkucs-Ltf/Reviso, and the paper is posted on Hugging Face. The release joins a recent wave of agentic-harness papers in open-source AI, reflecting growing interest in tool-augmented perception for long-form video understanding. The seven-university collaboration includes Peking University, Kling AI Research, HKUST(GZ), Fudan University, HKU, HKUST, NUS, and Tsinghua University.

What This Means

Reviso demonstrates that video understanding benefits from the same agentic planning workflows now common in code generation and web navigation. By treating captioning as active perception, where the model decides what to inspect and when, Reviso extracts richer semantic content from the same frame budget. This approach may influence future multimodal architectures that need to balance detail and computational cost in long-context video tasks.