V-CAST proposes training-free video-token compression, testing performance and runtime across models

V-CAST allocates token budgets using video changes while preserving spatial and temporal coordinates. Accuracy, memory and end-to-end runtime gains vary across the team's tests; halving runtime applies to a specific audiovisual configuration.
Editorial correction (November 17, 2026): A duplicated experimental-description passage has been removed. Runtime gains remain specific to the corresponding models and test settings described below.
To understand long videos, AI models encode frames into visual tokens and pass them to a language model.
More frames mean repeated images consume compute and GPU memory, but excessive compression can lose fleeting clues and changes between actions.
Researchers from Sun Yat-sen University, the Hong Kong Polytechnic University and other universities propose V-CAST: training-free, plug-and-play filtering and pruning before visual tokens enter the language model.

How much understanding survives pruning? The team tested six configurations across Qwen3-VL, LLaVA and Qwen2.5-Omni on ten video and audiovisual benchmarks:
- On 256-frame long-video evaluations, V-CAST's average score exceeded the next-best compression method by 1.5 points;
- In 768-frame audiovisual experiments, it retained 97.1% of the uncompressed model's performance while reducing total inference time by 50.0%;
- In 32-frame Qwen3-VL-8B experiments, retaining only 25% of visual tokens, average performance relative to the original model remained 98.2%.

△ V-CAST's long-video understanding, performance retention and inference efficiency.
Video-token compression depends on spatial and temporal coverage
Video models encode images as visual tokens. More frames increase prefill computation before the language model generates an answer. Repeated backgrounds and actions also consume compute and memory, making preliminary token filtering useful.
The difficulty is that action ordering, counting and event localization require linking clues across time. Concentrating the budget on a few frames can miss changes around actions. Compression must therefore cover different periods and retain details within frames. V-CAST first decides how many tokens to keep per frame, then which ones, preserving original spatial and temporal coordinates.
Two observations: temporal coverage and positional correspondence
Observation one: equal budgets can produce different temporal coverage
How many tokens should each frame receive? Uniform allocation covers every moment but does not distinguish brief action transitions from long, repetitive backgrounds.
Global-difference allocation finds distinctive frames but can miss changes between stages. V-CAST focuses on transitions between neighboring frames, assigning more budget where actions or scenes change.

△ Frame-budget allocation by different strategies in short and long videos.
Observation two: positions must remain aligned after feature compression
Compressed content must still correspond to its original position. Qwen3-VL records time, height and width coordinates for visual tokens. Merging tokens from different positions makes the new feature difficult to assign to any original position.
V-CAST directly selects a subset of tokens, preserving original order and coordinates. The paper tests this choice against local pooling, fixed-position selection and random selection.

△ Left: position changes from merging versus direct selection. Right: pooling versus direct selection in a local 2×2 grid.
At the same 25% retention rate, both direct-selection strategies outperform local pooling, supporting preserved positional correspondence.
Two-step compression: allocate budgets, then select detail
Based on these observations, V-CAST allocates budgets across frames and filters within frames after visual encoding and projection, but before language-model prefill.

△ V-CAST allocates each frame's token budget, then selects tokens within it.
- Allocate budgets from changes between adjacent frames
V-CAST summarizes each frame with a mean feature and uses local curvature to measure turning points in features over time. Stronger transitions receive more tokens, while a minimum allocation per frame balances change sensitivity with coverage.
- Select within-frame tokens with two complementary signals
Given a frame budget, V-CAST computes two token scores:
- Difference from the frame's mean feature, helping retain local details such as a small object in a corner.
- Feature activation strength, retaining regions that elicit strong visual-encoder responses.
It adds the scores and keeps the highest-scoring tokens within each frame's budget, balancing salient regions and local detail. No pairwise token-similarity computation is required, keeping added overhead small.
- Preserve original order and spatial/temporal coordinates
Selected tokens retain their original order, coordinates and positional encodings. The process needs no training or extraction of decoder-attention weights.
Mainly Qwen: six model configurations and ten benchmarks
Experiments extend from Qwen3-VL-8B to 32B, MoE, LLaVA and audiovisual models, comparing five other methods under unified settings.
Qwen3-VL-8B: input length versus compression budget
The main experiments cover 32/64-frame inputs and 25%/15% visual-token retention, comparing four video-understanding benchmarks.

△ Main Qwen3-VL-8B-Instruct experiments.
At 32 frames and 25% retention, V-CAST averages 63.1 across four benchmarks, retaining 98.2% of original performance. At 64 frames, it scores 65.0, exceeding runner-up VidCom² by 1.0 points. With retention reduced to 15%, it still has the highest mean among compared compression methods for both 32 and 64 frames.
From 8B to 32B and MoE
On Qwen3-VL-32B-Instruct, with 32 frames and 25% retention, V-CAST averages 66.5, above VidCom²'s 65.6.

△ Qwen3-VL-32B-Instruct results with 32-frame inputs.
Pruning also makes some memory-limited evaluations possible. Under the same runtime configuration, uncompressed Qwen3-VL-30B-A3B-Instruct cannot finish the 64-frame evaluation; V-CAST completes it at 25% retention, averaging 68.1.

△ Qwen3-VL-30B-A3B-Instruct with 64 frames. OOM means out of GPU memory.
Beyond Qwen: LLaVA and audiovisual models
With LLaVA-OneVision-7B (32 frames) and LLaVA-Video-7B (64 frames), retaining 25% of visual tokens preserves 99.5% and 95.4% of their original average performance respectively.

△ Results for LLaVA-OneVision-7B and LLaVA-Video-7B.
In 768-frame Qwen2.5-Omni-7B experiments, audio stays complete while visual tokens are retained at 25%. V-CAST averages 49.6 across five benchmarks, retaining 97.1% of original performance and reducing total inference time by 50.0%.

△ Five benchmark scores and total inference time for Qwen2.5-Omni-7B at 768 frames.
From 32 to 256 frames: testing long-context use
Extra frames bring both clues and repetition. Increasing inputs from 32 to 256 frames tests whether models can use the added information. At 256 frames, V-CAST leads VidCom² by 1.5 points on the average of four long-video evaluations.

△ Long-video understanding with 32–256 input frames.
Efficiency: 51.2% less prefill latency, 13.6% less total time
Pruning mainly reduces prefill computation. Full evaluation also includes video reading, visual encoding and answer generation, so overall savings are smaller. This test uses Qwen3-VL-8B-Instruct on VideoMME with 32 frames and 25% visual-token retention.

△ Qwen3-VL-8B-Instruct VideoMME scores and inference overhead.
Compared with the uncompressed model, V-CAST reduces prefill latency by 51.2%, peak GPU memory by 13.0%, and full-evaluation time by 13.6%. LLaVA-OneVision-7B prefill time also falls from 96.6 seconds to 26.5 seconds, with peak GPU memory down 11.4%.

△ LLaVA-OneVision-7B prefill latency, generation latency and peak GPU memory.
Ablation and transfer: where the benefits come from
Replacing components under equal budgets shows curvature allocation achieves the highest mean at both 32 and 64 frames. At 32 frames, the mean of MLVU and VideoMME overall scores rises from the global-uniqueness strategy's 62.6 to 63.1.
Within-frame filtering also needs both complementary signals: feature difference alone may miss key information similar to the frame mean, while activation alone may retain repetitive regions. Combining both scores gives the highest average.
Curvature allocation can be reused independently. Keeping VisionZip's and VidCom²'s original compression mechanisms but adding curvature budgets improves both on LongVideoBench, MLVU and VideoMME at 25% retention.

△ VisionZip and VidCom² before and after adding curvature-based allocation.
Examples: which moments receive the budget?
In a case asking for the date of a zodiacal-light appearance, the clue is fleeting. Counting glaze-color types requires several glazing stages. V-CAST retains more tokens around the astronomical event and glazing transitions respectively.

△ Tokens retained per frame in two videos. Red lines indicate V-CAST.
Retained frames also affect answers. V-CAST answers correctly in examples locating a wedding episode and identifying acrobatic movements.

△ Different methods' answers for plot localization and acrobatics recognition.
Overall, V-CAST allocates temporal budgets through curvature, selects spatial detail through dual-anchor scoring and preserves original coordinates, compressing visual tokens without training.
Results across models and input lengths suggest that allocating limited budgets appropriately can balance video understanding and inference efficiency.
Paper: V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models. Authors: Lin Xinying, Liu Xuyang, Wang Yiyu, Ma Teng, Li Jiasheng, Wen Zichen and Ren Wenqi. Affiliations: Sun Yat-sen University Shenzhen Campus, Shenzhen Hetao College, the Hong Kong Polytechnic University, the Hong Kong University of Science and Technology (Guangzhou), and Shanghai Jiao Tong University. Project page: https://xinyouu.github.io/V-CAST/ Code: https://github.com/xinyouu/V-CAST
This article comes from QbitAI. Author: the V-CAST team.
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




