Gemini can hunt video moments itself — agentic video understanding

DeepMind is changing how video is understood

DeepMind's newly announced agentic video understanding differs from conventional methods by having the model proactively search for and process only the critical segments it needs. It works with Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite; developers only need to set the processing mode in the Gemini API to agentic.

Static frame processing wastes tokens and time

Previous video understanding typically applied uniform frame-level analysis across entire videos. For long videos, unnecessary frames accumulated, driving up analysis costs and forcing developers to manually adjust sampling strategies.

Agentic video understanding solves this with an agent-like loop combined with video tools. The model first sets a goal, then selectively retrieves frames, audio, or metadata as needed, analyzing only the portions required.

Benchmark results: 66% lower cost, 88% fewer tokens, 7% higher accuracy

Using a standard video analysis benchmark, DeepMind reports three simultaneous improvements:

• Token usage: down up to 88%
• Analysis cost: down up to 66%
• Accuracy: up to 7%

This advancement was tested on tasks ranging from 10-minute clips to 90-minute feature-length films and even noisy live-stream footage. Where previous approaches forced tradeoffs between performance and efficiency, this approach can pursue both at once.

What you can actually build

DeepMind showcases several concrete use cases:

• sub-second moment retrieval: accurately locate specific moments or shot transitions within a second, enabling automated video editing.
• long-form needle-in-a-haystack search: on hour-long videos, process only the relevant segments when answering specific questions, avoiding token waste.
• anomaly detection: after flagging unusual segments, re-examine them with higher resolution to confirm.
• counting action & object: accurately count repeated actions or objects over time.

How to get started

The feature is available in the Gemini API starting September 1, 2026. Entry points are Google AI Studio and the Gemini Enterprise Agent Platform. No additional fees apply; only standard token pricing is used.

Activation is simple. Just set "processing": "agentic" in the request payload:

from google import genai

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-3.7-flash",
    input=[
        {
            "type": "video",
            "uri": "https://youtu.be/7Z5Vy9JBANs",
            "processing": "agentic"
        }
    ],
    prompt="List the top 3 key points discussed in this video."
)

for event in interaction:
    if event.type == "message":
        print(event.content.text)

This example requests the top 3 key points from a YouTube video. The video file path or public link can be used interchangeably.

References

• DeepMind announcement blog: Introducing agentic video understanding with Gemini
• Gemini API docs: Video understanding
• Gemini Enterprise: Video understanding models