MultiVENT-Raw

MultiVENT-Raw is a benchmark for text-to-video retrieval and for video-based retrieval-augmented generation (RAG) over raw, event-centric videos. For more information, please read the paper introducing the benchmark: MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos.

Data

For both our primary test collection (MultiVENT-Raw) and our dev set (MicroVENT), we divide the data release into two parts, each with its own HuggingFace repository:

1. Videos and Annotations: Covers the videos (.mp4 and raw audio files, plus keyframes) and annotations associated with these videos. Contains everything you need to run experiments.
2. Extracted Features: Covers various content automatically extracted from the videos, including automatic speech recognition (ASR) transcripts, optical character recognition (OCR) detections, and video embeddings. Note that these features are helpful but optional "extras" that are not needed to run experiments.

Below, we provide more information about these HuggingFace repos for both MultiVENT-Raw and MicroVENT.

Collection

MultiVENT-Raw

Videos and Annotations. The full MultiVENT-Raw collection can be downloaded from HuggingFace: https://huggingface.co/datasets/hltcoe/multivent-raw. Note that:

1. Access is gated. Access requests are generally approved within 24 to 48 hours.
2. The full collection is quite large (4.5 TB), so ensure that you have the sufficient space on disk.

Extracted Features. Extracted features are available at https://huggingface.co/datasets/hltcoe/multivent-raw-features. Further details on the extracted features are available in that repo. Access is gated.

MicroVENT

As noted in the paper, we also release a much smaller dev set that is useful for system development and is also available on HuggingFace: https://huggingface.co/datasets/hltcoe/microvent. Note that:

1. Access is gated.
2. In addition to the videos themselves, the MicroVENT download also includes extracted features for all videos.

Extracted Features. Extracted features are available at https://huggingface.co/datasets/hltcoe/microvent-features. Further details on the extracted features are available in that repo. Access is gated.

Annotations

Beyond the videos in the collection, MultiVENT-Raw also features annotations that include topics, personas, queries, claims, and qrels:

• topic: An event of interest (e.g., the 2025 World Humanoid Robot Games)
• persona: The job title and job description for someone who would likely be interested in, or affected by, the event named by a given topic (e.g., a technology policy analyst from the Center on Security and Emerging Technology).
• query: A query requesting specific information on a topic; each query is associated with a specific persona who might plausibly ask the query.
• claim: A short, self-contained observation about a video associated with a given topic. All claims are human-annotated, in some instances with light editing from an LLM.
• qrels: Relevance judgments that indicate which videos in the collection are relevant to which topic, persona, and query.

Further details on claims and qrels are provided below.

Claim Subsets

Subsets of the full, original set of claims are defined based on what they are relevant to. The subsets below are listed from largest to smallest:

• video: the original, human-annotated claims, each made on the basis of a specific video
• event: the subset of video claims that are judged (by Claude 5 Opus) to be relevant to the event that the video depicts
• persona: the subset of event claims that are judged (by Claude 5 Opus) to be relevant to an associated persona
• query: the subset of persona claims that are judged (by Claude 5 Opus) to be relevant to a given query for that persona

More details on these subsets can be found in Section 3.4 of the paper.

Qrels

A video is deemed relevant to a given event/persona/query if it contains at least one claim judged to be relevant to that event/persona/query. As such different .qrels files are provided at the event, persona, and query level. The query.qrels file is considered the "standard" .qrels file and the retrievel results presented in the paper are based on the relevance judgments in that file. The other .qrels files are provided for completeness.

Attribution

If you use MultiVENT-Raw, please cite our paper as follows:

@misc{kriz2026multiventraw,
      title={MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos}, 
      author={Reno Kriz and David Etter and Alexander Martin and Cameron Carpenter and Debashish Chakraborty and Hannah Recknor and Reihaneh Iranmanesh and Matthew Maciejewski and Kenton Murray and Eugene Yang and Benjamin Van Durme and Aaron Steven White and Andrew Yates and William Walden},
      year={2026},
      eprint={2609.28437},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.28437}
}