Video-based Generative Performance Benchmarking (Consistency)

The benchmark evaluates a generative Video Conversational Model with respect to Consistency.

We curate a test set based on the ActivityNet-200 dataset, featuring videos with rich, dense descriptive captions and associated question-answer pairs from human annotations. We develop an evaluation pipeline using the GPT-3.5 model that assigns a relative score to the generated predictions on a scale of 1-5.