Video-based Generative Performance Benchmarking

The benchmark evaluates a generative Video Conversational Model and covers five key aspects:

• Correctness of Information
• Detailed Orientation
• Contextual Understanding
• Temporal Understanding
• Consistency

We curate a test set based on the ActivityNet-200 dataset, featuring videos with rich, dense descriptive captions and associated question-answer pairs from human annotations. We develop an evaluation pipeline using the GPT-3.5 model that assigns a relative score to the generated predictions on a scale of 1-5.