Correctness of Information: 3.85Detail Orientation: 3.56Contextual Understanding: 4.21Temporal Understanding: 3.21
Papers with Code · CC BY-SA 4.0
Video-based Generative Performance Benchmarking
The benchmark evaluates a generative Video Conversational Model and covers five key aspects:
- Correctness of Information
- Detailed Orientation
- Contextual Understanding
- Temporal Understanding
- Consistency
We curate a test set based on the ActivityNet-200 dataset, featuring videos with rich, dense descriptive captions and associated question-answer pairs from human annotations. We develop an evaluation pipeline using the GPT-3.5 model that assigns a relative score to the generated predictions on a scale of 1-5.
개요는 각 항목의 일부입니다. 위 탭에서 전체 기록을 페이지별로 탐색할 수 있습니다.
벤치마크
Papers with Code 아카이브의 데이터셋별 기록 예시입니다. 평가 조건과 지표 방향이 검증되지 않아 순위를 의미하지 않습니다.
데이터셋
6개논문
Correctness of Information: 3.40Detail Orientation: 2.80Contextual Understanding: 3.61Temporal Understanding: 2.89
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
Correctness of Information: 3.08Detail Orientation: 2.95Contextual Understanding: 3.49Temporal Understanding: 2.81
Correctness of Information: 2.94Detail Orientation: 2.98Contextual Understanding: 3.43Temporal Understanding: 2.68
Correctness of Information: 3.40Detail Orientation: 2.91Contextual Understanding: 3.72Temporal Understanding: 2.65
Correctness of Information: 3.60Detail Orientation: 3.20Contextual Understanding: 3.90Temporal Understanding: 2.67
mean: 3.32
Correctness of Information: 3.23Detail Orientation: 3.05Contextual Understanding: 3.74Temporal Understanding: 2.93
mean: 3.38
Correctness of Information: 3.63Detail Orientation: 3.25Contextual Understanding: 4Temporal Understanding: 3.23
Correctness of Information: 3.27Detail Orientation: 3.18Contextual Understanding: 3.74Temporal Understanding: 2.83
라이브러리
Papers with Code 리더보드 row에 연결된 구현 저장소입니다.
하위 태스크
VG
Video-based Generative Performance Benchmarking (Consistency)
VG
Video-based Generative Performance Benchmarking (Contextual Understanding)
VG
Video-based Generative Performance Benchmarking (Correctness of Information)
VG
Video-based Generative Performance Benchmarking (Detail Orientation))
VG
Video-based Generative Performance Benchmarking (Temporal Understanding)