| GPT-2 + CLIP-14 + CLIP-multilingual (Zero-Shot) |
- Accuracy
- 61.2
|
Composing Ensembles of Pre-trained Models via Iterative Consensus
2022-10-20 |
연결 없음 |
| GPT-2 + CLIP-32 (Zero-Shot) |
- Accuracy
- 58.4
|
Composing Ensembles of Pre-trained Models via Iterative Consensus
2022-10-20 |
연결 없음 |
| VideoCoCa |
- Accuracy
- 56.1
|
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners
2022-12-09 |
연결 없음 |
| Mirasol3B |
- Accuracy
- 51.13
|
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities
2023-11-09 |
연결 없음 |
| VAST |
- Accuracy
- 50.4
|
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset
2023-05-29 |
코드 1 · 코드 2 |
| COSA |
- Accuracy
- 49.9
|
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
2023-06-15 |
코드 1 |
| MA-LMM |
- Accuracy
- 49.8
|
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
2024-04-08 |
코드 1 |
| VideoChat2 |
- Accuracy
- 49.1
- Confidence score
- 3.3
|
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
2023-11-28 |
코드 1 · 코드 2 · 코드 3 |
| VALOR |
- Accuracy
- 48.6
|
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset
2023-04-17 |
코드 1 |
| UMT-L (ViT-L/16) |
- Accuracy
- 47.9
|
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
2023-03-28 |
코드 1 |
| LLaMA-VID-13B (2 Token) |
- Accuracy
- 47.5
- Confidence score
- 3.3
|
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
2023-11-28 |
코드 1 · 코드 2 |
| LLaMA-VID-7B (2 Token) |
- Accuracy
- 47.4
- Confidence score
- 3.3
|
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
2023-11-28 |
코드 1 · 코드 2 |
| Chat-UniVi-13B |
- Accuracy
- 46.4
- Confidence score
- 3.3
|
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
2023-11-14 |
코드 1 · 코드 2 · 코드 3 · 코드 4 |
| BT-Adapter (zero-shot) |
- Accuracy
- 46.1
- Confidence score
- 3.6
|
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning
2023-09-27 |
코드 1 |
| MovieChat |
- Accuracy
- 45.7
- Confidence score
- 3.1
|
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
2023-07-31 |
코드 1 |
| Video-LLaVA |
- Accuracy
- 45.3
- Confidence score
- 3.3
|
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
2023-11-16 |
코드 1 · 코드 2 · 코드 3 · 코드 4 · 코드 5 · 코드 6 |
| TESTA (ViT-B/16) |
- Accuracy
- 45
|
TESTA: Temporal-Spatial Token Aggregation for Long-form Video-Language Understanding
2023-10-29 |
코드 1 |
| FrozenBiLM+ |
- Accuracy
- 44.8
|
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models
2023-08-18 |
코드 1 |
| VindLU |
- Accuracy
- 44.7
|
VindLU: A Recipe for Effective Video-and-Language Pretraining
2022-12-09 |
코드 1 |
| Singularity-temporal |
- Accuracy
- 44.1
|
Revealing Single Frame Bias for Video-and-Language Learning
2022-06-07 |
코드 1 · 코드 2 |
| FrozenBiLM |
- Accuracy
- 43.2
|
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
2022-06-16 |
코드 1 · 코드 2 · 코드 3 |
| Singularity |
- Accuracy
- 43.1
|
Revealing Single Frame Bias for Video-and-Language Learning
2022-06-07 |
코드 1 · 코드 2 |
| Text + Text (no Multimodal Pretext Training) |
- Accuracy
- 41.4
|
Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval
2022-06-05 |
코드 1 |
| All-in-one+ |
- Accuracy
- 40.0
|
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models
2023-08-18 |
코드 1 |
| VIOLET+ |
- Accuracy
- 39.7
|
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models
2023-08-18 |
코드 1 |
| Just Ask (fine-tune) |
- Accuracy
- 38.9
|
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
2020-12-01 |
코드 1 |
| LocVLM-Vid-B+ |
- Accuracy
- 38.2
|
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs
2024-04-11 |
코드 1 |
| LocVLM-Vid-B |
- Accuracy
- 37.4
|
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs
2024-04-11 |
코드 1 |
| Video-ChatGPT |
- Accuracy
- 35.2
- Confidence score
- 2.7
|
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
2023-06-08 |
코드 1 · 코드 2 |
| LLaMA Adapter V2 |
- Accuracy
- 34.2
- Confidence score
- 2.7
|
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
2023-04-28 |
코드 1 · 코드 2 · 코드 3 |