https://github.com/X-PLUG/mPLUG-2
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
Zero-shot video retrieval is the task of retrieving relevant videos based on a query (usually in text form) without any prior training on specific examples of those videos. Unlike traditional retrieval methods that rely on supervised learning with annotated datasets, zero-shot retrieval leverages pre-trained models, typically based on large-scale vision-language learning, to understand semantic relationships between textual descriptions and video content.
This approach enables retrieval of unseen video concepts by generalizing knowledge from diverse training data, making it highly useful for domains with limited labeled data, such as broadcast media, surveillance, and historical archives.
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
Multi-granularity Correspondence Learning from Long-term Noisy Videos
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
InternVideo: General Video Foundation Models via Generative and Discriminative Learning
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.