https://github.com/damo-nlp-sg/inf-clip
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Papers with Code 아카이브
Video Captioning
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 6
- 평가표
- 13
- 보존된 평가 기록
- 86
Video Captioning is a task of automatic captioning a video by understanding the action and event in the video which can help in the retrieval of the video efficiently through text.
Source: NITS-VC System for VATEX Video Captioning Challenge 2020
코드
관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.
https://github.com/damo-nlp-sg/videollama3
VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
https://github.com/doc-doc/NExT-OE
PaLI-X: On Scaling up a Multilingual Vision and Language Model
https://github.com/eric-xw/Video-guided-Machine-Translation
VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
https://github.com/facebookresearch/ActivityNet-Entities
Learning to Generate Grounded Visual Captions without Localization Supervision
https://github.com/gulucaptain/dynamictrl
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
https://github.com/hitachi-rd-cv/mmac_captions
Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal Attention
https://github.com/hobincar/RecNet
https://github.com/indigo-dc/conus-classification-theano
https://github.com/indigo-dc/phytoplankton-classification-theano
https://github.com/indigo-dc/seeds-classification-theano
https://github.com/jacobkimmel/pytorch_convgru
Delving Deeper into Convolutional Networks for Learning Video Representations
https://github.com/jianjieluo/openai-clip-feature
X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics
https://github.com/jpthu17/dicosa
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
https://github.com/jpthu17/diffusionret
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations
https://github.com/jyrao/MatchTime
SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries
https://github.com/jyrao/UniSoccer
SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries
https://github.com/kingcong/ecolite
ECO: Efficient Convolutional Network for Online Video Understanding
https://github.com/kyegomez/PALI
PaLI-X: On Scaling up a Multilingual Vision and Language Model
https://github.com/laramaktub/plant-classification-tf-train
https://github.com/linjieli222/hero_video_feature_extractor
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
https://github.com/lucidrains/mammut-pytorch
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks
https://github.com/luciferbobo/dae-aqa
What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment
https://github.com/luulinh90s/Contcap
ContCap: A scalable framework for continual image captioning
https://github.com/m-bain/webvid
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
https://github.com/mfayk/cu_icar_image_captioning
https://github.com/mindspore-ai/models/tree/master/research/cv/ecolite
ECO: Efficient Convolutional Network for Online Video Understanding
https://github.com/modelscope/modelscope
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video
https://github.com/nasib-ullah/video-captioning-models-in-Pytorch
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
https://github.com/nzl-thu/musdl
What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment
https://github.com/parkervg/allrecipes-bert
VideoBERT: A Joint Model for Video and Language Representation Learning
https://github.com/princetonvisualai/mqvr
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
https://github.com/rakshithShetty/captionGAN
https://github.com/salesforce/densecap
https://github.com/sczwangxiao/tsgvs-mm2023
RTQ: Rethinking Video-language Understanding Based on Image-text Model
https://github.com/sharegpt4omni/sharegpt4video
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.