Papers with Code 아카이브

Visual Grounding

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

전체 기법 전체 데이터셋

하위 태스크: 3
평가표: 4
보존된 평가 기록: 20

Visual Grounding (VG) aims to locate the most relevant object or region in an image, based on a natural language query. The query can be a phrase, a sentence, or even a multi-round dialogue. There are three main challenges in VG:

What is the main focus in a query?
How to understand an image?
How to locate an object?

코드

관련 논문과 평가 결과에 연결된 구현입니다. 출처에서 확인된 저자 구현에는 공식 코드 표시가 있습니다.

https://github.com/2toinf/ivm

공식 코드

Instruction-Guided Visual Masking

https://github.com/360cvgroup/inner-adaptor-architecture

공식 코드

IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities

https://github.com/3dlg-hcvc/M3DRef-CLIP

공식 코드

Multi3DRefer: Grounding Text Description to Multiple 3D Objects

https://github.com/4dvlab/wildrefer

공식 코드

WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language

https://github.com/4m4n5/CLIP-Lite

공식 코드

CLIP-Lite: Information Efficient Visual Representation Learning with Language Supervision

https://github.com/BierOne/VQA-AttReg

공식 코드

Answer Questions with Right Image Regions: A Visual Attention Regularization Approach

https://github.com/ControlNet/HYDRA

공식 코드

HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning

https://github.com/CurryYuan/InstanceRefer

공식 코드

InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring

https://github.com/CurryYuan/ZSVG3D

공식 코드

Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding

https://github.com/DannyMerkx/caption2image

공식 코드

Learning semantic sentence representations from visually grounded language without lexical knowledge

https://github.com/DannyMerkx/speech2image

공식 코드

Language learning using Speech to Image retrieval

https://github.com/EvLab-MIT/LexiContrastiveGrd

공식 코드

Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

https://github.com/FoundationVision/Groma

공식 코드

Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models

https://github.com/HorizonRobotics/BIP3D

공식 코드

BIP3D: Bridging 2D Images and 3D Perception for Embodied Intelligence

https://github.com/JHKim-snu/GVCCI

공식 코드

GVCCI: Lifelong Learning of Visual Grounding for Language-Guided Robotic Manipulation

https://github.com/JinYuanLi0012/RiVEG

공식 코드

LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognition

https://github.com/KuofengGao/MLLM-Grounding-Robustness

공식 코드

Adversarial Robustness for Visual Grounding of Multimodal Large Language Models

https://github.com/MILVLG/mmnas

공식 코드

Deep Multimodal Neural Architecture Search

https://github.com/OSU-NLP-Group/UGround

공식 코드

Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents

https://github.com/Paranioar/UniPT

공식 코드

UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory

https://github.com/THUNLP-MT/EscapeCraft

공식 코드

How Do Multimodal Large Language Models Handle Complex Multimodal Reasoning? Placing Them in An Extensible Escape Game

https://github.com/URSA-MATH/URSA-MATH

공식 코드

URSA: Understanding and Verifying Chain-of-thought Reasoning in Multimodal Mathematics

https://github.com/UniModal4Reasoning/DocGenome

공식 코드

DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models

https://github.com/VoyagerXvoyagerx/InstructSAM

공식 코드

InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition

https://github.com/WangYuxuan93/CVLUE

공식 코드

CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation

https://github.com/X7J92/VGNet

공식 코드

Phrase Decoupling Cross-Modal Hierarchical Matching and Progressive Position Correction for Visual Grounding

https://github.com/XuYunqiu/MC-Bench

공식 코드

MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs

https://github.com/YapengTian/CCOL-CVPR21

공식 코드

Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation

https://github.com/ZzZZCHS/WS-3DVG

공식 코드

Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding

https://github.com/ai4ce/seeunsafe

공식 코드

When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis

https://github.com/ai4math-shanzhang/sve-math

공식 코드

Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs

https://github.com/aim-uofa/SegAgent

공식 코드

SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories

https://github.com/akirafukui/vqa-mcb

공식 코드

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

https://github.com/akskuchi/dhm-visual-storytelling

공식 코드

Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition

https://github.com/akskuchi/groovist

공식 코드

GROOViST: A Metric for Grounding Objects in Visual Storytelling

https://github.com/alexrame/rewardedsoups

공식 코드

Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.