Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Grounding
원문 초록
Multimodal Large Language Models (MLLMs) inherit the superior text understanding capabilities of LLMs and extend these capabilities to multimodal scenarios. These models achieve excellent results in the general domain of multimodal tasks. However, in the medical domain, the substantial training costs and the requirement for extensive medical data pose challenges to the development of medical MLLMs. Furthermore, due to the free-text form of answers, tasks such as visual grounding that need to produce output in a prescribed form become difficult for MLLMs. So far, there have been no medical MLLMs works in medical visual grounding area. For the medical vision grounding task, which involves identifying locations in medical images based on short text descriptions, we propose Parameter-efficient Fine-tuning medical multimodal large language models for Medcial Visual Grounding (PFMVG). To validate the performance of the model, we evaluate it on a public benchmark dataset for medical visual grounding, where it achieves competitive results, and significantly outperforming GPT-4v. Our code will be open sourced after peer review.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Efficient Adaptation For Remote Sensing Visual Grounding
GTA1: GUI Test-time Scaling Agent
High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning
LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.