Medical Report Generation

Medical report generation (MRG) is a task which focus on training AI to automatically generate professional report according the input image data. This can help clinicians make faster and more accurate decision since the task itself is both time consuming and error prone even for experienced doctors.

Aggfgg

Deep neural network and transformer based architecture are currently the most popular methods for this certain task, however, when we try to transfer out pre-trained model into this certain domain, their performance always degrade.

The following are some of the reasons why RSG is hard for pre-trained models:

• Language datasets in a particular domain can sometimes be quite different from the large number of datasets available on the Internet
• During the fine-tuning phase, datasets in the medical field are often unevenly distributed

More recently, multi-modal learning and contrastive learning have shown some inspiring results in this field, but it's still challenging and requires further attention.

Here are some additional readings to go deeper on the task:

• On the Automatic Generation of Medical Imaging Reports

  https://doi.org/10.48550/arXiv.1711.08195

• A scoping review of transfer learning research on medical image analysis using ImageNet

  https://arxiv.org/abs/2004.13175

• A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis

https://arxiv.org/abs/2004.12150

(Image credit : Transformers in Medical Imaging: A Survey)