Anatomical and biophysical modeling of left atrium (LA) and proximal pulmonary veins (PPVs) is important for clinical management of several cardiac diseases. Magnetic resonance imaging (MRI) allows qualitative assessment of LA and PPVs through visualization. However, there is a strong need for an advanced image segmentation method to be applied to cardiac MRI for quantitative analysis of LA and PPVs. In this study, we address this unmet clinical need by exploring a new deep learning-based segmentation strategy for quantification of LA and PPVs with high accuracy and heightened efficiency. Our approach is based on a multi-view convolutional neural network (CNN) with an adaptive fusion strategy and a new loss function that allows fast and more accurate convergence of the backpropagation based optimization. After training our network from scratch by using more than 60K 2D MRI images (slices), we have evaluated our segmentation strategy to the STACOM 2013 cardiac segmentation challenge benchmark. Qualitative and quantitative evaluations, obtained from the segmentation challenge, indicate that the proposed method achieved the state-of-the-art sensitivity (90%), specificity (99%), precision (94%), and efficiency levels (10 seconds in GPU, and 7.5 minutes in CPU).
In this paper, we propose a fully automatic MRI cardiac segmentation method based on a novel deep convolutional neural network (CNN) designed for the 2017 ACDC MICCAI challenge. The novelty of our network comes with its embedded shape prior and its loss function tailored to the cardiac anatomy. Our model includes a cardiac centerof-mass regression module which allows for an automatic shape prior registration. Also, since our method processes raw MR images without any manual preprocessing and/or image cropping, our CNN learns both high-level features (useful to distinguish the heart from other organs with a similar shape) and low-level features (useful to get accurate segmentation results). Those features are learned with a multi-resolution conv-deconv "grid" architecture which can be seen as an extension of the U-Net. Experimental results reveal that our method can segment the left and right ventricles as well as the myocardium from a 3D MRI cardiac volume in 0.4 second with an average Dice coefficient of 0.90 and an average Hausdorff distance of 10.4 mm.
Deep fully convolutional neural network (FCN) based architectures have shown great potential in medical image segmentation. However, such architectures usually have millions of parameters and inadequate number of training samples leading to over-fitting and poor generalization. In this paper, we present a novel highly parameter and memory efficient FCN based architecture for medical image analysis. We propose a novel up-sampling path which incorporates long skip and short-cut connections to overcome the feature map explosion in FCN like architectures. In order to processes the input images at multiple scales and view points simultaneously, we propose to incorporate Inception module's parallel structures. We also propose a novel dual loss function whose weighting scheme allows to combine advantages of cross-entropy and dice loss. We have validated our proposed network architecture on two publicly available datasets, namely: (i) Automated Cardiac Disease Diagnosis Challenge (ACDC-2017), (ii) Left Ventricular Segmentation Challenge (LV-2011). Our approach in ACDC-2017 challenge stands second place for segmentation and first place in automated cardiac disease diagnosis tasks with an accuracy of 100%. In the LV-2011 challenge our approach attained 0.74 Jaccard index, which is so far the highest published result in fully automated algorithms. From the segmentation we extracted clinically relevant cardiac parameters and hand-crafted features which reflected the clinical diagnostic analysis to train an ensemble system for cardiac disease classification. Our approach combined both cardiac segmentation and disease diagnosis into a fully automated framework which is computational efficient and hence has the potential to be incorporated in computer-aided diagnosis (CAD) tools for clinical application.
We propose a method based on deep learning to perform cardiac segmentation on short axis MRI image stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of U-net is applied to propagate the segmentation of a slice to the adjacent slice below it. In other words, the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice. 3D-consistency is hence explicitly enforced. The method is trained on a large database of 3078 cases from UK Biobank. It is then tested on 756 different cases from UK Biobank and three other state-of-the-art cohorts (ACDC with 100 cases, Sunnybrook with 30 cases, RVSC with 16 cases). Results comparable or even better than the state-of-the-art in terms of distance measures are achieved. They also emphasize the assets of our method, namely enhanced spatial consistency (currently neither considered nor achieved by the state-of-the-art), and the generalization ability to unseen cases even from other databases.
Due to the irregular motion, similar appearance and diverse shape, accurate segmentation of kidney tumor in CT images is a difficult and challenging task. To this end, we present a novel automatic segmentation method, termed as Crossbar-Net, with the goal of accurate segmenting the kidney tumors. Firstly, considering that the traditional learning-based segmentation methods normally employ either whole images or squared patches as the training samples, we innovatively sample the orthogonal non-squared patches (namely crossbar patches), to fully cover the whole kidney tumors in either horizontal or vertical directions. These sampled crossbar patches could not only represent the detailed local information of kidney tumor as the traditional patches, but also describe the global appearance from either horizontal or vertical direction using contextual information. Secondly, with the obtained crossbar patches, we trained a convolutional neural network with two sub-models (i.e., horizontal sub-model and vertical sub-model) in a cascaded manner, to integrate the segmentation results from two directions (i.e., horizontal and vertical). This cascaded training strategy could effectively guarantee the consistency between sub-models, by feeding each other with the most difficult samples, for a better segmentation. In the experiment, we evaluate our method on a real CT kidney tumor dataset, collected from 94 different patients including 3,500 images. Compared with the state-of-the-art segmentation methods, the results demonstrate the superior results of our method on dice ratio score, true positive fraction, centroid distance and Hausdorff distance. Moreover, we have extended our crossbar-net to a different task: cardiac segmentation, showing the promising results for the better generalization.
Cardiac motion estimation and segmentation play important roles in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In this paper, we propose a novel deep learning method for joint estimation of motion and segmentation from cardiac MR image sequences. The proposed network consists of two branches: a cardiac motion estimation branch which is built on a novel unsupervised Siamese style recurrent spatial transformer network, and a cardiac segmentation branch that is based on a fully convolutional network. In particular, a joint multi-scale feature encoder is learned by optimizing the segmentation branch and the motion estimation branch simultaneously. This enables the weakly-supervised segmentation by taking advantage of features that are unsupervisedly learned in the motion estimation branch from a large amount of unannotated data. Experimental results using cardiac MRI images from 220 subjects show that the joint learning of both tasks is complementary and the proposed models outperform the competing methods significantly in terms of accuracy and speed.
A Unified Framework for Comprehensive Cardiac CT Segmentation and Phenotyping: Human-in-the-Loop Data Annotation, Vision Foundation Model Development, Multicenter Evaluation and Clinical Validation
전산화 단층촬영(CT)으로부터 심장 구조물을 포괄적으로 정량화하는 작업은 데이터 가용성이 아닌 측정의 확장성에 의해 제한되어 있어, 임상 현장에서의 일상적 활용이 비현실적이다. 본 연구에서는 휴먼-인-더-루프 어노테이션 파이프라인, 심장 CT 증강 기법, 그리고 60,000개의 레이블 없는 심장 CT 스캔으로 사전학습된 자기지도(self-supervised) 파운데이션 모델을 결합한 포괄적 심장 CT 분할 및 표현형 분석을 위한 통합 프레임워크를 제시한다. 이 방법을 통해 현재까지 구축된 것 중 가장 크고 포괄적인 전문가 어노테이션 심장 CT 분할 데이터셋을 구성하였으며, 총 1,598개 케이스와 14가지 심장 구조물로 이루어져 있다(학습용 1,000개, 외부 테스트 세트 598개). 다섯 개의 외부 데이터셋에서 본 프레임워크는 기존 오픈소스 도구보다 더 정확하고 포괄적으로 모든 구조물을 분할하였다. 자기지도 사전학습은 레이블링 효율성을 향상시켰으며, 저데이터 환경에서의 외부 평가에서 가장 두드러진 성능 향상이 관찰되었다. 합성곱, 트랜스포머, 상태 공간 아키텍처에 걸친 벤치마킹에서 유사한 성능이 나타나, 정확도를 결정하는 요인이 아키텍처가 아닌 데이터 품질과 사전학습임을 시사하였다. 본 프레임워크는 집단 수준의 표현형 분석으로 확장되었으며, 분할된 해부 구조는 인구통계학적 변수 이상으로 심실 기능 및 질병 중증도에 관한 기능적으로 유의미한 정보를 담고 있음이 확인되었다. 사람 레이블이 포함된 최대 규모의 데이터셋, 코드, 모델 가중치, CT 증강 라이브러리, 소프트웨어를 공개함으로써, 본 연구는 일상적으로 획득된 CT 스캔으로부터의 기회적 심장 표현형 분석을 위한 재현 가능한 기반을 제공한다.
임상 심장 영상 파이프라인은 현재 각 데이터셋과 모달리티별로 별도의 모델을 배포하여 중복적인 학습 비용이 발생하고, 해부학적으로 연관된 태스크 간의 지식 공유가 불가능하다. 따라서 반지도 학습(semi-supervised learning), 비지도 도메인 적응(unsupervised domain adaptation), 도메인 일반화(domain generalisation)를 하나의 모델로 통합하는 것은 실질적인 필요 사항이나, 단순한 결합 학습은 근본적인 장벽을 드러낸다: 데이터셋 간 레이블 의미 충돌로 LA Dice 점수가 90.31%에서 83.38%로 하락하고, 복잡도가 불균등한 태스크 간 그래디언트 불균형이 학습 전반에 걸쳐 약한 태스크를 억제한다. 우리는 세 가지 목표 지향적 메커니즘을 통해 이러한 충돌을 해소하는 통합 확산 이미지 분할 프레임워크인 UniT-Diff를 제안한다. 11채널 태스크별 출력 공간이 레이블 범주를 물리적으로 분리하여, 설계 자체에서 태스크 간 그래디언트 부호 반전을 제거한다. SNR 적응형 태스크 컨디셔닝(SATC)은 현재 확산 타임스텝의 로그 신호 대 잡음비로 태스크 토큰을 스케일링하여, 거친 노이즈 제거 단계에서 도메인 특화 바이어스를 억제하고 신호가 명확해짐에 따라 완전한 태스크 가이던스를 복원한다. 태스크 유형 인식 조건부 드롭아웃(TTACD)은 도메인 일반화 입력에 대해 태스크 토큰을 영구적으로 제거하여, 소스 벤더 통계보다 데이터셋 간 심장 해부학 정보를 활용하는 공유 중립 경로를 통해 라우팅한다. 단일 매개변수 세트로, UniT-Diff는 세 가지 벤치마크 모두에서 독립적으로 학습된 태스크별 기준 모델을 동시에 능가한다: LA에서 +0.87%, MMWHS에서 +1.77%, MNMS에서 +0.88%.
Purpose: To evaluate the feasibility and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation and deep learning-based segmentation. Approach: We developed ChameleonNet, a framework utilizing the Contrastive Unpaired Translation (CUT) network with decoupled contrastive learning (DCL) loss to synthesize non-contrast CT from contrast CT scans. Using annotations of four heart chambers (left atrium (LA), left ventricle (LV), right atrium (RA), and right ventricle (RV)) from contrast scans, we trained a Hausdorff distance loss-enhanced nnU-Net on synthesized non-contrast images. The translation model was trained with 35,538 contrast-enhanced and 37,197 non-contrast CT slices. The segmentation model was trained with 292 synthesized non-contrast scans. Performance was evaluated using Dice similarity coefficient (DSC) and 95th Hausdorff distance (HD95) on 36 synthesized non-contrast scans, and volume agreement on 36 real non-contrast CT scans was assessed using Pearson correlation, mean absolute percentage error (MAPE), and mean percentage error (MPE). Results: The segmentation model achieved DSC of 0.94 (0.01), 0.91 (0.04), 0.92 (0.03), 0.93 (0.02), and HD95 of 3.63 (1.49), 5.74 (4.08), 5.18 (1.77), 5.51 (3.21) mm on synthesized non-contrast images for LA, LV, RA, and RV, respectively. On real non-contrast CT scans, Pearson correlations were 0.93, 0.82, 0.87, and 0.89 (all p<0.001), with MAPE ranging from 9.22% to 20.79%, and MPE ranging from -12.52% to 4.67%. Conclusions: ChameleonNet demonstrated feasibility for heart chamber segmentation from non-contrast CT without manual non-contrast annotations. However, volume errors, particularly for LV and RV, indicate that further refinement and validation are needed before clinical use.