PyTorchKR Papers

최신 논문

총 12,272편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2302.11983

Category-level Shape Estimation for Densely Cluttered Objects

Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

arXiv:2302.11951

Pixel Difference Convolutional Network for RGB-D Semantic Segmentation

RGB-D semantic segmentation can be advanced with convolutional neural networks due to the availability of Depth data. Although objects cannot be easily discriminated by just the 2D appearance, with the local pixel difference and geometric patterns in Depth, they can be well separated in some cases. Considering the fixed grid kernel structure, CNNs are limited to lack the ability to capture detailed, fine-grained information and thus cannot achieve accurate pixel-level semantic segmentation. To solve this problem, we propose a Pixel Difference Convolutional Network (PDCNet) to capture detailed intrinsic patterns by aggregating both intensity and gradient information in the local range for Depth data and global range for RGB data, respectively. Precisely, PDCNet consists of a Depth branch and an RGB branch. For the Depth branch, we propose a Pixel Difference Convolution (PDC) to consider local and detailed geometric information in Depth data via aggregating both intensity and gradient information. For the RGB branch, we contribute a lightweight Cascade Large Kernel (CLK) to extend PDC, namely CPDC, to enjoy global contexts for RGB data and further boost performance. Consequently, both modal data's local and global pixel differences are seamlessly incorporated into PDCNet during the information propagation process. Experiments on two challenging benchmark datasets, i.e., NYUDv2 and SUN RGB-D reveal that our PDCNet achieves state-of-the-art performance for the semantic segmentation task.

arXiv:2302.11806

PLU-Net: Extraction of multi-scale feature fusion

Deep learning algorithms have achieved remarkable results in medical image segmentation in recent years. These networks are unable to handle with image boundaries and details with enormous parameters, resulting in poor segmentation results. To address the issue, we develop atrous spatial pyramid pooling (ASPP) and combine it with the Squeeze-and-Excitation block (SE block), as well as present the PS module, which employs a broader and multi-scale receptive field at the network's bottom to obtain more detailed semantic information. We also propose the Local Guided block (LG block) and also its combination with the SE block to form the LS block, which can obtain more abundant local features in the feature map, so that more edge information can be retained in each down sampling process, thereby improving the performance of boundary segmentation. We propose PLU-Net and integrate our PS module and LS block into U-Net. We put our PLU-Net to the test on three benchmark datasets, and the results show that by fewer parameters and FLOPs, it outperforms on medical semantic segmentation tasks.

arXiv:2302.11802

Patch Network for medical image Segmentation

Accurate and fast segmentation of medical images is clinically essential, yet current research methods include convolutional neural networks with fast inference speed but difficulty in learning image contextual features, and transformer with good performance but high hardware requirements. In this paper, we present a Patch Network (PNet) that incorporates the Swin Transformer notion into a convolutional neural network, allowing it to gather richer contextual information while achieving the balance of speed and accuracy. We test our PNet on Polyp(CVC-ClinicDB and ETIS- LaribPolypDB), Skin(ISIC-2018 Skin lesion segmentation challenge dataset) segmentation datasets. Our PNet achieves SOTA performance in both speed and accuracy.

arXiv:2302.12242공식 코드 · Papers with Code

Side Adapter Network for Open-Vocabulary Semantic Segmentation

This paper presents a new framework for open-vocabulary semantic segmentation with the pre-trained vision-language model, named Side Adapter Network (SAN). Our approach models the semantic segmentation task as a region recognition problem. A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks. This decoupled design has the benefit CLIP in recognizing the class of mask proposals. Since the attached side network can reuse CLIP features, it can be very light. In addition, the entire network can be trained end-to-end, allowing the side network to be adapted to the frozen CLIP model, which makes the predicted mask proposals CLIP-aware. Our approach is fast, accurate, and only adds a few additional trainable parameters. We evaluate our approach on multiple semantic segmentation benchmarks. Our method significantly outperforms other counterparts, with up to 18 times fewer trainable parameters and 19 times faster inference speed. We hope our approach will serve as a solid baseline and help ease future research in open-vocabulary semantic segmentation. The code will be available at https://github.com/MendelXu/SAN.

arXiv:2302.11200

Semi-Supervised Segmentation of Multi-vendor and Multi-center Cardiac MRI using Histogram Matching

Automatic segmentation of the heart cavity is an essential task for the diagnosis of cardiac diseases. In this paper, we propose a semi-supervised segmentation setup for leveraging unlabeled data to segment Left-ventricle, Right-ventricle, and Myocardium. We utilize an enhanced version of residual U-Net architecture on a large-scale cardiac MRI dataset. Handling the class imbalanced data issue using dice loss, the enhanced supervised model is able to achieve better dice scores in comparison with a vanilla U-Net model. We applied several augmentation techniques including histogram matching to increase the performance of our model in other domains. Also, we introduce a simple but efficient semi-supervised segmentation method to improve segmentation results without the need for large labeled data. Finally, we applied our method on two benchmark datasets, STACOM2018, and M\&Ms 2020 challenges, to show the potency of the proposed model. The effectiveness of our proposed model is demonstrated by the quantitative results. The model achieves average dice scores of 0.921, 0.926, and 0.891 for Left-ventricle, Right-ventricle, and Myocardium respectively.

arXiv:2302.11524

Slim U-Net: Efficient Anatomical Feature Preserving U-net Architecture for Ultrasound Image Segmentation

We investigate the applicability of U-Net based models for segmenting Urinary Bladder (UB) in male pelvic view UltraSound (US) images. The segmentation of UB in the US image aids radiologists in diagnosing the UB. However, UB in US images has arbitrary shapes, indistinct boundaries and considerably large inter- and intra-subject variability, making segmentation a quite challenging task. Our study of the state-of-the-art (SOTA) segmentation network, U-Net, for the problem reveals that it often fails to capture the salient characteristics of UB due to the varying shape and scales of anatomy in the noisy US image. Also, U-net has an excessive number of trainable parameters, reporting poor computational efficiency during training. We propose a Slim U-Net to address the challenges of UB segmentation. Slim U-Net proposes to efficiently preserve the salient features of UB by reshaping the structure of U-Net using a less number of 2D convolution layers in the contracting path, in order to preserve and impose them on expanding path. To effectively distinguish the blurred boundaries, we propose a novel annotation methodology, which includes the background area of the image at the boundary of a marked region of interest (RoI), thereby steering the model's attention towards boundaries. In addition, we suggested a combination of loss functions for network training in the complex segmentation of UB. The experimental results demonstrate that Slim U-net is statistically superior to U-net for UB segmentation. The Slim U-net further decreases the number of trainable parameters and training time by 54% and 57.7%, respectively, compared to the standard U-Net, without compromising the segmentation accuracy.

arXiv:2302.11517공식 코드 · Papers with Code

A Global and Patch-wise Contrastive Loss for Accurate Automated Exudate Detection

Diabetic retinopathy (DR) is a leading global cause of blindness. Early detection of hard exudates plays a crucial role in identifying DR, which aids in treating diabetes and preventing vision loss. However, the unique characteristics of hard exudates, ranging from their inconsistent shapes to indistinct boundaries, pose significant challenges to existing segmentation techniques. To address these issues, we present a novel supervised contrastive learning framework to optimize hard exudate segmentation. Specifically, we introduce a patch-wise density contrasting scheme to distinguish between areas with varying lesion concentrations, and therefore improve the model's proficiency in segmenting small lesions. To handle the ambiguous boundaries, we develop a discriminative edge inspection module to dynamically analyze the pixels that lie around the boundaries and accurately delineate the exudates. Upon evaluation using the IDRiD dataset and comparison with state-of-the-art frameworks, our method exhibits its effectiveness and shows potential for computer-assisted hard exudate detection. The code to replicate experiments is available at github.com/wetang7/HECL/.

arXiv:2302.11325공식 코드 · Papers with Code

Video-SwinUNet: Spatio-temporal Deep Learning Framework for VFSS Instance Segmentation

This paper presents a deep learning framework for medical video segmentation. Convolution neural network (CNN) and transformer-based methods have achieved great milestones in medical image segmentation tasks due to their incredible semantic feature encoding and global information comprehension abilities. However, most existing approaches ignore a salient aspect of medical video data - the temporal dimension. Our proposed framework explicitly extracts features from neighbouring frames across the temporal dimension and incorporates them with a temporal feature blender, which then tokenises the high-level spatio-temporal feature to form a strong global feature encoded via a Swin Transformer. The final segmentation results are produced via a UNet-like encoder-decoder architecture. Our model outperforms other approaches by a significant margin and improves the segmentation benchmarks on the VFSS2022 dataset, achieving a dice coefficient of 0.8986 and 0.8186 for the two datasets tested. Our studies also show the efficacy of the temporal feature blending scheme and cross-dataset transferability of learned capabilities. Code and models are fully available at https://github.com/SimonZeng7108/Video-SwinUNet.

arXiv:2302.10182

PrecTime: A Deep Learning Architecture for Precise Time Series Segmentation in Industrial Manufacturing Operations

The fourth industrial revolution creates ubiquitous sensor data in production plants. To generate maximum value out of these data, reliable and precise time series-based machine learning methods like temporal neural networks are needed. This paper proposes a novel sequence-to-sequence deep learning architecture for time series segmentation called PrecTime which tries to combine the concepts and advantages of sliding window and dense labeling approaches. The general-purpose architecture is evaluated on a real-world industry dataset containing the End-of-Line testing sensor data of hydraulic pumps. We are able to show that PrecTime outperforms five implemented state-of-the-art baseline networks based on multiple metrics. The achieved segmentation accuracy of around 96% shows that PrecTime can achieve results close to human intelligence in operational state segmentation within a testing cycle.

arXiv:2302.10307공식 코드 · Papers with Code

ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency

Recently, great success has been made in learning visual representations from text supervision, facilitating the emergence of text-supervised semantic segmentation. However, existing works focus on pixel grouping and cross-modal semantic alignment, while ignoring the correspondence among multiple augmented views of the same image. To overcome such limitation, we propose multi-\textbf{View} \textbf{Co}nsistent learning (ViewCo) for text-supervised semantic segmentation. Specifically, we first propose text-to-views consistency modeling to learn correspondence for multiple views of the same input image. Additionally, we propose cross-view segmentation consistency modeling to address the ambiguity issue of text supervision by contrasting the segment features of Siamese visual encoders. The text-to-views consistency benefits the dense assignment of the visual features by encouraging different crops to align with the same text, while the cross-view segmentation consistency modeling provides additional self-supervision, overcoming the limitation of ambiguous text supervision for segmentation masks. Trained with large-scale image-text data, our model can directly segment objects of arbitrary categories in a zero-shot manner. Extensive experiments show that ViewCo outperforms state-of-the-art methods on average by up to 2.9\%, 1.6\%, and 2.4\% mIoU on PASCAL VOC2012, PASCAL Context, and COCO, respectively.

arXiv:2302.10661

Clinically Acceptable Segmentation of Organs at Risk in Cervical Cancer Radiation Treatment from Clinically Available Annotations

Deep learning models benefit from training with a large dataset (labeled or unlabeled). Following this motivation, we present an approach to learn a deep learning model for the automatic segmentation of Organs at Risk (OARs) in cervical cancer radiation treatment from a large clinically available dataset of Computed Tomography (CT) scans containing data inhomogeneity, label noise, and missing annotations. We employ simple heuristics for automatic data cleaning to minimize data inhomogeneity and label noise. Further, we develop a semi-supervised learning approach utilizing a teacher-student setup, annotation imputation, and uncertainty-guided training to learn in presence of missing annotations. Our experimental results show that learning from a large dataset with our approach yields a significant improvement in the test performance despite missing annotations in the data. Further, the contours generated from the segmentation masks predicted by our model are found to be equally clinically acceptable as manually generated contours.

arXiv:2302.10635

Semantic Segmentation of Urban Textured Meshes Through Point Sampling

Textured meshes are becoming an increasingly popular representation combining the 3D geometry and radiometry of real scenes. However, semantic segmentation algorithms for urban mesh have been little investigated and do not exploit all radiometric information. To address this problem, we adopt an approach consisting in sampling a point cloud from the textured mesh, then using a point cloud semantic segmentation algorithm on this cloud, and finally using the obtained semantic to segment the initial mesh. In this paper, we study the influence of different parameters such as the sampling method, the density of the extracted cloud, the features selected (color, normal, elevation) as well as the number of points used at each training period. Our result outperforms the state-of-the-art on the SUM dataset, earning about 4 points in OA and 18 points in mIoU.

arXiv:2302.10634

A Deep Learning-Based Fully Automated Pipeline for Regurgitant Mitral Valve Anatomy Analysis From 3D Echocardiography

Three-dimensional transesophageal echocardiography (3DTEE) is the recommended imaging technique for the assessment of mitral valve (MV) morphology and lesions in case of mitral regurgitation (MR) requiring surgical or transcatheter repair. Such assessment is key to thorough intervention planning and to intraprocedural guidance. However, it requires segmentation from 3DTEE images, which is timeconsuming, operator-dependent, and often merely qualitative. In the present work, a novel workflow to quantify the patient-specific MV geometry from 3DTEE is proposed. The developed approach relies on a 3D multi-decoder residual convolutional neural network (CNN) with a U-Net architecture for multi-class segmentation of MV annulus and leaflets. The CNN was trained and tested on a dataset comprising 55 3DTEE examinations of MR-affected patients. After training, the CNN is embedded into a fully automatic, and hence fully repeatable, pipeline that refines the predicted segmentation, detects MV anatomical landmarks and quantifies MV morphology. The trained 3D CNN achieves an average Dice score of $0.82 \pm 0.06$, mean surface distance of $0.43 \pm 0.14$ mm and 95% Hausdorff Distance (HD) of $3.57 \pm 1.56$ mm before segmentation refinement, outperforming a state-of-the-art baseline residual U-Net architecture, and provides an unprecedented multi-class segmentation of the annulus, anterior and posterior leaflet. The automatic 3D linear morphological measurements of the annulus and leaflets, specifically diameters and lengths, exhibit differences of less than 1.45 mm when compared to ground truth values. These measurements also demonstrate strong overall agreement with analyses conducted by semi-automated commercial software. The whole process requires minimal user interaction and requires approximately 15 seconds

arXiv:2302.10501

Few-Shot Point Cloud Semantic Segmentation via Contrastive Self-Supervision and Multi-Resolution Attention

This paper presents an effective few-shot point cloud semantic segmentation approach for real-world applications. Existing few-shot segmentation methods on point cloud heavily rely on the fully-supervised pretrain with large annotated datasets, which causes the learned feature extraction bias to those pretrained classes. However, as the purpose of few-shot learning is to handle unknown/unseen classes, such class-specific feature extraction in pretrain is not ideal to generalize into new classes for few-shot learning. Moreover, point cloud datasets hardly have a large number of classes due to the annotation difficulty. To address these issues, we propose a contrastive self-supervision framework for few-shot learning pretrain, which aims to eliminate the feature extraction bias through class-agnostic contrastive supervision. Specifically, we implement a novel contrastive learning approach with a learnable augmentor for a 3D point cloud to achieve point-wise differentiation, so that to enhance the pretrain with managed overfitting through the self-supervision. Furthermore, we develop a multi-resolution attention module using both the nearest and farthest points to extract the local and global point information more effectively, and a center-concentrated multi-prototype is adopted to mitigate the intra-class sparsity. Comprehensive experiments are conducted to evaluate the proposed approach, which shows our approach achieves state-of-the-art performance. Moreover, a case study on practical CAM/CAD segmentation is presented to demonstrate the effectiveness of our approach for real-world applications.

arXiv:2302.10318

Hadamard Layer to Improve Semantic Segmentation

The Hadamard Layer, a simple and computationally efficient way to improve results in semantic segmentation tasks, is presented. This layer has no free parameters that require to be trained. Therefore it does not increase the number of model parameters, and the extra computational cost is marginal. Experimental results show that the new Hadamard layer substantially improves the performance of the investigated models (variants of the Pix2Pix model). The performance's improvement can be explained by the Hadamard layer forcing the network to produce an internal encoding of the classes so that all bins are active. Therefore, the network computation is more distributed. In a sort that the Hadamard layer requires that to change the predicted class, it is necessary to modify $2^{k-1}$ bins, assuming $k$ bins in the encoding. A specific loss function allows a stable and fast training convergence.

arXiv:2302.09931

An algorithm for DC segmentation of AC power systems to mitigate electromechanical oscillations

In the last decades, various events have shown that electromechanical oscillations are a major concern for large interconnected Alternative Current (AC) power systems. DC segmentation - a method that consist in turning large AC grids into a set of asynchronous AC clusters linked by Direct Current (DC) links - is a promising solution to mitigate this and other issues. However, no systematic segmentation procedure for a given AC power system exists so far. This paper aims at filling this gap and proposes an algorithm for DC segmentation for a given AC power system to mitigate electromechanical oscillations. In this proposal, DC segmentation is implemented with High Voltage Direct Current links based on Voltage Source Converters (VSC-HVDC). The algorithm uses small-signal stability techniques and the concept of dominant inter-area oscillation paths to stop the main inter-area mode of the power system. The algorithm will be explained using a six-generator test system and it will then be used on the Nordic 44 test system. The proposed algorithm for DC segmentation has been validated by means of non-linear time-domain simulation and small-signal stability analysis (SSSA).

arXiv:2302.09785

Towards Simultaneous Segmentation of Liver Tumors and Intrahepatic Vessels via Cross-attention Mechanism

Accurate visualization of liver tumors and their surrounding blood vessels is essential for noninvasive diagnosis and prognosis prediction of tumors. In medical image segmentation, there is still a lack of in-depth research on the simultaneous segmentation of liver tumors and peritumoral blood vessels. To this end, we collect the first liver tumor, and vessel segmentation benchmark datasets containing 52 portal vein phase computed tomography images with liver, liver tumor, and vessel annotations. In this case, we propose a 3D U-shaped Cross-Attention Network (UCA-Net) that utilizes a tailored cross-attention mechanism instead of the traditional skip connection to effectively model the encoder and decoder feature. Specifically, the UCA-Net uses a channel-wise cross-attention module to reduce the semantic gap between encoder and decoder and a slice-wise cross-attention module to enhance the contextual semantic learning ability among distinct slices. Experimental results show that the proposed UCA-Net can accurately segment 3D medical images and achieve state-of-the-art performance on the liver tumor and intrahepatic vessel segmentation task.

arXiv:2302.09569

SEMI-PointRend: Improved Semiconductor Wafer Defect Classification and Segmentation as Rendering

In this study, we applied the PointRend (Point-based Rendering) method to semiconductor defect segmentation. PointRend is an iterative segmentation algorithm inspired by image rendering in computer graphics, a new image segmentation method that can generate high-resolution segmentation masks. It can also be flexibly integrated into common instance segmentation meta-architecture such as Mask-RCNN and semantic meta-architecture such as FCN. We implemented a model, termed as SEMI-PointRend, to generate precise segmentation masks by applying the PointRend neural network module. In this paper, we focus on comparing the defect segmentation predictions of SEMI-PointRend and Mask-RCNN for various defect types (line-collapse, single bridge, thin bridge, multi bridge non-horizontal). We show that SEMI-PointRend can outperforms Mask R-CNN by up to 18.8% in terms of segmentation mean average precision.

arXiv:2302.09516

A Bibliography of Multiple Sclerosis Lesions Detection Methods using Brain MRIs

Introduction: Multiple Sclerosis (MS) is a chronic disease that affects millions of people across the globe. MS can critically affect different organs of the central nervous system such as the eyes, the spinal cord, and the brain. Background: To help physicians in diagnosing MS lesions, computer-aided methods are widely used. In this regard, a considerable research has been carried out in the area of automatic detection and segmentation of MS lesions in magnetic resonance images (MRIs). Methodology: In this study, we review the different approaches that have been used in computer-aided detection and segmentation of MS lesions. Our review resulted in categorizing MS lesion segmentation approaches into six broad categories: data-driven, statistical, supervised machine learning, unsupervised machine learning, fuzzy, and deep learning-based techniques. We critically analyze the different techniques under these approaches and highlight their strengths and weaknesses. Results: From the study, we observe that a considerable amount of work, around 25% of related literature, is focused on statistical-based MS lesion segmentation techniques, followed by 21.15% for data-driven based methods, 19.23% for deep learning and 15.38% for supervised methods. Implication: The study points out the challenges/gaps to be addressed in future research. The study shows the work which has been done in last one decade in detection and segmentation of MS lesions. The results show that, in recent years, deep learning methods are outperforming all the others methods.