PyTorchKR Papers

“Image Classification”

총 18,403편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2202.07954

Unified smoke and fire detection in an evolutionary framework with self-supervised progressive data augment

Few researches have studied simultaneous detection of smoke and flame accompanying fires due to their different physical natures that lead to uncertain fluid patterns. In this study, we collect a large image data set to re-label them as a multi-label image classification problem so as to identify smoke and flame simultaneously. In order to solve the generalization ability of the detection model on account of the movable fluid objects with uncertain shapes like fire and smoke, and their not compactible natures as well as the complex backgrounds with high variations, we propose a data augment method by random image stitch to deploy resizing, deforming, position variation, and background altering so as to enlarge the view of the learner. Moreover, we propose a self-learning data augment method by using the class activation map to extract the highly trustable region as new data source of positive examples to further enhance the data augment. By the mutual reinforcement between the data augment and the detection model that are performed iteratively, both modules make progress in an evolutionary manner. Experiments show that the proposed method can effectively improve the generalization performance of the model for concurrent smoke and fire detection.

arXiv:2208.13314

Fluorescence molecular optomic signatures improve identification of tumors in head and neck specimens

In this study, a radiomics approach was extended to optical fluorescence molecular imaging data for tissue classification, termed 'optomics'. Fluorescence molecular imaging is emerging for precise surgical guidance during head and neck squamous cell carcinoma (HNSCC) resection. However, the tumor-to-normal tissue contrast is confounded by intrinsic physiological limitations of heterogeneous expression of the target molecule, epidermal growth factor receptor (EGFR). Optomics seek to improve tumor identification by probing textural pattern differences in EGFR expression conveyed by fluorescence. A total of 1,472 standardized optomic features were extracted from fluorescence image samples. A supervised machine learning pipeline involving a support vector machine classifier was trained with 25 top-ranked features selected by minimum redundancy maximum relevance criterion. Model predictive performance was compared to fluorescence intensity thresholding method by classifying testing set image patches of resected tissue with histologically confirmed malignancy status. The optomics approach provided consistent improvement in prediction accuracy on all test set samples, irrespective of dose, compared to fluorescence intensity thresholding method (mean accuracies of 89% vs. 81%; P = 0.0072). The improved performance demonstrates that extending the radiomics approach to fluorescence molecular imaging data offers a promising image analysis technique for cancer detection in fluorescence-guided surgery.

arXiv:2212.10556공식 코드 · Papers with Code

Unleashing the Power of Visual Prompting At the Pixel Level

This paper presents a simple and effective visual prompting method for adapting pre-trained models to downstream recognition tasks. Our method includes two key designs. First, rather than directly adding together the prompt and the image, we treat the prompt as an extra and independent learnable component. We show that the strategy of reconciling the prompt and the image matters, and find that warping the prompt around a properly shrinked image empirically works the best. Second, we re-introduce two "old tricks" commonly used in building transferable adversarial examples, i.e., input diversity and gradient normalization, into visual prompting. These techniques improve optimization and enable the prompt to generalize better. We provide extensive experimental results to demonstrate the effectiveness of our method. Using a CLIP model, our prompting method sets a new record of 82.8% average accuracy across 12 popular classification datasets, substantially surpassing the prior art by +5.6%. It is worth noting that this prompting performance already outperforms linear probing by +2.1% and can even match fully fine-tuning in certain datasets. In addition, our prompting method shows competitive performance across different data scales and against distribution shifts. The code is publicly available at https://github.com/UCSC-VLAA/EVP.

arXiv:2305.10110공식 코드 · Papers with Code

Adaptive aggregation of Monte Carlo augmented decomposed filters for efficient group-equivariant convolutional neural network

Group-equivariant convolutional neural networks (G-CNN) heavily rely on parameter sharing to increase CNN's data efficiency and performance. However, the parameter-sharing strategy greatly increases the computational burden for each added parameter, which hampers its application to deep neural network models. In this paper, we address these problems by proposing a non-parameter-sharing approach for group equivariant neural networks. The proposed methods adaptively aggregate a diverse range of filters by a weighted sum of stochastically augmented decomposed filters. We give theoretical proof about how the continuous group convolution can be approximated by our methods. Our method applies to both continuous and discrete groups, where the augmentation is implemented using Monte Carlo sampling and bootstrap resampling, respectively. We demonstrate that our methods serve as an efficient extension of standard CNN. Experiments on group equivariance tests show how our methods can achieve superior performance to parameter-sharing group equivariant networks. Experiments on image classification and image denoising tasks show that in certain scenarios, with a suitable set of filter bases, our method helps improve the performance of standard CNNs and build efficient lightweight image denoising networks. The code will be available at https://github.com/ZhaoWenzhao/MCG_CNN.

arXiv:2405.09789공식 코드 · Papers with Code

LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation

Due to spatial redundancy in remote sensing images, sparse tokens containing rich information are usually involved in self-attention (SA) to reduce the overall token numbers within the calculation, avoiding the high computational cost issue in Vision Transformers. However, such methods usually obtain sparse tokens by hand-crafted or parallel-unfriendly designs, posing a challenge to reach a better balance between efficiency and performance. Different from them, this paper proposes to use learnable meta tokens to formulate sparse tokens, which effectively learn key information meanwhile improving the inference speed. Technically, the meta tokens are first initialized from image tokens via cross-attention. Then, we propose Dual Cross-Attention (DCA) to promote information exchange between image tokens and meta tokens, where they serve as query and key (value) tokens alternatively in a dual-branch structure, significantly reducing the computational complexity compared to self-attention. By employing DCA in the early stages with dense visual tokens, we obtain the hierarchical architecture LeMeViT with various sizes. Experimental results in classification and dense prediction tasks show that LeMeViT has a significant $1.7 \times$ speedup, fewer parameters, and competitive performance compared to the baseline models, and achieves a better trade-off between efficiency and performance.

arXiv:2503.11070공식 코드 · Papers with Code

Falcon: A Remote Sensing Vision-Language Foundation Model

This paper introduces a holistic vision-language foundation model tailored for remote sensing, named Falcon. Falcon offers a unified, prompt-based paradigm that effectively executes comprehensive and complex remote sensing tasks. Falcon demonstrates powerful understanding and reasoning abilities at the image, region, and pixel levels. Specifically, given simple natural language instructions and remote sensing images, Falcon can produce impressive results in text form across 14 distinct tasks, i.e., image classification, object detection, segmentation, image captioning, and etc. To facilitate Falcon's training and empower its representation capacity to encode rich spatial and semantic information, we developed Falcon_SFT, a large-scale, multi-task, instruction-tuning dataset in the field of remote sensing. The Falcon_SFT dataset consists of approximately 78 million high-quality data samples, covering 5.6 million multi-spatial resolution and multi-view remote sensing images with diverse instructions. It features hierarchical annotations and undergoes manual sampling verification to ensure high data quality and reliability. Extensive comparative experiments are conducted, which verify that Falcon achieves remarkable performance over 67 datasets and 14 tasks, despite having only 0.7B parameters. We release the complete dataset, code, and model weights at https://github.com/TianHuiLab/Falcon, hoping to help further develop the open-source community.

arXiv:2208.11776

Comprehensive Dataset of Face Manipulations for Development and Evaluation of Forensic Tools

Digital media (e.g., photographs, video) can be easily created, edited, and shared. Tools for editing digital media are capable of doing so while also maintaining a high degree of photo-realism. While many types of edits to digital media are generally benign, others can also be applied for malicious purposes. State-of-the-art face editing tools and software can, for example, artificially make a person appear to be smiling at an inopportune time, or depict authority figures as frail and tired in order to discredit individuals. Given the increasing ease of editing digital media and the potential risks from misuse, a substantial amount of effort has gone into media forensics. To this end, we created a challenge dataset of edited facial images to assist the research community in developing novel approaches to address and classify the authenticity of digital media. Our dataset includes edits applied to controlled, portrait-style frontal face images and full-scene in-the-wild images that may include multiple (i.e., more than one) face per image. The goals of our dataset is to address the following challenge questions: (1) Can we determine the authenticity of a given image (edit detection)? (2) If an image has been edited, can we \textit{localize} the edit region? (3) If an image has been edited, can we deduce (classify) what edit type was performed? The majority of research in image forensics generally attempts to answer item (1), detection. To the best of our knowledge, there are no formal datasets specifically curated to evaluate items (2) and (3), localization and classification, respectively. Our hope is that our prepared evaluation protocol will assist researchers in improving the state-of-the-art in image forensics as they pertain to these challenges.

arXiv:2401.03922공식 코드 · Papers with Code

SNeurodCNN: Structure-focused Neurodegeneration Convolutional Neural Network for Modelling and Classification of Alzheimer's Disease

Alzheimer's disease (AD), the predominant form of dementia, is a growing global challenge, emphasizing the urgent need for accurate and early diagnosis. Current clinical diagnoses rely on radiologist expert interpretation, which is prone to human error. Deep learning has thus far shown promise for early AD diagnosis. However, existing methods often overlook focal structural atrophy critical for enhanced understanding of the cerebral cortex neurodegeneration. This paper proposes a deep learning framework that includes a novel structure-focused neurodegeneration CNN architecture named SNeurodCNN and an image brightness enhancement preprocessor using gamma correction. The SNeurodCNN architecture takes as input the focal structural atrophy features resulting from segmentation of brain structures captured through magnetic resonance imaging (MRI). As a result, the architecture considers only necessary CNN components, which comprises of two downsampling convolutional blocks and two fully connected layers, for achieving the desired classification task, and utilises regularisation techniques to regularise learnable parameters. Leveraging mid-sagittal and para-sagittal brain image viewpoints from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, our framework demonstrated exceptional performance. The para-sagittal viewpoint achieved 97.8% accuracy, 97.0% specificity, and 98.5% sensitivity, while the mid-sagittal viewpoint offered deeper insights with 98.1% accuracy, 97.2% specificity, and 99.0% sensitivity. Model analysis revealed the ability of SNeurodCNN to capture the structural dynamics of mild cognitive impairment (MCI) and AD in the frontal lobe, occipital lobe, cerebellum, temporal, and parietal lobe, suggesting its potential as a brain structural change digi-biomarker for early AD diagnosis. This work can be reproduced using code we made available on GitHub.

arXiv:2407.18534공식 코드 · Papers with Code

Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers

Semantic pattern of an object point cloud is determined by its topological configuration of local geometries. Learning discriminative representations can be challenging due to large shape variations of point sets in local regions and incomplete surface in a global perspective, which can be made even more severe in the context of unsupervised domain adaptation (UDA). In specific, traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries, which greatly limits their cross-domain generalization. Recently, the transformer-based models have achieved impressive performance gain in a range of image-based tasks, benefiting from its strong generalization capability and scalability stemming from capturing long range correlation across local patches. Inspired by such successes of visual transformers, we propose a novel Relational Priors Distillation (RPD) method to extract relational priors from the well-trained transformers on massive images, which can significantly empower cross-domain representations with consistent topological priors of objects. To this end, we establish a parameter-frozen pre-trained transformer module shared between 2D teacher and 3D student models, complemented by an online knowledge distillation strategy for semantically regularizing the 3D student model. Furthermore, we introduce a novel self-supervised task centered on reconstructing masked point cloud patches using corresponding masked multi-view image features, thereby empowering the model with incorporating 3D geometric information. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification. The source code of this work is available at https://github.com/zou-longkun/RPD.git.

arXiv:2503.01785cs.CV공식 코드 · Papers with Code

Visual-RFT: Visual Reinforcement Fine-Tuning

Visual Reinforcement Fine-Tuning (Visual-RFT)은 대규모 비전-언어 모델(Large Vision-Language Models, LVLMs)의 성능을 향상시키기 위해 설계된 혁신적인 방법론이다. 본 연구는 기존의 Reinforcement Fine-Tuning (RFT) 기법을 시각적 작업에 적용하여, 데이터가 부족한 상황에서도 효과적으로 모델을 파인튜닝할 수 있는 가능성을 제시한다. 기존의 RFT는 주로 언어 모델에 국한되어 있었으나, Visual-RFT는 멀티모달 도메인에서의 활용을 통해 그 적용 범위를 확장하였다. Visual-RFT의 핵심은 LVLM을 사용하여 각 입력에 대해 여러 응답을 생성하고, 이를 기반으로 제안된 시각적 인식 검증 보상 함수를 통해 모델을 업데이트하는 것이다. 이 과정에서 Group Relative Policy Optimization (GRPO)와 같은 정책 최적화 알고리즘이 활용된다. 다양한 인식 작업에 맞춰 설계된 검증 가능한 보상 함수는 모델의 학습을 보다 효율적으로 이끌어준다. 예를 들어, 객체 탐지 작업에서는 Intersection over Union (IoU) 보상을 사용하여 성능을 극대화하였다. 실험 결과, Visual-RFT는 세밀한 이미지 분류, 몇 샷 객체 탐지, 추론 기반 작업 및 개방어휘 객체 탐지 벤치마크에서 Supervised Fine-Tuning (SFT) 기준선을 초과하는 뛰어난 성능을 보여주었다. 특히, 일회성 세밀한 이미지 분류에서는 기준선보다 24.3%의 정확도 향상을 달성하였으며, COCO 데이터셋의 두 샷 설정에서도 21.9 포인트, LVIS에서는 15.4 포인트의 성능 향상을 기록하였다. 이러한 성과는 Visual-RFT가 데이터 효율적이고 보상 기반의 접근 방식을 통해 도메인 특정 작업에 대한 추론과 적응력을 크게 향상시킬 수 있음을 시사한다. 본 연구는 LVLM의 파인튜닝에 있어 새로운 패러다임을 제시하며, 향후 다양한 멀티모달 작업에 대한 연구에 기여할 수 있는 기초 자료를 제공한다. 모든 훈련 코드와 데이터는 GitHub에 오픈 소스로 제공되어, 추가 연구와 발전을 촉진할 수 있는 기반을 마련하였다. Visual-RFT는 시각적 인식 작업에 대한 강화 학습의 가능성을 확장하며, 향후 연구에 있어 중요한 이정표가 될 것으로 기대된다.

arXiv:2607.08928v1quant-ph

효율적인 고전 시뮬레이션 가능성을 갖춘 새로운 병렬 QCNN 아키텍처

A Novel Parallel QCNN Architecture with Efficient Classical Simulability

본 연구는 MNIST(Modified National Institute of Standards and Technology) 데이터셋의 이미지 이진 분류를 위한 새로운 양자 합성곱 신경망(QCNN, Quantum Convolutional Neural Network) 구현에 대한 연구를 제시한다. 기존 QCNN 및 고전적 합성곱 신경망(CNN) 구현에서 영감을 받은 새로운 아키텍처를 사용하여, 대규모 문제에 대해 고전 컴퓨터에서 효율적으로 근사 및 시뮬레이션할 수 있는 QCNN 회로를 구현하기 위해 계층적 분할 방식을 채택하였다. 먼저 원본 이미지를 분할하여 각 프로세스가 이미지의 더 작은 부분을 처리하도록 하고, 이를 독립적인 상태로 인코딩한다. 이후 이러한 분할 영역들이 병합되고 결합되어, 두 분할 영역의 정보를 포함하는 상태를 생성하면서 프로세스 수를 절반으로 줄인다. 하나의 프로세스만 남을 때까지 이 과정을 반복한 후, 측정을 위해 단일 큐비트가 남을 때까지 상태의 차원을 축소한다. 이 방식을 통해 큐비트 수가 증가해도 하드웨어 요구사항이 지수적으로 증가할 필요 없이, 여러 프로세스를 병렬로 활용하여 대규모 QCNN 프로그램을 시뮬레이션할 수 있다. 본 연구에서는 이 방식을 사용하여 128큐비트 모델을 학습하였는데, 이는 새로운 아키텍처 없이는 어떠한 고전 슈퍼컴퓨터에서도 실행이 불가능한 규모이다. 또한 소수의 큐비트로 MNIST 데이터셋에 대한 이진 분류를 수행하도록 학습시키고 분할을 사용하지 않는 모델과 비교함으로써, 새로운 모델 아키텍처가 예측 정확도에 미치는 영향을 탐구하였다. 초기 결과에 따르면, 이 아키텍처를 통해 이미지를 더 작은 부분 이미지로 분할하더라도 모델의 성능이 저하되지 않으며, 분할 과정에서 Barren Plateau 문제를 완화하기 때문으로 보이는 성능 향상이 나타나는 경우도 있었다.

arXiv:1711.01575

Adversarial Dropout Regularization

We present a method for transferring neural representations from label-rich source domains to unlabeled target domains. Recent adversarial methods proposed for this task learn to align features across domains by fooling a special domain critic network. However, a drawback of this approach is that the critic simply labels the generated features as in-domain or not, without considering the boundaries between classes. This can lead to ambiguous features being generated near class boundaries, reducing target classification accuracy. We propose a novel approach, Adversarial Dropout Regularization (ADR), to encourage the generator to output more discriminative features for the target domain. Our key idea is to replace the critic with one that detects non-discriminative features, using dropout on the classifier network. The generator then learns to avoid these areas of the feature space and thus creates better features. We apply our ADR approach to the problem of unsupervised domain adaptation for image classification and semantic segmentation tasks, and demonstrate significant improvement over the state of the art. We also show that our approach can be used to train Generative Adversarial Networks for semi-supervised learning.

arXiv:2001.11248

Weakly Supervised Segmentation of Cracks on Solar Cells using Normalized Lp Norm

Photovoltaic is one of the most important renewable energy sources for dealing with world-wide steadily increasing energy consumption. This raises the demand for fast and scalable automatic quality management during production and operation. However, the detection and segmentation of cracks on electroluminescence (EL) images of mono- or polycrystalline solar modules is a challenging task. In this work, we propose a weakly supervised learning strategy that only uses image-level annotations to obtain a method that is capable of segmenting cracks on EL images of solar cells. We use a modified ResNet-50 to derive a segmentation from network activation maps. We use defect classification as a surrogate task to train the network. To this end, we apply normalized Lp normalization to aggregate the activation maps into single scores for classification. In addition, we provide a study how different parameterizations of the normalized Lp layer affect the segmentation performance. This approach shows promising results for the given task. However, we think that the method has the potential to solve other weakly supervised segmentation problems as well.

arXiv:2102.08602

LambdaNetworks: Modeling Long-Range Interactions Without Attention

We present lambda layers -- an alternative framework to self-attention -- for capturing long-range interactions between an input and structured contextual information (e.g. a pixel surrounded by other pixels). Lambda layers capture such interactions by transforming available contexts into linear functions, termed lambdas, and applying these linear functions to each input separately. Similar to linear attention, lambda layers bypass expensive attention maps, but in contrast, they model both content and position-based interactions which enables their application to large structured inputs such as images. The resulting neural network architectures, LambdaNetworks, significantly outperform their convolutional and attentional counterparts on ImageNet classification, COCO object detection and COCO instance segmentation, while being more computationally efficient. Additionally, we design LambdaResNets, a family of hybrid architectures across different scales, that considerably improves the speed-accuracy tradeoff of image classification models. LambdaResNets reach excellent accuracies on ImageNet while being 3.2 - 4.4x faster than the popular EfficientNets on modern machine learning accelerators. When training with an additional 130M pseudo-labeled images, LambdaResNets achieve up to a 9.5x speed-up over the corresponding EfficientNet checkpoints.

arXiv:2207.10664공식 코드 · Papers with Code

Exploring Fine-Grained Audiovisual Categorization with the SSW60 Dataset

We present a new benchmark dataset, Sapsucker Woods 60 (SSW60), for advancing research on audiovisual fine-grained categorization. While our community has made great strides in fine-grained visual categorization on images, the counterparts in audio and video fine-grained categorization are relatively unexplored. To encourage advancements in this space, we have carefully constructed the SSW60 dataset to enable researchers to experiment with classifying the same set of categories in three different modalities: images, audio, and video. The dataset covers 60 species of birds and is comprised of images from existing datasets, and brand new, expert-curated audio and video datasets. We thoroughly benchmark audiovisual classification performance and modality fusion experiments through the use of state-of-the-art transformer methods. Our findings show that performance of audiovisual fusion methods is better than using exclusively image or audio based methods for the task of video classification. We also present interesting modality transfer experiments, enabled by the unique construction of SSW60 to encompass three different modalities. We hope the SSW60 dataset and accompanying baselines spur research in this fascinating area.

arXiv:2210.09643

Improving Adversarial Robustness by Contrastive Guided Diffusion Process

Synthetic data generation has become an emerging tool to help improve the adversarial robustness in classification tasks since robust learning requires a significantly larger amount of training samples compared with standard classification tasks. Among various deep generative models, the diffusion model has been shown to produce high-quality synthetic images and has achieved good performance in improving the adversarial robustness. However, diffusion-type methods are typically slow in data generation as compared with other generative models. Although different acceleration techniques have been proposed recently, it is also of great importance to study how to improve the sample efficiency of generated data for the downstream task. In this paper, we first analyze the optimality condition of synthetic distribution for achieving non-trivial robust accuracy. We show that enhancing the distinguishability among the generated data is critical for improving adversarial robustness. Thus, we propose the Contrastive-Guided Diffusion Process (Contrastive-DP), which adopts the contrastive loss to guide the diffusion model in data generation. We verify our theoretical results using simulations and demonstrate the good performance of Contrastive-DP on image datasets.

arXiv:2311.01349공식 코드 · Papers with Code

Post-hoc Orthogonalization for Mitigation of Protected Feature Bias in CXR Embeddings

Purpose: To analyze and remove protected feature effects in chest radiograph embeddings of deep learning models. Methods: An orthogonalization is utilized to remove the influence of protected features (e.g., age, sex, race) in CXR embeddings, ensuring feature-independent results. To validate the efficacy of the approach, we retrospectively study the MIMIC and CheXpert datasets using three pre-trained models, namely a supervised contrastive, a self-supervised contrastive, and a baseline classifier model. Our statistical analysis involves comparing the original versus the orthogonalized embeddings by estimating protected feature influences and evaluating the ability to predict race, age, or sex using the two types of embeddings. Results: Our experiments reveal a significant influence of protected features on predictions of pathologies. Applying orthogonalization removes these feature effects. Apart from removing any influence on pathology classification, while maintaining competitive predictive performance, orthogonalized embeddings further make it infeasible to directly predict protected attributes and mitigate subgroup disparities. Conclusion: The presented work demonstrates the successful application and evaluation of the orthogonalization technique in the domain of chest X-ray image classification.

arXiv:2311.03052

MixUp-MIL: A Study on Linear & Multilinear Interpolation-Based Data Augmentation for Whole Slide Image Classification

For classifying digital whole slide images in the absence of pixel level annotation, typically multiple instance learning methods are applied. Due to the generic applicability, such methods are currently of very high interest in the research community, however, the issue of data augmentation in this context is rarely explored. Here we investigate linear and multilinear interpolation between feature vectors, a data augmentation technique, which proved to be capable of improving the generalization performance classification networks and also for multiple instance learning. Experiments, however, have been performed on only two rather small data sets and one specific feature extraction approach so far and a strong dependence on the data set has been identified. Here we conduct a large study incorporating 10 different data set configurations, two different feature extraction approaches (supervised and self-supervised), stain normalization and two multiple instance learning architectures. The results showed an extraordinarily high variability in the effect of the method. We identified several interesting aspects to bring light into the darkness and identified novel promising fields of research.

arXiv:2411.11939

Fair Distillation: Teaching Fairness from Biased Teachers in Medical Imaging

Deep learning has achieved remarkable success in image classification and segmentation tasks. However, fairness concerns persist, as models often exhibit biases that disproportionately affect demographic groups defined by sensitive attributes such as race, gender, or age. Existing bias-mitigation techniques, including Subgroup Re-balancing, Adversarial Training, and Domain Generalization, aim to balance accuracy across demographic groups, but often fail to simultaneously improve overall accuracy, group-specific accuracy, and fairness due to conflicts among these interdependent objectives. We propose the Fair Distillation (FairDi) method, a novel fairness approach that decomposes these objectives by leveraging biased ``teacher'' models, each optimized for a specific demographic group. These teacher models then guide the training of a unified ``student'' model, which distills their knowledge to maximize overall and group-specific accuracies, while minimizing inter-group disparities. Experiments on medical imaging datasets show that FairDi achieves significant gains in both overall and group-specific accuracy, along with improved fairness, compared to existing methods. FairDi is adaptable to various medical tasks, such as classification and segmentation, and provides an effective solution for equitable model performance.

arXiv:2107.11574

Reconstructing Images of Two Adjacent Objects through Scattering Medium Using Generative Adversarial Network

Reconstruction of image by using convolutional neural networks (CNNs) has been vigorously studied in the last decade. Until now, there have being developed several techniques for imaging of a single object through scattering medium by using neural networks, however how to reconstruct images of more than one object simultaneously seems hard to realize. In this paper, we demonstrate an approach by using generative adversarial network (GAN) to reconstruct images of two adjacent objects through scattering media. We construct an imaging system for imaging of two adjacent objects behind the scattering media. In general, as the light field of two adjacent object images pass through the scattering slab, a speckle pattern is obtained. The designed adversarial network, which is called as YGAN, is employed to reconstruct the images simultaneously. It is shown that based on the trained YGAN, we can reconstruct images of two adjacent objects from one speckle pattern with high fidelity. In addition, we study the influence of the object image types, and the distance between the two adjacent objects on the fidelity of the reconstructed images. Moreover even if another scattering medium is inserted between the two objects, we can also reconstruct the images of two objects from a speckle with high quality. The technique presented in this work can be used for applications in areas of medical image analysis, such as medical image classification, segmentation, and studies of multi-object scattering imaging etc.