PyTorchKR Papers

“Image Classification”

총 18,351편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2609.10778v1cs.LG

반사실적 주변화: 방해 변수에 대한 강건성 평가 프레임워크

Counterfactual Marginalisation: Framework for Evaluating Robustness to Nuisance Variables

기계학습 모델은 인구통계학적 또는 영상 획득 관련 지름길에 의존하면서도 강한 테스트 성능을 달성할 수 있다. 우리는 그러한 변수에 대한 분류 모델의 강건성을 평가하기 위한 테스트 시점 평가 절차로서 반사실적(CF) 주변화를 제안한다. CF 이미지 생성기가 주어지면, 나이나 성별과 같은 방해 부모 변수에 개입하고, 각 테스트 이미지의 CF 버전을 생성한 뒤, 목표 개입 분포에 대해 예측을 평균한다. 이는 인구통계학적 효과를 주변화하면서 환자 특이적 잠재 정보를 보존하는, 개입을 인식하는 예측을 만든다. 우리는 이러한 예측을 사용해 CF 위험, 교정(calibration), 안정성, 최악 경우 민감도에 대한 지표를 정의한다. 정량적 강건성 평가를 위해 이 프레임워크의 유용성을 입증한다.

arXiv:2108.01625

From augmented microscopy to the topological transformer: a new approach in cell image analysis for Alzheimer's research

Cell image analysis is crucial in Alzheimer's research to detect the presence of A$\beta$ protein inhibiting cell function. Deep learning speeds up the process by making only low-level data sufficient for fruitful inspection. We first found Unet is most suitable in augmented microscopy by comparing performance in multi-class semantics segmentation. We develop the augmented microscopy method to capture nuclei in a brightfield image and the transformer using Unet model to convert an input image into a sequence of topological information. The performance regarding Intersection-over-Union is consistent concerning the choice of image preprocessing and ground-truth generation. Training model with data of a specific cell type demonstrates transfer learning applies to some extent. The topological transformer aims to extract persistence silhouettes or landscape signatures containing geometric information of a given image of cells. This feature extraction facilitates studying an image as a collection of one-dimensional data, substantially reducing computational costs. Using the transformer, we attempt grouping cell images by their cell type relying solely on topological features. Performances of the transformers followed by SVM, XGBoost, LGBM, and simple convolutional neural network classifiers are inferior to the conventional image classification. However, since this research initiates a new perspective in biomedical research by combining deep learning and topology for image analysis, we speculate follow-up investigation will reinforce our genuine regime.

arXiv:1909.12612

Biomedical Image Segmentation by Retina-like Sequential Attention Mechanism Using Only A Few Training Images

In this paper we propose a novel deep learning-based algorithm for biomedical image segmentation which uses a sequential attention mechanism able to shift the focus of attention across the image in a selective way, allowing subareas which are more difficult to classify to be processed at increased resolution. The spatial distribution of class information in each subarea is learned using a retina-like representation where resolution decreases with distance from the center of attention. The final segmentation is achieved by averaging class predictions over overlapping subareas, utilizing the power of ensemble learning to increase segmentation accuracy. Experimental results for semantic segmentation task for which only a few training images are available show that a CNN using the proposed method outperforms both a patch-based classification CNN and a fully convolutional-based method.

arXiv:2103.14339공식 코드 · Papers with Code

MedSelect: Selective Labeling for Medical Image Classification Combining Meta-Learning with Deep Reinforcement Learning

We propose a selective learning method using meta-learning and deep reinforcement learning for medical image interpretation in the setting of limited labeling resources. Our method, MedSelect, consists of a trainable deep learning selector that uses image embeddings obtained from contrastive pretraining for determining which images to label, and a non-parametric selector that uses cosine similarity to classify unseen images. We demonstrate that MedSelect learns an effective selection strategy outperforming baseline selection strategies across seen and unseen medical conditions for chest X-ray interpretation. We also perform an analysis of the selections performed by MedSelect comparing the distribution of latent embeddings and clinical features, and find significant differences compared to the strongest performing baseline. We believe that our method may be broadly applicable across medical imaging settings where labels are expensive to acquire.

arXiv:2111.07624공식 코드 · Papers with Code

Attention Mechanisms in Computer Vision: A Survey

Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multi-modal tasks and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention and branch attention; a related repository https://github.com/MenghaoGuo/Awesome-Vision-Attentions is dedicated to collecting related work. We also suggest future directions for attention mechanism research.

arXiv:2212.02400공식 코드 · Papers with Code

Location-Aware Self-Supervised Transformers for Semantic Segmentation

Pixel-level labels are particularly expensive to acquire. Hence, pretraining is a critical step to improve models on a task like semantic segmentation. However, prominent algorithms for pretraining neural networks use image-level objectives, e.g. image classification, image-text alignment a la CLIP, or self-supervised contrastive learning. These objectives do not model spatial information, which might be sub-optimal when finetuning on downstream tasks with spatial reasoning. In this work, we pretrain network with a location-aware (LOCA) self-supervised method which fosters the emergence of strong dense features. Specifically, we use both a patch-level clustering scheme to mine dense pseudo-labels and a relative location prediction task to encourage learning about object parts and their spatial arrangements. Our experiments show that LOCA pretraining leads to representations that transfer competitively to challenging and diverse semantic segmentation datasets.

arXiv:2407.09551

Diminishing Stereotype Bias in Image Generation Model using Reinforcemenlent Learning Feedback

This study addresses gender bias in image generation models using Reinforcement Learning from Artificial Intelligence Feedback (RLAIF) with a novel Denoising Diffusion Policy Optimization (DDPO) pipeline. By employing a pretrained stable diffusion model and a highly accurate gender classification Transformer, the research introduces two reward functions: Rshift for shifting gender imbalances, and Rbalance for achieving and maintaining gender balance. Experiments demonstrate the effectiveness of this approach in mitigating bias without compromising image quality or requiring additional data or prompt modifications. While focusing on gender bias, this work establishes a foundation for addressing various forms of bias in AI systems, emphasizing the need for responsible AI development. Future research directions include extending the methodology to other bias types, enhancing the RLAIF pipeline's robustness, and exploring multi-prompt fine-tuning to further advance fairness and inclusivity in AI.

arXiv:2607.19315v2cs.CV

잠재 공간에서의 ERank: 이미지 복잡도 및 풍부도 측정

ERank in Latent Space as an Image-Complexity and Richness Measure

우리는 이미지의 깊은 특징 맵 채널 공분산의 유효 순위(effective rank, ERank)를 시각적 풍부도에 대한 샘플별·레이블 없는 측정치로 제안하며, 이는 고정된 사전학습 인코더에 대한 단일 순전파로 계산된다. ERank는 이미지가 활성화하는 비상관 채널 방향의 수를 세며, 노이즈 하에서의 동작을 포함하여 그 특성을 분석한다. 실험적으로 ERank는 이미지를 단순한 것에서 시각적으로 풍부한 것까지 순서화하고, 코덱 비트레이트, 선명도, 에지 밀도와 상관하며, IC9600의 인간 복잡도 주석과 $r = 0.72$의 상관을 보인다. 데이터 선별 기준으로, 저 ERank 샘플을 제거하면 초해상도(super-resolution)가 향상되고 고 ERank 샘플을 제거하면 OCR이 향상되며, 이는 사전학습과 파인튜닝 모두에서 성립한다. 반면 선별은 분류, 분할(segmentation), 노이즈 제거(denoising)에는 도움이 되지 않는다. 따라서 ERank는 저비용의 풍부도 신호로, 입력 풍부도가 과제 난이도를 좌우할 때 정확히 유용하다.

arXiv:1909.10708

Unsupervised Deep Features for Privacy Image Classification

Sharing images online poses security threats to a wide range of users due to the unawareness of privacy information. Deep features have been demonstrated to be a powerful representation for images. However, deep features usually suffer from the issues of a large size and requiring a huge amount of data for fine-tuning. In contrast to normal images (e.g., scene images), privacy images are often limited because of sensitive information. In this paper, we propose a novel approach that can work on limited data and generate deep features of smaller size. For training images, we first extract the initial deep features from the pre-trained model and then employ the K-means clustering algorithm to learn the centroids of these initial deep features. We use the learned centroids from training features to extract the final features for each testing image and encode our final features with the triangle encoding. To improve the discriminability of the features, we further perform the fusion of two proposed unsupervised deep features obtained from different layers. Experimental results show that the proposed features outperform state-of-the-art deep features, in terms of both classification accuracy and testing time.

arXiv:2001.06268공식 코드 · Papers with Code

Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network

Recent studies in image classification have demonstrated a variety of techniques for improving the performance of Convolutional Neural Networks (CNNs). However, attempts to combine existing techniques to create a practical model are still uncommon. In this study, we carry out extensive experiments to validate that carefully assembling these techniques and applying them to basic CNN models (e.g. ResNet and MobileNet) can improve the accuracy and robustness of the models while minimizing the loss of throughput. Our proposed assembled ResNet-50 shows improvements in top-1 accuracy from 76.3\% to 82.78\%, mCE from 76.0\% to 48.9\% and mFR from 57.7\% to 32.3\% on ILSVRC2012 validation set. With these improvements, inference throughput only decreases from 536 to 312. To verify the performance improvement in transfer learning, fine grained classification and image retrieval tasks were tested on several public datasets and showed that the improvement to backbone network performance boosted transfer learning performance significantly. Our approach achieved 1st place in the iFood Competition Fine-Grained Visual Recognition at CVPR 2019, and the source code and trained models are available at https://github.com/clovaai/assembled-cnn

arXiv:2009.14657

CAD Applications and Emerging Research Potential in Medical Imaging

Computer Aided Detection (CAD) is a valuable technique for precisely interpreting medical images and it has a global business opportunity of about USD 1.8 billion. The current aspects with reference to the four sub stages such as image pre-processing, segmentation, feature extraction and classification and the future scope of CAD in medical imaging has been discussed in this paper. Many reviewers have emphasized the need for synergy between engineers and medical professionals for successful development of CAD systems and the current work is a move in that direction. The engineering aspects of the above four stages in four imaging modalities viz. computed tomography, magnetic resonance imaging, mammography and bone scintigraphy used in the diagnosis of five critical diseases have been discussed with a clinical background. Automatic classification of image can play an important role in preliminary screening of very critical ailments bringing down the cost of health care. Another recent advancement is using artificial intelligence and machine learning techniques. This paper reviews these engineering aspects with a view to explore the opportunities to researchers as well as the medical industry to offer affordable medical services with accessibility in even remote locations.

arXiv:2304.06258공식 코드 · Papers with Code

MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imaging

Recent applications of deep convolutional neural networks in medical imaging raise concerns about their interpretability. While most explainable deep learning applications use post hoc methods (such as GradCAM) to generate feature attribution maps, there is a new type of case-based reasoning models, namely ProtoPNet and its variants, which identify prototypes during training and compare input image patches with those prototypes. We propose the first medical prototype network (MProtoNet) to extend ProtoPNet to brain tumor classification with 3D multi-parametric magnetic resonance imaging (mpMRI) data. To address different requirements between 2D natural images and 3D mpMRIs especially in terms of localizing attention regions, a new attention module with soft masking and online-CAM loss is introduced. Soft masking helps sharpen attention maps, while online-CAM loss directly utilizes image-level labels when training the attention module. MProtoNet achieves statistically significant improvements in interpretability metrics of both correctness and localization coherence (with a best activation precision of $0.713\pm0.058$) without human-annotated labels during training, when compared with GradCAM and several ProtoPNet variants. The source code is available at https://github.com/aywi/mprotonet.

arXiv:2311.07634

ActiveDC: Distribution Calibration for Active Finetuning

The pretraining-finetuning paradigm has gained popularity in various computer vision tasks. In this paradigm, the emergence of active finetuning arises due to the abundance of large-scale data and costly annotation requirements. Active finetuning involves selecting a subset of data from an unlabeled pool for annotation, facilitating subsequent finetuning. However, the use of a limited number of training samples can lead to a biased distribution, potentially resulting in model overfitting. In this paper, we propose a new method called ActiveDC for the active finetuning tasks. Firstly, we select samples for annotation by optimizing the distribution similarity between the subset to be selected and the entire unlabeled pool in continuous space. Secondly, we calibrate the distribution of the selected samples by exploiting implicit category information in the unlabeled pool. The feature visualization provides an intuitive sense of the effectiveness of our approach to distribution calibration. We conducted extensive experiments on three image classification datasets with different sampling ratios. The results indicate that ActiveDC consistently outperforms the baseline performance in all image classification tasks. The improvement is particularly significant when the sampling ratio is low, with performance gains of up to 10%. Our code will be released.

arXiv:2403.10912공식 코드 · Papers with Code

Automatic location detection based on deep learning

The proliferation of digital images and the advancements in deep learning have paved the way for innovative solutions in various domains, especially in the field of image classification. Our project presents an in-depth study and implementation of an image classification system specifically tailored to identify and classify images of Indian cities. Drawing from an extensive dataset, our model classifies images into five major Indian cities: Ahmedabad, Delhi, Kerala, Kolkata, and Mumbai to recognize the distinct features and characteristics of each city/state. To achieve high precision and recall rates, we adopted two approaches. The first, a vanilla Convolutional Neural Network (CNN) and then we explored the power of transfer learning by leveraging the VGG16 model. The vanilla CNN achieved commendable accuracy and the VGG16 model achieved a test accuracy of 63.6%. Evaluations highlighted the strengths and potential areas of improvement, positioning our model as not only competitive but also scalable for broader applications. With an emphasis on open-source ethos, our work aims to contribute to the community, encouraging further development and diverse applications. Our findings demonstrate the potential applications in tourism, urban planning, and even real-time location identification systems, among others.

arXiv:2406.00672

Task-oriented Embedding Counts: Heuristic Clustering-driven Feature Fine-tuning for Whole Slide Image Classification

In the field of whole slide image (WSI) classification, multiple instance learning (MIL) serves as a promising approach, commonly decoupled into feature extraction and aggregation. In this paradigm, our observation reveals that discriminative embeddings are crucial for aggregation to the final prediction. Among all feature updating strategies, task-oriented ones can capture characteristics specifically for certain tasks. However, they can be prone to overfitting and contaminated by samples assigned with noisy labels. To address this issue, we propose a heuristic clustering-driven feature fine-tuning method (HC-FT) to enhance the performance of multiple instance learning by providing purified positive and hard negative samples. Our method first employs a well-trained MIL model to evaluate the confidence of patches. Then, patches with high confidence are marked as positive samples, while the remaining patches are used to identify crucial negative samples. After two rounds of heuristic clustering and selection, purified positive and hard negative samples are obtained to facilitate feature fine-tuning. The proposed method is evaluated on both CAMELYON16 and BRACS datasets, achieving an AUC of 97.13% and 85.85%, respectively, consistently outperforming all compared methods.

arXiv:2409.10286

Enhancing Image Classification in Small and Unbalanced Datasets through Synthetic Data Augmentation

Accurate and robust medical image classification is a challenging task, especially in application domains where available annotated datasets are small and present high imbalance between target classes. Considering that data acquisition is not always feasible, especially for underrepresented classes, our approach introduces a novel synthetic augmentation strategy using class-specific Variational Autoencoders (VAEs) and latent space interpolation to improve discrimination capabilities. By generating realistic, varied synthetic data that fills feature space gaps, we address issues of data scarcity and class imbalance. The method presented in this paper relies on the interpolation of latent representations within each class, thus enriching the training set and improving the model's generalizability and diagnostic accuracy. The proposed strategy was tested in a small dataset of 321 images created to train and validate an automatic method for assessing the quality of cleanliness of esophagogastroduodenoscopy images. By combining real and synthetic data, an increase of over 18\% in the accuracy of the most challenging underrepresented class was observed. The proposed strategy not only benefited the underrepresented class but also led to a general improvement in other metrics, including a 6\% increase in global accuracy and precision.

arXiv:2412.02275

PCIM: Learning Pixel Attributions via Pixel-wise Channel Isolation Mixing in High Content Imaging

Deep Neural Networks (DNNs) have shown remarkable success in various computer vision tasks. However, their black-box nature often leads to difficulty in interpreting their decisions, creating an unfilled need for methods to explain the decisions, and ultimately forming a barrier to their wide acceptance especially in biomedical applications. This work introduces a novel method, Pixel-wise Channel Isolation Mixing (PCIM), to calculate pixel attribution maps, highlighting the image parts most crucial for a classification decision but without the need to extract internal network states or gradients. Unlike existing methods, PCIM treats each pixel as a distinct input channel and trains a blending layer to mix these pixels, reflecting specific classifications. This unique approach allows the generation of pixel attribution maps for each image, but agnostic to the choice of the underlying classification network. Benchmark testing on three application relevant, diverse high content Imaging datasets show state-of-the-art performance, particularly for model fidelity and localization ability in both, fluorescence and bright field High Content Imaging. PCIM contributes as a unique and effective method for creating pixel-level attribution maps from arbitrary DNNs, enabling interpretability and trust.

arXiv:2405.03486

UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Images

With the advent of text-to-image models and concerns about their misuse, developers are increasingly relying on image safety classifiers to moderate their generated unsafe images. Yet, the performance of current image safety classifiers remains unknown for both real-world and AI-generated images. In this work, we propose UnsafeBench, a benchmarking framework that evaluates the effectiveness and robustness of image safety classifiers, with a particular focus on the impact of AI-generated images on their performance. First, we curate a large dataset of 10K real-world and AI-generated images that are annotated as safe or unsafe based on a set of 11 unsafe categories of images (sexual, violent, hateful, etc.). Then, we evaluate the effectiveness and robustness of five popular image safety classifiers, as well as three classifiers that are powered by general-purpose visual language models. Our assessment indicates that existing image safety classifiers are not comprehensive and effective enough to mitigate the multifaceted problem of unsafe images. Also, there exists a distribution shift between real-world and AI-generated images in image qualities, styles, and layouts, leading to degraded effectiveness and robustness. Motivated by these findings, we build a comprehensive image moderation tool called PerspectiveVision, which addresses the main drawbacks of existing classifiers with improved effectiveness and robustness, especially on AI-generated images. UnsafeBench and PerspectiveVision can aid the research community in better understanding the landscape of image safety classification in the era of generative AI.

arXiv:2608.24935v1cs.CV

상업 과수원에서 조기 단계 해부학적 청과 분류를 위한 경량 다중모달 비전-언어 프레임워크

A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards

로봇 적화, 작물 부하 관리, 기타 정밀 과수원 작업을 위해 calyx(과받침), 소과 본체, peduncle(과경) 등 조기 단계 사과 소과의 해부학적 구조를 정확히 식별하는 것은 필수적이다. 본 연구는 복잡한 과수원 환경에서 세밀한 소과 해부학 분류를 위해 TinyCLIP을 적용한 경량 다중모달 비전-언어 프레임워크를 제안한다. Scilate 및 Scifresh 사과 과수원에서 수집한 600장의 고해상도 RGB 이미지 데이터셋을 224×224 이미지 패치로 변환하고 세 가지 해부학적 클래스에 대해 주석을 부여했다. ``a photo of a class,''와 같은 도메인 특화 언어 프롬프트를 사용하여 과수원 영상과 원예학적 구조 간의 다중모달 정렬(alignment)을 유도했다. stride 112픽셀의 슬라이딩 윈도우 추론 전략으로 패치 수준 예측을 공간 히트맵에 집계하여, 로봇 적화에 관련된 소과 구성요소의 해석 가능한 전체 이미지 국소화를 가능하게 한다. NVIDIA T4 GPU에서 패치 수준 평가 결과, calyx·소과·peduncle에 대한 F1 점수는 각각 0.95, 0.98, 0.85였으며 macro-F1은 0.93이었다. 배포 지향 최적화를 위해 ONNX와 TensorRT를 사용하여 NVIDIA Jetson 하드웨어에서 효율적인 추론을 가능하게 했고, INT8 양자화에서도 정확도를 유지했으며, 약 127–137MB 규모의 모델로 밀리초 단위의 패치 추론을 지원했다. 이 결과는 경량 비전-언어 모델이 자동화된 소과 분석 및 향후 로봇 적화 시스템을 위한 해석 가능하고 엣지 배포 가능한 지각(perception)을 제공할 수 있음을 보여준다. 소스 코드와 구현 세부사항은 https://github.com/WilliamBu1/A-Lightweight-Vision-Language-Model-for-Early-Stage-Fruitlet-Classification-in-Apple-Orchards 에서 공개적으로 제공된다.

arXiv:2104.14994공식 코드 · Papers with Code

GeoWINE: Geolocation based Wiki, Image,News and Event Retrieval

In the context of social media, geolocation inference on news or events has become a very important task. In this paper, we present the GeoWINE (Geolocation-based Wiki-Image-News-Event retrieval) demonstrator, an effective modular system for multimodal retrieval which expects only a single image as input. The GeoWINE system consists of five modules in order to retrieve related information from various sources. The first module is a state-of-the-art model for geolocation estimation of images. The second module performs a geospatial-based query for entity retrieval using the Wikidata knowledge graph. The third module exploits four different image embedding representations, which are used to retrieve most similar entities compared to the input image. The embeddings are derived from the tasks of geolocation estimation, place recognition, ImageNet-based image classification, and their combination. The last two modules perform news and event retrieval from EventRegistry and the Open Event Knowledge Graph (OEKG). GeoWINE provides an intuitive interface for end-users and is insightful for experts for reconfiguration to individual setups. The GeoWINE achieves promising results in entity label prediction for images on Google Landmarks dataset. The demonstrator is publicly available at http://cleopatra.ijs.si/geowine/.