PyTorchKR Papers

“Image Classification”

총 18,385편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2611.24331v1cs.LG

OpenGNS: 비전, 언어, 확산 모델 전반의 그래디언트 노이즈 스케일 공개 데이터셋

OpenGNS: An Open Dataset of the Gradient Noise Scale across Vision, Language, and Diffusion Models

그래디언트 노이즈 스케일(Gradient Noise Scale, GNS) 지표는 확률적 경사 하강법에서 그래디언트의 신호 대 잡음비를 측정하며, McCandlish et al. (2018)이 보인 바와 같이 신경망 학습 중 임계 배치 크기(Critical Batch Size, CBS)를 근사하는 데 사용할 수 있다. 최전선 모델의 기술 보고서에는 등장함에도 불구하고, 현대 아키텍처 전반에 걸친 GNS 측정의 공개 데이터셋은 존재하지 않아 이 지표의 체계적 연구가 제한되어 왔다. 본 연구는 이러한 공백을 메우기 위해 세 영역, 즉 ResNet을 이용한 이미지 분류, 생성적 사전학습 트랜스포머(Generative Pre-trained Transformers, GPT)를 이용한 자기회귀 언어 모델링, 확산 트랜스포머(Diffusion Transformers, DiT)를 이용한 이미지 합성에 걸친 400개 이상의 GNS 측정 궤적으로 이루어진 데이터셋 OpenGNS를 공개한다. 이 데이터셋은 조밀한 배치 크기 및 학습률 스윕과 함께 최대 업데이트 파라미터화(Maximal Update Parameterization, muP) 하에서 학습된 11M에서 1B 파라미터의 모델을 포괄한다. 이 데이터셋을 기반으로 GNS가 하이퍼파라미터, 규모, 학습 진행에 어떻게 의존하는지, 그리고 CBS와 어떤 관계를 갖는지 연구한다. GNS가 일관된 "학습의 온도"로 거동하여, 동등한 손실에서 muP 하에 모델 크기에 걸쳐 안정적으로 유지됨을 발견하였으며, 결과는 CBS와의 관계에 과제 의존적 편향이 있음을 시사한다. 즉 분류에서는 CBS를 과대추정하고 생성 과제에서는 과소추정하는 경향이 있으며, 이는 일반적인 CBS 정의 전반에서 방향이 일관된다. 데이터셋은 https://huggingface.co/datasets/olympique-marcel/OpenGNS 에서 이용할 수 있다.

arXiv:1510.07136

Image Parsing with a Wide Range of Classes and Scene-Level Context

This paper presents a nonparametric scene parsing approach that improves the overall accuracy, as well as the coverage of foreground classes in scene images. We first improve the label likelihood estimates at superpixels by merging likelihood scores from different probabilistic classifiers. This boosts the classification performance and enriches the representation of less-represented classes. Our second contribution consists of incorporating semantic context in the parsing process through global label costs. Our method does not rely on image retrieval sets but rather assigns a global likelihood estimate to each label, which is plugged into the overall energy function. We evaluate our system on two large-scale datasets, SIFTflow and LMSun. We achieve state-of-the-art performance on the SIFTflow dataset and near-record results on LMSun.

arXiv:1901.11074

A Convolutional Neural Network for the Automatic Diagnosis of Collagen VI related Muscular Dystrophies

The development of machine learning systems for the diagnosis of rare diseases is challenging mainly due the lack of data to study them. Despite this challenge, this paper proposes a system for the Computer Aided Diagnosis (CAD) of low-prevalence, congenital muscular dystrophies from confocal microscopy images. The proposed CAD system relies on a Convolutional Neural Network (CNN) which performs an independent classification for non-overlapping patches tiling the input image, and generates an overall decision summarizing the individual decisions for the patches on the query image. This decision scheme points to the possibly problematic areas in the input images and provides a global quantitative evaluation of the state of the patients, which is fundamental for diagnosis and to monitor the efficiency of therapies.

arXiv:2111.11236v4cs.RO

나노로봇 대열: 팀원 간 통신과 영상 처리에 기반한 암의 협동 치료

Nanorobot queue: Cooperative treatment of cancer based on team member communication and image processing

나노로봇은 위내시경 등의 작업에 임상 처방으로 사용되어 왔고, 광음향 단층촬영 기술을 통해 나노로봇이 지정된 전달 지점에서 실시간으로 약물을 전달하도록 제어하는 방안도 제안되었으며, 나노로봇으로 혈액 속 '슈퍼박테리아'를 제거한 사례도 있다. 그러나 대부분의 기술은 아직 미성숙하여 효율이 낮거나 정확도가 낮거나 대량 생산이 불가능하다. 따라서 현 단계에서 암 질환을 치료하는 가장 효과적인 방법은 화학요법과 방사선요법이며, 환자는 고통받고 있으나 완치되지 못한다. 이에 본 논문은 암을 완전히 치료할 수 있는 치료 방법의 이상적인 모델, 즉 팀원 간 통신과 컴퓨터 비전 영상 분류(표적 탐지)를 통한 나노로봇 대열 기반의 협동 치료 방법을 제안한다.

arXiv:2611.10331v1cs.AI

다중 모달 암석학 이미지 분석에서의 분류 및 설명 가능성 평가

Evaluating classification and explainability in multimodal petrographic image analysis

자동화된 박편 분류기는 암석학적 식별의 핵심인 두 가지 상보적 관점, 즉 평면편광(PPL)과 직교편광(XPL)을 점점 더 결합하고 있다. 분류 정확도만으로는 모델 오류가 광물학적 관계를 반영하는지, 예측이 각 광학적 관점에 어떻게 의존하는지, 또는 기여도 지도가 모델 결정에 영향을 미치는 영역을 식별하는지를 알 수 없다. 본 연구에서는 25개 범주에 걸친 전문가 주석 입자 105,802개에 대한 이전 평가의 출력을 사용하여, 공개된 LITHOS 이중 스트림 비전 트랜스포머의 이러한 상보적 측면들을 검토한다. 분류 오류는 흔히 레이블 분류 체계를 따랐다. 잘못 분류된 사장석, 미사장석, 투명석 입자의 72.6%가 포괄적인 Feldspar 레이블로 지정되었고, 석영 계열 입자의 87.1%는 해당 계열 내의 레이블을 받았으며 정확한 레이블을 받은 비율은 66.6%였다. XPL 입력을 채널별 공간 평균으로 대체하면 25개 범주 중 20개에서 동등한 PPL 대체보다 참 클래스 확률이 더 크게 감소했으나, 세 개의 석영 범주에서는 그렇지 않았다. 다섯 가지 설명 가능한 인공지능(XAI) 기여도 절차는 동일한 입력에 대해 시각적으로 서로 다른 지도를 생성했다. 29개 박편의 입자 30개를 대상으로 한 슈퍼픽셀 삭제 분석에서 방법 대비 무작위 차이는 $-0.043$에서 $0.057$ 범위였으며, 41개의 동시 95% 신뢰구간 모두 0을 포함했다. 종합하면, 이러한 결과는 오류 구조, 모달리티 민감도, 기여도 행동이 모델 성능의 서로 구별되는 측면을 포착하며, 이는 분류 정확도나 기여도 지도의 시각적 검토만으로는 확립될 수 없음을 보여준다.

arXiv:2002.01244

Machine Learning Techniques to Detect and Characterise Whistler Radio Waves

Lightning strokes create powerful electromagnetic pulses that routinely cause very low frequency (VLF) waves to propagate across hemispheres along geomagnetic field lines. VLF antenna receivers can be used to detect these whistler waves generated by these lightning strokes. The particular time/frequency dependence of the received whistler wave enables the estimation of electron density in the plasmasphere region of the magnetosphere. Therefore the identification and characterisation of whistlers are important tasks to monitor the plasmasphere in real-time and to build large databases of events to be used for statistical studies. The current state of the art in detecting whistler is the Automatic Whistler Detection (AWD) method developed by Lichtenberger (2009). This method is based on image correlation in 2 dimensions and requires significant computing hardware situated at the VLF receiver antennas (e.g. in Antarctica). The aim of this work is to develop a machine learning-based model capable of automatically detecting whistlers in the data provided by the VLF receivers. The approach is to use a combination of image classification and localisation on the spectrogram data generated by the VLF receivers to identify and localise each whistler. The data at hand has around 2300 events identified by AWD at SANAE and Marion and will be used as training, validation, and testing data. Three detector designs have been proposed. The first one using a similar method to AWD, the second using image classification on regions of interest extracted from a spectrogram, and the last one using YOLO, the current state of the art in object detection. It has been shown that these detectors can achieve a misdetection and false alarm of less than 15% on Marion's dataset.

arXiv:2403.15378공식 코드 · Papers with Code

Long-CLIP: Unlocking the Long-Text Capability of CLIP

Contrastive Language-Image Pre-training (CLIP) has been the cornerstone for zero-shot classification, text-image retrieval, and text-image generation by aligning image and text modalities. Despite its widespread adoption, a significant limitation of CLIP lies in the inadequate length of text input. The length of the text token is restricted to 77, and an empirical study shows the actual effective length is even less than 20. This prevents CLIP from handling detailed descriptions, limiting its applications for image retrieval and text-to-image generation with extensive prerequisites. To this end, we propose Long-CLIP as a plug-and-play alternative to CLIP that supports long-text input, retains or even surpasses its zero-shot generalizability, and aligns the CLIP latent space, making it readily replace CLIP without any further adaptation in downstream frameworks. Nevertheless, achieving this goal is far from straightforward, as simplistic fine-tuning can result in a significant degradation of CLIP's performance. Moreover, substituting the text encoder with a language model supporting longer contexts necessitates pretraining with vast amounts of data, incurring significant expenses. Accordingly, Long-CLIP introduces an efficient fine-tuning solution on CLIP with two novel strategies designed to maintain the original capabilities, including (1) a knowledge-preserved stretching of positional embedding and (2) a primary component matching of CLIP features. With leveraging just one million extra long text-image pairs, Long-CLIP has shown the superiority to CLIP for about 20% in long caption text-image retrieval and 6% in traditional text-image retrieval tasks, e.g., COCO and Flickr30k. Furthermore, Long-CLIP offers enhanced capabilities for generating images from detailed text descriptions by replacing CLIP in a plug-and-play manner.

arXiv:2302.07157

Classification of Lung Pathologies in Neonates using Dual Tree Complex Wavelet Transform

Annually 8500 neonatal deaths are reported in the US due to respiratory failure. Recently, Lung Ultrasound (LUS), due to its radiation free nature, portability, and being cheaper is gaining wide acceptability as a diagnostic tool for lung conditions. However, lack of highly trained medical professionals has limited its use especially in remote areas. To address this, an automated screening system that captures characteristics of the LUS patterns can be of significant assistance to clinicians who are not experts in lung ultrasound (LUS) images. In this paper, we propose a feature extraction method designed to quantify the spatially-localized line patterns and texture patterns found in LUS images. Using the dual-tree complex wavelet transform (DTCWT) and four types of common image features we propose a method to classify the LUS images into 6 common neonatal lung conditions. These conditions are normal lung, pneumothorax (PTX), transient tachypnea of the newborn (TTN), respiratory distress syndrome (RDS), chronic lung disease (CLD) and consolidation (CON) that could be pneumonia or atelectasis. The proposed method using DTCWT decomposition extracted global statistical, grey-level co-occurrence matrix (GLCM), grey-level run length matrix (GLRLM) and linear binary pattern (LBP) features to be fed to a linear discriminative analysis (LDA) based classifier. Using 15 best DTCWT features along with 3 clinical features the proposed approach achieved a per-image classification accuracy of 92.78% with a balanced dataset containing 720 images from 24 patients and 74.39% with the larger unbalanced dataset containing 1550 images from 42 patients. Likewise, the proposed method achieved a maximum per-subject classification accuracy of 81.53% with 43 DTCWT features and 3 clinical features using the balanced dataset and 64.97% with 13 DTCWT features and 3 clinical features using the unbalanced dataset.

arXiv:2006.11397

Semantically Tied Paired Cycle Consistency for Any-Shot Sketch-based Image Retrieval

Low-shot sketch-based image retrieval is an emerging task in computer vision, allowing to retrieve natural images relevant to hand-drawn sketch queries that are rarely seen during the training phase. Related prior works either require aligned sketch-image pairs that are costly to obtain or inefficient memory fusion layer for mapping the visual information to a semantic space. In this paper, we address any-shot, i.e. zero-shot and few-shot, sketch-based image retrieval (SBIR) tasks, where we introduce the few-shot setting for SBIR. For solving these tasks, we propose a semantically aligned paired cycle-consistent generative adversarial network (SEM-PCYC) for any-shot SBIR, where each branch of the generative adversarial network maps the visual information from sketch and image to a common semantic space via adversarial training. Each of these branches maintains cycle consistency that only requires supervision at the category level, and avoids the need of aligned sketch-image pairs. A classification criteria on the generators' outputs ensures the visual to semantic space mapping to be class-specific. Furthermore, we propose to combine textual and hierarchical side information via an auto-encoder that selects discriminating side information within a same end-to-end model. Our results demonstrate a significant boost in any-shot SBIR performance over the state-of-the-art on the extended version of the challenging Sketchy, TU-Berlin and QuickDraw datasets.

arXiv:2008.09306공식 코드 · Papers with Code

Robustness and Overfitting Behavior of Implicit Background Models

In this paper, we examine the overfitting behavior of image classification models modified with Implicit Background Estimation (SCrIBE), which transforms them into weakly supervised segmentation models that provide spatial domain visualizations without affecting performance. Using the segmentation masks, we derive an overfit detection criterion that does not require testing labels. In addition, we assess the change in model performance, calibration, and segmentation masks after applying data augmentations as overfitting reduction measures and testing on various types of distorted images.

arXiv:2111.07048

Image Classification with Consistent Supporting Evidence

Adoption of machine learning models in healthcare requires end users' trust in the system. Models that provide additional supportive evidence for their predictions promise to facilitate adoption. We define consistent evidence to be both compatible and sufficient with respect to model predictions. We propose measures of model inconsistency and regularizers that promote more consistent evidence. We demonstrate our ideas in the context of edema severity grading from chest radiographs. We demonstrate empirically that consistent models provide competitive performance while supporting interpretation.

arXiv:2202.04251

Improving greedy core-set configurations for active learning with uncertainty-scaled distances

We scale perceived distances of the core-set algorithm by a factor of uncertainty and search for low-confidence configurations, finding significant improvements in sample efficiency across CIFAR10/100 and SVHN image classification, especially in larger acquisition sizes. We show the necessity of our modifications and explain how the improvement is due to a probabilistic quadratic speed-up in the convergence of core-set loss, under assumptions about the relationship of model uncertainty and misclassification.

arXiv:1704.05645

Skeleton based action recognition using translation-scale invariant image mapping and multi-scale deep cnn

This paper presents an image classification based approach for skeleton-based video action recognition problem. Firstly, A dataset independent translation-scale invariant image mapping method is proposed, which transformes the skeleton videos to colour images, named skeleton-images. Secondly, A multi-scale deep convolutional neural network (CNN) architecture is proposed which could be built and fine-tuned on the powerful pre-trained CNNs, e.g., AlexNet, VGGNet, ResNet etal.. Even though the skeleton-images are very different from natural images, the fine-tune strategy still works well. At last, we prove that our method could also work well on 2D skeleton video data. We achieve the state-of-the-art results on the popular benchmard datasets e.g. NTU RGB+D, UTD-MHAD, MSRC-12, and G3D. Especially on the largest and challenge NTU RGB+D, UTD-MHAD, and MSRC-12 dataset, our method outperforms other methods by a large margion, which proves the efficacy of the proposed method.

arXiv:1910.05770

A CNN-RNN Framework for Image Annotation from Visual Cues and Social Network Metadata

Images represent a commonly used form of visual communication among people. Nevertheless, image classification may be a challenging task when dealing with unclear or non-common images needing more context to be correctly annotated. Metadata accompanying images on social-media represent an ideal source of additional information for retrieving proper neighborhoods easing image annotation task. To this end, we blend visual features extracted from neighbors and their metadata to jointly leverage context and visual cues. Our models use multiple semantic embeddings to achieve the dual objective of being robust to vocabulary changes between train and test sets and decoupling the architecture from the low-level metadata representation. Convolutional and recurrent neural networks (CNNs-RNNs) are jointly adopted to infer similarity among neighbors and query images. We perform comprehensive experiments on the NUS-WIDE dataset showing that our models outperform state-of-the-art architectures based on images and metadata, and decrease both sensory and semantic gaps to better annotate images.

arXiv:2006.13681

Multi-view Drone-based Geo-localization via Style and Spatial Alignment

In this paper, we focus on the task of multi-view multi-source geo-localization, which serves as an important auxiliary method of GPS positioning by matching drone-view image and satellite-view image with pre-annotated GPS tag. To solve this problem, most existing methods adopt metric loss with an weighted classification block to force the generation of common feature space shared by different view points and view sources. However, these methods fail to pay sufficient attention to spatial information (especially viewpoint variances). To address this drawback, we propose an elegant orientation-based method to align the patterns and introduce a new branch to extract aligned partial feature. Moreover, we provide a style alignment strategy to reduce the variance in image style and enhance the feature unification. To demonstrate the performance of the proposed approach, we conduct extensive experiments on the large-scale benchmark dataset. The experimental results confirm the superiority of the proposed approach compared to state-of-the-art alternatives.

arXiv:2008.01183공식 코드 · Papers with Code

Weakly-Supervised Semantic Segmentation via Sub-category Exploration

Existing weakly-supervised semantic segmentation methods using image-level annotations typically rely on initial responses to locate object regions. However, such response maps generated by the classification network usually focus on discriminative object parts, due to the fact that the network does not need the entire object for optimizing the objective function. To enforce the network to pay attention to other parts of an object, we propose a simple yet effective approach that introduces a self-supervised task by exploiting the sub-category information. Specifically, we perform clustering on image features to generate pseudo sub-categories labels within each annotated parent class, and construct a sub-category objective to assign the network to a more challenging task. By iteratively clustering image features, the training process does not limit itself to the most discriminative object parts, hence improving the quality of the response maps. We conduct extensive analysis to validate the proposed method and show that our approach performs favorably against the state-of-the-art approaches.

arXiv:2009.06138공식 코드 · Papers with Code

SCOUTER: Slot Attention-based Classifier for Explainable Image Recognition

Explainable artificial intelligence has been gaining attention in the past few years. However, most existing methods are based on gradients or intermediate features, which are not directly involved in the decision-making process of the classifier. In this paper, we propose a slot attention-based classifier called SCOUTER for transparent yet accurate classification. Two major differences from other attention-based methods include: (a) SCOUTER's explanation is involved in the final confidence for each category, offering more intuitive interpretation, and (b) all the categories have their corresponding positive or negative explanation, which tells "why the image is of a certain category" or "why the image is not of a certain category." We design a new loss tailored for SCOUTER that controls the model's behavior to switch between positive and negative explanations, as well as the size of explanatory regions. Experimental results show that SCOUTER can give better visual explanations in terms of various metrics while keeping good accuracy on small and medium-sized datasets.

arXiv:2205.15955공식 코드 · Papers with Code

CropMix: Sampling a Rich Input Distribution via Multi-Scale Cropping

We present a simple method, CropMix, for the purpose of producing a rich input distribution from the original dataset distribution. Unlike single random cropping, which may inadvertently capture only limited information, or irrelevant information, like pure background, unrelated objects, etc, we crop an image multiple times using distinct crop scales, thereby ensuring that multi-scale information is captured. The new input distribution, serving as training data, useful for a number of vision tasks, is then formed by simply mixing multiple cropped views. We first demonstrate that CropMix can be seamlessly applied to virtually any training recipe and neural network architecture performing classification tasks. CropMix is shown to improve the performance of image classifiers on several benchmark tasks across-the-board without sacrificing computational simplicity and efficiency. Moreover, we show that CropMix is of benefit to both contrastive learning and masked image modeling towards more powerful representations, where preferable results are achieved when learned representations are transferred to downstream tasks. Code is available at GitHub.

arXiv:2403.02642

UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimation

Autonomous off-road navigation requires an accurate semantic understanding of the environment, often converted into a bird's-eye view (BEV) representation for various downstream tasks. While learning-based methods have shown success in generating local semantic terrain maps directly from sensor data, their efficacy in off-road environments is hindered by challenges in accurately representing uncertain terrain features. This paper presents a learning-based fusion method for generating dense terrain classification maps in BEV. By performing LiDAR-image fusion at multiple scales, our approach enhances the accuracy of semantic maps generated from an RGB image and a single-sweep LiDAR scan. Utilizing uncertainty-aware pseudo-labels further enhances the network's ability to learn reliably in off-road environments without requiring precise 3D annotations. By conducting thorough experiments using off-road driving datasets, we demonstrate that our method can improve accuracy in off-road terrains, validating its efficacy in facilitating reliable and safe autonomous navigation in challenging off-road settings.

arXiv:2503.06894

Improving cognitive diagnostics in pathology: a deep learning approach for augmenting perceptional understanding of histopathology images

In Recent Years, Digital Technologies Have Made Significant Strides In Augmenting-Human-Health, Cognition, And Perception, Particularly Within The Field Of Computational-Pathology. This Paper Presents A Novel Approach To Enhancing The Analysis Of Histopathology Images By Leveraging A Mult-modal-Model That Combines Vision Transformers (Vit) With Gpt-2 For Image Captioning. The Model Is Fine-Tuned On The Specialized Arch-Dataset, Which Includes Dense Image Captions Derived From Clinical And Academic Resources, To Capture The Complexities Of Pathology Images Such As Tissue Morphologies, Staining Variations, And Pathological Conditions. By Generating Accurate, Contextually Captions, The Model Augments The Cognitive Capabilities Of Healthcare Professionals, Enabling More Efficient Disease Classification, Segmentation, And Detection. The Model Enhances The Perception Of Subtle Pathological Features In Images That Might Otherwise Go Unnoticed, Thereby Improving Diagnostic Accuracy. Our Approach Demonstrates The Potential For Digital Technologies To Augment Human Cognitive Abilities In Medical Image Analysis, Providing Steps Toward More Personalized And Accurate Healthcare Outcomes.