PyTorchKR Papers

“Object Detection”

총 13,455편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2303.12621

OcTr: Octree-based Transformer for 3D Object Detection

A key challenge for LiDAR-based 3D object detection is to capture sufficient features from large scale 3D scenes especially for distant or/and occluded objects. Albeit recent efforts made by Transformers with the long sequence modeling capability, they fail to properly balance the accuracy and efficiency, suffering from inadequate receptive fields or coarse-grained holistic correlations. In this paper, we propose an Octree-based Transformer, named OcTr, to address this issue. It first constructs a dynamic octree on the hierarchical feature pyramid through conducting self-attention on the top level and then recursively propagates to the level below restricted by the octants, which captures rich global context in a coarse-to-fine manner while maintaining the computational complexity under control. Furthermore, for enhanced foreground perception, we propose a hybrid positional embedding, composed of the semantic-aware positional embedding and attention mask, to fully exploit semantic and geometry clues. Extensive experiments are conducted on the Waymo Open Dataset and KITTI Dataset, and OcTr reaches newly state-of-the-art results.

arXiv:2403.09359공식 코드 · Papers with Code

D3T: Distinctive Dual-Domain Teacher Zigzagging Across RGB-Thermal Gap for Domain-Adaptive Object Detection

Domain adaptation for object detection typically entails transferring knowledge from one visible domain to another visible domain. However, there are limited studies on adapting from the visible to the thermal domain, because the domain gap between the visible and thermal domains is much larger than expected, and traditional domain adaptation can not successfully facilitate learning in this situation. To overcome this challenge, we propose a Distinctive Dual-Domain Teacher (D3T) framework that employs distinct training paradigms for each domain. Specifically, we segregate the source and target training sets for building dual-teachers and successively deploy exponential moving average to the student model to individual teachers of each domain. The framework further incorporates a zigzag learning method between dual teachers, facilitating a gradual transition from the visible to thermal domains during training. We validate the superiority of our method through newly designed experimental protocols with well-known thermal datasets, i.e., FLIR and KAIST. Source code is available at https://github.com/EdwardDo69/D3T .

arXiv:2409.05425공식 코드 · Papers with Code

Distribution Discrepancy and Feature Heterogeneity for Active 3D Object Detection

LiDAR-based 3D object detection is a critical technology for the development of autonomous driving and robotics. However, the high cost of data annotation limits its advancement. We propose a novel and effective active learning (AL) method called Distribution Discrepancy and Feature Heterogeneity (DDFH), which simultaneously considers geometric features and model embeddings, assessing information from both the instance-level and frame-level perspectives. Distribution Discrepancy evaluates the difference and novelty of instances within the unlabeled and labeled distributions, enabling the model to learn efficiently with limited data. Feature Heterogeneity ensures the heterogeneity of intra-frame instance features, maintaining feature diversity while avoiding redundant or similar instances, thus minimizing annotation costs. Finally, multiple indicators are efficiently aggregated using Quantile Transform, providing a unified measure of informativeness. Extensive experiments demonstrate that DDFH outperforms the current state-of-the-art (SOTA) methods on the KITTI and Waymo datasets, effectively reducing the bounding box annotation cost by 56.3% and showing robustness when working with both one-stage and two-stage models.

arXiv:2607.25455v1cs.CV

고대 동전의 명문 판독: 새로운 로마 공화정 데이터셋에서의 문자 인식을 위한 객체 검출 접근법

Reading Legends on Ancient Coins: An Object Detection Approach for Character Recognition on a Novel Roman Republican Dataset

고대 동전의 주조 시기와 발행처를 올바르게 분류(classification)할 때, '레전드(legend)'라고도 알려진 동전의 텍스트 명문은 무엇보다 중요합니다. 이러한 명문은 오늘날 영어에서도 사용되는 알파벳 또는 문자로 구성되어 있습니다. 본 논문은 심층 학습(deep learning) 기반 객체 검출(object detection) 전략을 통해 고대 로마 공화정 동전의 이미지 기반 문자 인식을 다룹니다. 그러나 이 동전들의 명문은 불균일한 배치, 원시적인 각인 기술, 마모 및 손상으로 인해 변동성이 매우 큽니다. 조명, 방향, 크기(scale)와 같은 일관되지 않은 촬영 조건 역시 추가적인 난제입니다. 이를 해결하기 위해, 본 연구에서는 21개 문자 라벨로 수작업 주석을 달아 총 38,808개의 어노테이션을 포함하는 5,654장의 로마 공화정 동전 이미지로 구성된 새로운 대규모 데이터셋(dataset)을 구축했습니다. 인식을 위해 YOLO(You Only Look Once) 계열 모델인 YOLOv3, v4, v5, v7, v8을 사용합니다. 실험 결과, YOLOv7-Large가 90.4%의 가장 높은 mAP50을 달성했으며, YOLOv7-Extended와 YOLOv7-xl이 각각 90.2%와 90.1%로 그 뒤를 이었습니다.

arXiv:2012.02218

Traffic Surveillance using Vehicle License Plate Detection and Recognition in Bangladesh

Computer vision coupled with Deep Learning (DL) techniques bring out a substantial prospect in the field of traffic control, monitoring and law enforcing activities. This paper presents a YOLOv4 object detection model in which the Convolutional Neural Network (CNN) is trained and tuned for detecting the license plate of the vehicles of Bangladesh and recognizing characters using tesseract from the detected license plates. Here we also present a Graphical User Interface (GUI) based on Tkinter, a python package. The license plate detection model is trained with mean average precision (mAP) of 90.50% and performed in a single TESLA T4 GPU with an average of 14 frames per second (fps) on real time video footage.

arXiv:2203.10202공식 코드 · Papers with Code

Relationformer: A Unified Framework for Image-to-Graph Generation

A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blood-vessel network extraction, or scene graph generation. Traditionally, image-to-graph generation is addressed with a two-stage approach consisting of object detection followed by a separate relation prediction, which prevents simultaneous object-relation interaction. This work proposes a unified one-stage transformer-based framework, namely Relationformer, that jointly predicts objects and their relations. We leverage direct set-based object prediction and incorporate the interaction among the objects to learn an object-relation representation jointly. In addition to existing [obj]-tokens, we propose a novel learnable token, namely [rln]-token. Together with [obj]-tokens, [rln]-token exploits local and global semantic reasoning in an image through a series of mutual associations. In combination with the pair-wise [obj]-token, the [rln]-token contributes to a computationally efficient relation prediction. We achieve state-of-the-art performance on multiple, diverse and multi-domain datasets that demonstrate our approach's effectiveness and generalizability.

arXiv:2205.10766

Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey

Multi-object tracking (MOT) aims to associate target objects across video frames in order to obtain entire moving trajectories. With the advancement of deep neural networks and the increasing demand for intelligent video analysis, MOT has gained significantly increased interest in the computer vision community. Embedding methods play an essential role in object location estimation and temporal identity association in MOT. Unlike other computer vision tasks, such as image classification, object detection, re-identification, and segmentation, embedding methods in MOT have large variations, and they have never been systematically analyzed and summarized. In this survey, we first conduct a comprehensive overview with in-depth analysis for embedding methods in MOT from seven different perspectives, including patch-level embedding, single-frame embedding, cross-frame joint embedding, correlation embedding, sequential embedding, tracklet embedding, and cross-track relational embedding. We further summarize the existing widely used MOT datasets and analyze the advantages of existing state-of-the-art methods according to their embedding strategies. Finally, some critical yet under-investigated areas and future research directions are discussed.

arXiv:2407.08872공식 코드 · Papers with Code

Visual Multi-Object Tracking with Re-Identification and Occlusion Handling using Labeled Random Finite Sets

This paper proposes an online visual multi-object tracking (MOT) algorithm that resolves object appearance-reappearance and occlusion. Our solution is based on the labeled random finite set (LRFS) filtering approach, which in principle, addresses disappearance, appearance, reappearance, and occlusion via a single Bayesian recursion. However, in practice, existing numerical approximations cause reappearing objects to be initialized as new tracks, especially after long periods of being undetected. In occlusion handling, the filter's efficacy is dictated by trade-offs between the sophistication of the occlusion model and computational demand. Our contribution is a novel modeling method that exploits object features to address reappearing objects whilst maintaining a linear complexity in the number of detections. Moreover, to improve the filter's occlusion handling, we propose a fuzzy detection model that takes into consideration the overlapping areas between tracks and their sizes. We also develop a fast version of the filter to further reduce the computational time. The source code is publicly available at https://github.com/linh-gist/mv-glmb-ab.

arXiv:2410.15518공식 코드 · Papers with Code

TrackMe:A Simple and Effective Multiple Object Tracking Annotation Tool

Object tracking, especially animal tracking, is one of the key topics that attract a lot of attention due to its benefits of animal behavior understanding and monitoring. Recent state-of-the-art tracking methods are founded on deep learning architectures for object detection, appearance feature extraction and track association. Despite the good tracking performance, these methods are trained and evaluated on common objects such as human and cars. To perform on the animal, there is a need to create large datasets of different types in multiple conditions. The dataset construction comprises of data collection and data annotation. In this work, we put more focus on the latter task. Particularly, we renovate the well-known tool, LabelMe, so as to assist common user with or without in-depth knowledge about computer science to annotate the data with less effort. The new tool named as TrackMe inherits the simplicity, high compatibility with varied systems, minimal hardware requirement and convenient feature utilization from the predecessor. TrackMe is an upgraded version with essential features for multiple object tracking annotation.

arXiv:2403.06681

Out-of-distribution Partial Label Learning

Partial Label Learning (PLL) tackles model learning from the data with inexact labels under the assumption that training and test objects are in the same distribution, i.e., closed-set scenario. Nevertheless, this assumption does not hold in real-world open-set scenarios where test data may come from Out-Of-Distribution (OOD), resulting in object detection failure and hence significantly compromising the PLL model's security and trustworthiness. This is a previously unexplored problem called Out-Of-Distribution Partial Label Learning (OODPLL) that our newly proposed PLOOD framework can effectively resolve. During the training phase, our framework leverages self-supervised learning strategy to generate positive and negative samples for each object, emulating in and out-of-distributions respectively. Under these distributions, PLL methods can learn discriminative features for OOD objects. In the inference phase, a novel Partial Energy (PE) scoring technique is proposed which leverages the label confidence established during the above training phase to mine the actual labels. In this way, the issue of inexact labeling in PLL can be effectively addressed for significantly better performance in OOD object detection. PLOOD is compared with SOTA PLL models and OOD scores on CIFAR-10 and CIFAR-100 datasets against various OOD datasets. The results demonstrate the effectiveness of our PLOOD framework, significantly outperforming SOTA PLL models and marking a substantial advancement in addressing PLL problems in real-world OOD scenarios.

arXiv:2403.19079

A Real-Time Framework for Domain-Adaptive Underwater Object Detection with Image Enhancement

In recent years, significant progress has been made in the field of underwater image enhancement (UIE). However, its practical utility for high-level vision tasks, such as underwater object detection (UOD) in Autonomous Underwater Vehicles (AUVs), remains relatively unexplored. It may be attributed to several factors: (1) Existing methods typically employ UIE as a pre-processing step, which inevitably introduces considerable computational overhead and latency. (2) The process of enhancing images prior to training object detectors may not necessarily yield performance improvements. (3) The complex underwater environments can induce significant domain shifts across different scenarios, seriously deteriorating the UOD performance. To address these challenges, we introduce EnYOLO, an integrated real-time framework designed for simultaneous UIE and UOD with domain-adaptation capability. Specifically, both the UIE and UOD task heads share the same network backbone and utilize a lightweight design. Furthermore, to ensure balanced training for both tasks, we present a multi-stage training strategy aimed at consistently enhancing their performance. Additionally, we propose a novel domain-adaptation strategy to align feature embeddings originating from diverse underwater environments. Comprehensive experiments demonstrate that our framework not only achieves state-of-the-art (SOTA) performance in both UIE and UOD tasks, but also shows superior adaptability when applied to different underwater scenarios. Our efficiency analysis further highlights the substantial potential of our framework for onboard deployment.

arXiv:2412.11807공식 코드 · Papers with Code

PhysAug: A Physical-guided and Frequency-based Data Augmentation for Single-Domain Generalized Object Detection

Single-Domain Generalized Object Detection~(S-DGOD) aims to train on a single source domain for robust performance across a variety of unseen target domains by taking advantage of an object detector. Existing S-DGOD approaches often rely on data augmentation strategies, including a composition of visual transformations, to enhance the detector's generalization ability. However, the absence of real-world prior knowledge hinders data augmentation from contributing to the diversity of training data distributions. To address this issue, we propose PhysAug, a novel physical model-based non-ideal imaging condition data augmentation method, to enhance the adaptability of the S-DGOD tasks. Drawing upon the principles of atmospheric optics, we develop a universal perturbation model that serves as the foundation for our proposed PhysAug. Given that visual perturbations typically arise from the interaction of light with atmospheric particles, the image frequency spectrum is harnessed to simulate real-world variations during training. This approach fosters the detector to learn domain-invariant representations, thereby enhancing its ability to generalize across various settings. Without altering the network architecture or loss function, our approach significantly outperforms the state-of-the-art across various S-DGOD datasets. In particular, it achieves a substantial improvement of $7.3\%$ and $7.2\%$ over the baseline on DWD and Cityscape-C, highlighting its enhanced generalizability in real-world settings.

arXiv:2501.13898공식 코드 · Papers with Code

PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection

With the growing demand for oriented object detection (OOD), recent studies on point-supervised OOD have attracted significant interest. In this paper, we propose PointOBB-v3, a stronger single point-supervised OOD framework. Compared to existing methods, it generates pseudo rotated boxes without additional priors and incorporates support for the end-to-end paradigm. PointOBB-v3 functions by integrating three unique image views: the original view, a resized view, and a rotated/flipped (rot/flp) view. Based on the views, a scale augmentation module and an angle acquisition module are constructed. In the first module, a Scale-Sensitive Consistency (SSC) loss and a Scale-Sensitive Feature Fusion (SSFF) module are introduced to improve the model's ability to estimate object scale. To achieve precise angle predictions, the second module employs symmetry-based self-supervised learning. Additionally, we introduce an end-to-end version that eliminates the pseudo-label generation process by integrating a detector branch and introduces an Instance-Aware Weighting (IAW) strategy to focus on high-quality predictions. We conducted extensive experiments on the DIOR-R, DOTA-v1.0/v1.5/v2.0, FAIR1M, STAR, and RSAR datasets. Across all these datasets, our method achieves an average improvement in accuracy of 3.56% in comparison to previous state-of-the-art methods. The code will be available at https://github.com/ZpyWHU/PointOBB-v3.

arXiv:1904.04514공식 코드 · Papers with Code

High-Resolution Representations for Labeling Pixels and Regions

High-resolution representation learning plays an essential role in many vision problems, e.g., pose estimation and semantic segmentation. The high-resolution network (HRNet)~\cite{SunXLW19}, recently developed for human pose estimation, maintains high-resolution representations through the whole process by connecting high-to-low resolution convolutions in \emph{parallel} and produces strong high-resolution representations by repeatedly conducting fusions across parallel convolutions. In this paper, we conduct a further study on high-resolution representations by introducing a simple yet effective modification and apply it to a wide range of vision tasks. We augment the high-resolution representation by aggregating the (upsampled) representations from all the parallel convolutions rather than only the representation from the high-resolution convolution as done in~\cite{SunXLW19}. This simple modification leads to stronger representations, evidenced by superior results. We show top results in semantic segmentation on Cityscapes, LIP, and PASCAL Context, and facial landmark detection on AFLW, COFW, $300$W, and WFLW. In addition, we build a multi-level representation from the high-resolution representation and apply it to the Faster R-CNN object detection framework and the extended frameworks. The proposed approach achieves superior results to existing single-model networks on COCO object detection. The code and models have been publicly available at \url{https://github.com/HRNet}.

arXiv:1908.09539

Error Bounded Foreground and Background Modeling for Moving Object Detection in Satellite Videos

Detecting moving objects from ground-based videos is commonly achieved by using background subtraction techniques. Low-rank matrix decomposition inspires a set of state-of-the-art approaches for this task. It is integrated with structured sparsity regularization to achieve background subtraction in the developed method of Low-rank and Structured Sparse Decomposition (LSD). However, when this method is applied to satellite videos where spatial resolution is poor and targets' contrast to the background is low, its performance is limited as the data no longer fits adequately either the foreground structure or the background model. In this paper, we handle these unexplained data explicitly and address the moving target detection from space as one of the pioneer studies. We propose a technique by extending the decomposition formulation with bounded errors, named Extended Low-rank and Structured Sparse Decomposition (E-LSD). This formulation integrates low-rank background, structured sparse foreground and their residuals in a matrix decomposition problem. We provide an effective solution by introducing an alternative treatment and adopting the direct extension of Alternating Direction Method of Multipliers (ADMM). The proposed E-LSD was validated on two satellite videos, and experimental results demonstrate the improvement in background modeling with boosted moving object detection precision over state-of-the-art methods.

arXiv:2402.03235

ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detection

The curation of large-scale datasets is still costly and requires much time and resources. Data is often manually labeled, and the challenge of creating high-quality datasets remains. In this work, we fill the research gap using active learning for multi-modal 3D object detection. We propose ActiveAnno3D, an active learning framework to select data samples for labeling that are of maximum informativeness for training. We explore various continuous training methods and integrate the most efficient method regarding computational demand and detection performance. Furthermore, we perform extensive experiments and ablation studies with BEVFusion and PV-RCNN on the nuScenes and TUM Traffic Intersection dataset. We show that we can achieve almost the same performance with PV-RCNN and the entropy-based query strategy when using only half of the training data (77.25 mAP compared to 83.50 mAP) of the TUM Traffic Intersection dataset. BEVFusion achieved an mAP of 64.31 when using half of the training data and 75.0 mAP when using the complete nuScenes dataset. We integrate our active learning framework into the proAnno labeling tool to enable AI-assisted data selection and labeling and minimize the labeling costs. Finally, we provide code, weights, and visualization results on our website: https://active3d-framework.github.io/active3d-framework.

arXiv:2406.04829

IOR: Inversed Objects Replay for Incremental Object Detection

Existing Incremental Object Detection (IOD) methods partially alleviate catastrophic forgetting when incrementally detecting new objects in real-world scenarios. However, many of these methods rely on the assumption that unlabeled old-class objects may co-occur with labeled new-class objects in the incremental data. When unlabeled old-class objects are absent, the performance of existing methods tends to degrade. The absence can be mitigated by generating old-class samples, but it incurs high costs. This paper argues that previous generation-based IOD suffers from redundancy, both in the use of generative models, which require additional training and storage, and in the overproduction of generated samples, many of which do not contribute significantly to performance improvements. To eliminate the redundancy, we propose Inversed Objects Replay (IOR). Specifically, we generate old-class samples by inversing the original detectors, thus eliminating the necessity of training and storing additional generative models. We propose augmented replay to reuse the objects in generated samples, reducing redundant generations. Moreover, we propose high-value knowledge distillation focusing on the positions of old-class objects overwhelmed by the background, which transfers the knowledge to the incremental detector. Extensive experiments conducted on MS COCO 2017 demonstrate that our method can efficiently improve detection performance in IOD scenarios with the absence of old-class objects.

arXiv:2608.22368v1cs.CV

DiD, 87분 만에 완료: 객체 검출을 위한 비전 트랜스포머의 레이블 없는 소프트맥스-선형 어텐션 전환

DiD It in 87 Minutes: A Label-Free Softmax-to-Linear Adaptation of Vision Transformers for Object Detection

선형 어텐션(linear attention)은 전역 토큰 혼합(global token mixing)의 비용이 낮아 고해상도 객체 검출에 매력적인 메커니즘이지만, 학습이 완료된 검출기의 소프트맥스 어텐션 ViT 백본을 선형 어텐션 백본으로 변환하는 일은 단순한 대체로 해결되지 않는다. 어텐션 연산자를 그대로 교체하면 성능이 심각하게 저하되며, 분류에서는 효과적인 일반적인 레이블 없는(label-free) 지식 증류도 검출 작업에서는 종종 실패한다. 우리는 핵심 과제가 \textit{검출기 인터페이스 보존(detector-interface preservation)}이라고 주장한다. 즉, 변환된 백본은 소프트맥스 내부 은닉 상태를 단순히 모방하는 것이 아니라, 고정된 하위 검출기가 기대하는 특징 텐서를 정확히 재현해야 한다. 이를 위해 우리는 레이블 없는 변환 방법인 검출기 인터페이스 증류(Detector-Interface Distillation, DiD)를 제안한다. DiD는 검출기와 맞닿은 인터페이스 텐서를 고정된(frozen) 소프트맥스 교사 모델의 텐서와 정렬함으로써 선형 어텐션 백본만을 학습한다. DOTA-v1.5에서 DiD는 기존 베이스라인을 크게 능가하며, 지도 학습으로 완전히 학습된 선형 모델과 대등한 성능을 보인다. 적응은 GPU 4개로 약 87분 만에 완료되며, 선형화된 백본은 추론 지연 시간을 약 62%, 최대 메모리를 약 49% 줄인다. 우리는 이 연구 결과가 학습된 소프트맥스 검출기를 효율적인 선형 검출기로 재사용하는 단순하고 레이블 없는 방법을 커뮤니티에 제공하고, 향후 아키텍처 변환 연구에서 인터페이스를 고려한 목적 함수를 사용하도록 장려하기를 바란다.

arXiv:2012.02206

Scan2Cap: Context-aware Dense Captioning in RGB-D Scans

We introduce the task of dense captioning in 3D scans from commodity RGB-D sensors. As input, we assume a point cloud of a 3D scene; the expected output is the bounding boxes along with the descriptions for the underlying objects. To address the 3D object detection and description problems, we propose Scan2Cap, an end-to-end trained method, to detect objects in the input scene and describe them in natural language. We use an attention mechanism that generates descriptive tokens while referring to the related components in the local context. To reflect object relations (i.e. relative spatial relations) in the generated captions, we use a message passing graph module to facilitate learning object relation features. Our method can effectively localize and describe 3D objects in scenes from the ScanRefer dataset, outperforming 2D baseline methods by a significant margin (27.61% [email omitted]).

arXiv:2106.09259공식 코드 · Papers with Code

A Random CNN Sees Objects: One Inductive Bias of CNN and Its Applications

This paper starts by revealing a surprising finding: without any learning, a randomly initialized CNN can localize objects surprisingly well. That is, a CNN has an inductive bias to naturally focus on objects, named as Tobias ("The object is at sight") in this paper. This empirical inductive bias is further analyzed and successfully applied to self-supervised learning (SSL). A CNN is encouraged to learn representations that focus on the foreground object, by transforming every image into various versions with different backgrounds, where the foreground and background separation is guided by Tobias. Experimental results show that the proposed Tobias significantly improves downstream tasks, especially for object detection. This paper also shows that Tobias has consistent improvements on training sets of different sizes, and is more resilient to changes in image augmentations. Code is available at https://github.com/CupidJay/Tobias.