PyTorchKR Papers

“Object Detection”

총 13,444편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2103.00236공식 코드 · Papers with Code

Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection

Unsupervised domain adaptive object detection aims to adapt detectors from a labelled source domain to an unlabelled target domain. Most existing works take a two-stage strategy that first generates region proposals and then detects objects of interest, where adversarial learning is widely adopted to mitigate the inter-domain discrepancy in both stages. However, adversarial learning may impair the alignment of well-aligned samples as it merely aligns the global distributions across domains. To address this issue, we design an uncertainty-aware domain adaptation network (UaDAN) that introduces conditional adversarial learning to align well-aligned and poorly-aligned samples separately in different manners. Specifically, we design an uncertainty metric that assesses the alignment of each sample and adjusts the strength of adversarial learning for well-aligned and poorly-aligned samples adaptively. In addition, we exploit the uncertainty metric to achieve curriculum learning that first performs easier image-level alignment and then more difficult instance-level alignment progressively. Extensive experiments over four challenging domain adaptive object detection datasets show that UaDAN achieves superior performance as compared with state-of-the-art methods.

arXiv:2406.12225

The Solution for CVPR2024 Foundational Few-Shot Object Detection Challenge

This report introduces an enhanced method for the Foundational Few-Shot Object Detection (FSOD) task, leveraging the vision-language model (VLM) for object detection. However, on specific datasets, VLM may encounter the problem where the detected targets are misaligned with the target concepts of interest. This misalignment hinders the zero-shot performance of VLM and the application of fine-tuning methods based on pseudo-labels. To address this issue, we propose the VLM+ framework, which integrates the multimodal large language model (MM-LLM). Specifically, we use MM-LLM to generate a series of referential expressions for each category. Based on the VLM predictions and the given annotations, we select the best referential expression for each category by matching the maximum IoU. Subsequently, we use these referential expressions to generate pseudo-labels for all images in the training set and then combine them with the original labeled data to fine-tune the VLM. Additionally, we employ iterative pseudo-label generation and optimization to further enhance the performance of the VLM. Our approach achieve 32.56 mAP in the final test.

arXiv:2408.13243

MCTR: Multi Camera Tracking Transformer

Multi-camera tracking plays a pivotal role in various real-world applications. While end-to-end methods have gained significant interest in single-camera tracking, multi-camera tracking remains predominantly reliant on heuristic techniques. In response to this gap, this paper introduces Multi-Camera Tracking tRansformer (MCTR), a novel end-to-end approach tailored for multi-object detection and tracking across multiple cameras with overlapping fields of view. MCTR leverages end-to-end detectors like DEtector TRansformer (DETR) to produce detections and detection embeddings independently for each camera view. The framework maintains set of track embeddings that encaplusate global information about the tracked objects, and updates them at every frame by integrating the local information from the view-specific detection embeddings. The track embeddings are probabilistically associated with detections in every camera view and frame to generate consistent object tracks. The soft probabilistic association facilitates the design of differentiable losses that enable end-to-end training of the entire system. To validate our approach, we conduct experiments on MMPTrack and AI City Challenge, two recently introduced large-scale multi-camera multi-object tracking datasets.

arXiv:1905.06653

Robust Real-time Pedestrian Detection in Aerial Imagery on Jetson TX2

Detection of pedestrians in aerial imagery captured by drones has many applications including intersection monitoring, patrolling, and surveillance, to name a few. However, the problem is involved due to continuouslychanging camera viewpoint and object appearance as well as the need for lightweight algorithms to run on on-board embedded systems. To address this issue, the paper proposes a framework for pedestrian detection in videos based on the YOLO object detection network [6] while having a high throughput of more than 5 FPS on the Jetson TX2 embedded board. The framework exploits deep learning for robust operation and uses a pre-trained model without the need for any additional training which makes it flexible to apply on different setups with minimum amount of tuning. The method achieves ~81 mAP when applied on a sample video from the Embedded Real-Time Inference (ERTI) Challenge where pedestrians are monitored by a UAV.

arXiv:2309.06313

Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle

It is difficult to perform 3D reconstruction from on-vehicle gathered video due to the large forward motion of the vehicle. Even object detection and human sensing models perform significantly worse on onboard videos when compared to standard benchmarks because objects often appear far away from the camera compared to the standard object detection benchmarks, image quality is often decreased by motion blur and occlusions occur often. This has led to the popularisation of traffic data-specific benchmarks. Recently Light Detection And Ranging (LiDAR) sensors have become popular to directly estimate depths without the need to perform 3D reconstructions. However, LiDAR-based methods still lack in articulated human detection at a distance when compared to image-based methods. We hypothesize that benchmarks targeted at articulated human sensing from LiDAR data could bring about increased research in human sensing and prediction in traffic and could lead to improved traffic safety for pedestrians.

arXiv:2407.17956

SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images

The advancement of deep learning in object detection has predominantly focused on megapixel images, leaving a critical gap in the efficient processing of gigapixel images. These super high-resolution images present unique challenges due to their immense size and computational demands. To address this, we introduce 'SaccadeDet', an innovative architecture for gigapixel-level object detection, inspired by the human eye saccadic movement. The cornerstone of SaccadeDet is its ability to strategically select and process image regions, dramatically reducing computational load. This is achieved through a two-stage process: the 'saccade' stage, which identifies regions of probable interest, and the 'gaze' stage, which refines detection in these targeted areas. Our approach, evaluated on the PANDA dataset, not only achieves an 8x speed increase over the state-of-the-art methods but also demonstrates significant potential in gigapixel-level pathology analysis through its application to Whole Slide Imaging.

논문 ID: document/1029444

Real-Time Object Detection on High-Voltage Powerlines Using an Unmanned Aerial Vehicle (UAV)

Unmanned Aerial Vehicles (UAVs) are gaining significant scientific interest in critical infrastructure inspection due to their flexibility, cost-effectiveness and advanced computer vision capabilities. This research focuses on high-voltage powerline surveillance, where automatic inspection is a priority for grid companies to prevent power failures. To address the need for real-time detection with limited computational power, we evaluate the recently developed object detection algorithm, YOLOvS. We propose a fine-tuned model trained on a custom dataset to detect key components, i. e. towers, insulators and conductors. The proposed method achieves an overall accuracy rate of 82.3% ([email omitted]) and enables real-time detection, demonstrating its suitability for inspection tasks and visual-based navigation. Our model was also tested on a custom-built quadcopter with an Nvidia Jetson Nano (4GB) on board, achieving a frame rate of 33fps on live video under real environmental conditions.

arXiv:1807.07428

Modeling Visual Context is Key to Augmenting Object Detection Datasets

Performing data augmentation for learning deep neural networks is well known to be important for training visual recognition systems. By artificially increasing the number of training examples, it helps reducing overfitting and improves generalization. For object detection, classical approaches for data augmentation consist of generating images obtained by basic geometrical transformations and color changes of original training images. In this work, we go one step further and leverage segmentation annotations to increase the number of object instances present on training data. For this approach to be successful, we show that modeling appropriately the visual context surrounding objects is crucial to place them in the right environment. Otherwise, we show that the previous strategy actually hurts. With our context model, we achieve significant mean average precision improvements when few labeled examples are available on the VOC'12 benchmark.

arXiv:2407.17738공식 코드 · Papers with Code

Enhancing Fine-grained Object Detection in Aerial Images via Orthogonal Mapping

Fine-Grained Object Detection (FGOD) is a critical task in high-resolution aerial image analysis. This letter introduces Orthogonal Mapping (OM), a simple yet effective method aimed at addressing the challenge of semantic confusion inherent in FGOD. OM introduces orthogonal constraints in the feature space by decoupling features from the last layer of the classification branch with a class-wise orthogonal vector basis. This effectively mitigates semantic confusion and enhances classification accuracy. Moreover, OM can be seamlessly integrated into mainstream object detectors. Extensive experiments conducted on three FGOD datasets (FAIR1M, ShipRSImageNet, and MAR20) demonstrate the effectiveness and superiority of the proposed approach. Notably, with just one line of code, OM achieves a 4.08% improvement in mean Average Precision (mAP) over FCOS on the ShipRSImageNet dataset. Codes are released at https://github.com/ZhuHaoranEIS/Orthogonal-FGOD.

arXiv:2504.06176v4cs.LG

우주 객체 행동 특성화를 위한 자기지도 학습 프레임워크

A Self-Supervised Framework for Space Object Behaviour Characterisation

라벨 없는 데이터에 대해 사전학습된 대규모 신경망을 활용한 뒤 특정 작업에 미세조정하여 사용하는 파운데이션 모델(Foundation Model)은 점점 더 전문 분야에 적용되고 있다. 최근 예로는 기후 분야의 ClimaX와 위성 지구 관측을 위한 Clay가 있지만, 우주 객체 행동 분석(SOBA, Space Object Behavioural Analysis)을 위한 파운데이션 모델은 아직 개발되지 않았다. 궤도상 객체 수가 증가함에 따라 우주 객체 행동을 특성화하는 자동화 방법은 우주 안전성(safety)에 필수적이다. 본 연구에서는 SOBA를 위한 파운데이션 모델로 나아가는 첫걸음으로, 우주 객체 행동 분석을 위한 자기지도 학습 프레임워크를 제안한다. 백본은 Perceiver-변분 오토인코더(VAE) 아키텍처이며, MMT-9 관측소의 약 227,000개 광곡선(light curve)에 대해 자기지도 재구성 및 마스크 재구성으로 사전학습했다. VAE는 이상 탐지, 운동 예측, 합성 광곡선 생성을 가능하게 한다. boxwing, Sentinel-3, SMOS, Starlink 플랫폼의 CAD 모델을 사용하여 두 개의 독립적인 광곡선 시뮬레이터(CASSANDRA 및 GRIAL)로 모델을 미세조정했다. 사전학습된 모델은 재구성 평균제곱오차(MSE) 0.0012를 달성했으며, 재구성 난이도를 통해 잠재적으로 이상적인 광곡선을 식별했다. 미세조정 후 모델은 이상 탐지 및 운동 모드 예측(예: sun-pointing, spin, tumbling)에서 85% 및 82% 정확도와 0.92 및 0.95 ROC AUC 점수를 기록했다. 실제 데이터에 대한 고신뢰도 예측 분석을 통해 특성적인 객체 프로파일과 위성 glinting 등 뚜렷한 패턴이 드러났다. 본 연구는 풍부한 사전학습 표현으로부터 자기지도 학습이 이상 탐지, 운동 예측, 합성 데이터 생성을 동시에 가능하게 하는 방식을 보여주며, 자동화된 모니터링과 시뮬레이션을 통해 우주 안전성과 지속가능성을 지원한다.

arXiv:2109.07107공식 코드 · Papers with Code

Anchor DETR: Query Design for Transformer-Based Object Detection

In this paper, we propose a novel query design for the transformer-based object detection. In previous transformer-based detectors, the object queries are a set of learned embeddings. However, each learned embedding does not have an explicit physical meaning and we cannot explain where it will focus on. It is difficult to optimize as the prediction slot of each object query does not have a specific mode. In other words, each object query will not focus on a specific region. To solved these problems, in our query design, object queries are based on anchor points, which are widely used in CNN-based detectors. So each object query focuses on the objects near the anchor point. Moreover, our query design can predict multiple objects at one position to solve the difficulty: "one region, multiple objects". In addition, we design an attention variant, which can reduce the memory cost while achieving similar or better performance than the standard attention in DETR. Thanks to the query design and the attention variant, the proposed detector that we called Anchor DETR, can achieve better performance and run faster than the DETR with 10$\times$ fewer training epochs. For example, it achieves 44.2 AP with 19 FPS on the MSCOCO dataset when using the ResNet50-DC5 feature for training 50 epochs. Extensive experiments on the MSCOCO benchmark prove the effectiveness of the proposed methods. Code is available at \url{https://github.com/megvii-research/AnchorDETR}.

arXiv:2107.13269

Aug3D-RPN: Improving Monocular 3D Object Detection by Synthetic Images with Virtual Depth

Current geometry-based monocular 3D object detection models can efficiently detect objects by leveraging perspective geometry, but their performance is limited due to the absence of accurate depth information. Though this issue can be alleviated in a depth-based model where a depth estimation module is plugged to predict depth information before 3D box reasoning, the introduction of such module dramatically reduces the detection speed. Instead of training a costly depth estimator, we propose a rendering module to augment the training data by synthesizing images with virtual-depths. The rendering module takes as input the RGB image and its corresponding sparse depth image, outputs a variety of photo-realistic synthetic images, from which the detection model can learn more discriminative features to adapt to the depth changes of the objects. Besides, we introduce an auxiliary module to improve the detection model by jointly optimizing it through a depth estimation task. Both modules are working in the training time and no extra computation will be introduced to the detection model. Experiments show that by working with our proposed modules, a geometry-based model can represent the leading accuracy on the KITTI 3D detection benchmark.

arXiv:2203.04129공식 코드 · Papers with Code

YouTube-GDD: A challenging gun detection dataset with rich contextual information

An automatic gun detection system can detect potential gun-related violence at an early stage that is of paramount importance for citizens security. In the whole system, object detection algorithm is the key to perceive the environment so that the system can detect dangerous objects such as pistols and rifles. However, mainstream deep learning-based object detection algorithms depend heavily on large-scale high-quality annotated samples, and the existing gun datasets are characterized by low resolution, little contextual information and little data volume. To promote the development of security, this work presents a new challenging dataset called YouTube Gun Detection Dataset (YouTube-GDD). Our dataset is collected from 343 high-definition YouTube videos and contains 5000 well-chosen images, in which 16064 instances of gun and 9046 instances of person are annotated. Compared to other datasets, YouTube-GDD is "dynamic", containing rich contextual information and recording shape changes of the gun during shooting. To build a baseline for gun detection, we evaluate YOLOv5 on YouTube-GDD and analyze the influence of additional related annotated information on gun detection. YouTube-GDD and subsequent updates will be released at https://github.com/UCAS-GYX/YouTube-GDD.

arXiv:2311.17286공식 코드 · Papers with Code

LEOD: Label-Efficient Object Detection for Event Cameras

Object detection with event cameras benefits from the sensor's low latency and high dynamic range. However, it is costly to fully label event streams for supervised training due to their high temporal resolution. To reduce this cost, we present LEOD, the first method for label-efficient event-based detection. Our approach unifies weakly- and semi-supervised object detection with a self-training mechanism. We first utilize a detector pre-trained on limited labels to produce pseudo ground truth on unlabeled events. Then, the detector is re-trained with both real and generated labels. Leveraging the temporal consistency of events, we run bi-directional inference and apply tracking-based post-processing to enhance the quality of pseudo labels. To stabilize training against label noise, we further design a soft anchor assignment strategy. We introduce new experimental protocols to evaluate the task of label-efficient event-based detection on Gen1 and 1Mpx datasets. LEOD consistently outperforms supervised baselines across various labeling ratios. For example, on Gen1, it improves mAP by 8.6% and 7.8% for RVT-S trained with 1% and 2% labels. On 1Mpx, RVT-S with 10% labels even surpasses its fully-supervised counterpart using 100% labels. LEOD maintains its effectiveness even when all labeled data are available, reaching new state-of-the-art results. Finally, we show that our method readily scales to improve larger detectors as well. Code is released at https://github.com/Wuziyi616/LEOD

arXiv:2606.06978v1cs.CV

CL-CLIP: Cost-Volume 범주 분리를 갖춘 CLIP 기반 객체 검출 지속 학습 프레임워크

CL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection

지속 객체 검출(COD)은 검출기가 이전에 학습한 범주를 유지하면서 시간에 따라 새로운 범주를 습득해야 한다. 이 목표는 개방 어휘 검출(open-vocabulary detection)과 밀접하게 관련되어 있는데, 두 설정 모두 현재 학습 단계에서 사용 가능한 주석으로 완전히 포괄되지 않은 범주에 대해 추론해야 하기 때문이다. 최근 CLIP 기반 개방 어휘 검출기는 강력한 제로샷 일반화 능력을 보여 주었으며, F-ViT와 같은 프레임워크는 비전-언어 사전학습이 보지 못한 범주에 대한 강력한 제로샷 검출 능력을 제공할 수 있음을 보여준다. 그러나 실제 배포 환경에서는 순수하게 제로샷만으로 유지할 수 없다. 이 검출기들이 새로 도입된 범주에 대해 지속적으로 업데이트되면 심각한 파국적 망각(catastrophic forgetting)을 겪고, 이전에 보정된 검출 능력을 빠르게 상실한다. 따라서 우리는 cost-volume 기반 범주 분리(category decoupling)를 통해 개방 어휘 검출기에 더 나은 지속 학습 능력을 부여하는 CLIP 기반 COD 프레임워크인 CL-CLIP을 제안한다. 구체적으로, CAT-Seg를 따라 CLIP 이미지-텍스트 유사도 cost volume을 계산하는데, 이는 시각 토큰과 클래스 텍스트 임베딩 사이의 밀집 범주별 응답 맵으로 정의된다. 이 제로샷 공간 사전(spatial prior)은 공유 영역 특징을 클래스별 경로로 분해하고, 이를 Multi-Expert RoI head가 처리한다. PASCAL VOC 및 MS-COCO에 대한 광범위한 실험 결과, CL-CLIP은 지속적 미세조정 하에서 F-ViT 기준선을 크게 개선하고, 특히 새로 도입된 범주에 적응하면서도 경쟁력 있는 기본 클래스 성능을 유지하는 데 있어 기존 지속 객체 검출기와 경쟁력 있는 성능을 달성함을 보여준다.

arXiv:2012.12412공식 코드 · Papers with Code

Optical Braille Recognition Using Object Detection CNN

This paper proposes an optical Braille recognition method that uses an object detection convolutional neural network to detect whole Braille characters at once. The proposed algorithm is robust to the deformation of the page shown in the image and perspective distortions. It makes it usable for recognition of Braille texts being shoot on a smartphone camera, including bowed pages and perspective distorted images. The proposed algorithm shows high performance and accuracy compared to existing methods. We also introduce a new "Angelina Braille Images Dataset" containing 240 annotated photos of Braille texts. The proposed algorithm and dataset are available at GitHub.

arXiv:2304.00788공식 코드 · Papers with Code

Open-Vocabulary Point-Cloud Object Detection without 3D Annotation

The goal of open-vocabulary detection is to identify novel objects based on arbitrary textual descriptions. In this paper, we address open-vocabulary 3D point-cloud detection by a dividing-and-conquering strategy, which involves: 1) developing a point-cloud detector that can learn a general representation for localizing various objects, and 2) connecting textual and point-cloud representations to enable the detector to classify novel object categories based on text prompting. Specifically, we resort to rich image pre-trained models, by which the point-cloud detector learns localizing objects under the supervision of predicted 2D bounding boxes from 2D pre-trained detectors. Moreover, we propose a novel de-biased triplet cross-modal contrastive learning to connect the modalities of image, point-cloud and text, thereby enabling the point-cloud detector to benefit from vision-language pre-trained models,i.e.,CLIP. The novel use of image and vision-language pre-trained models for point-cloud detectors allows for open-vocabulary 3D object detection without the need for 3D annotations. Experiments demonstrate that the proposed method improves at least 3.03 points and 7.47 points over a wide range of baselines on the ScanNet and SUN RGB-D datasets, respectively. Furthermore, we provide a comprehensive analysis to explain why our approach works.

arXiv:2608.25808v1cs.CV

TDFNet: 파노라마 돌출 객체 검출을 위한 삼중 투영 변형 가능 융합 네트워크

TDFNet: Tri-projection Deformable Fusion Network for Panoramic Salient Object Detection

최근 몇 년간 파노라마 돌출 객체 검출(panoramic salient object detection)은 로봇 비전, 가상 현실 및 관련 응용 분야에서 점점 더 큰 잠재력을 보이고 있다. 그러나 구형 장면을 2D 평면에 투영하면 필연적으로 기하학적 왜곡이 발생하며, 이는 기존 투영 기반 방법의 효과를 근본적으로 제한한다. 구체적으로, 등장방형 투영(Equirectangular Projection, ERP)은 극점 부근에서 심한 늘어짐 왜곡이 발생하고, 큐브맵 투영(cube map projection)은 큐브 면 경계에서 불연속성이 발생하여 특징 판별력이 저하되고 기하학적 일관성이 손상된다. 이러한 한계를 해결하기 위해, 본 논문은 상호 보완적인 투영 표현을 활용하여 기하학적 왜곡을 완화하고 검출 성능을 향상시키는, 파노라마 돌출 객체 검출을 위한 최초의 삼중 투영 변형 가능 융합 네트워크(Tri-projection Deformable Fusion Network)인 TDFNet을 제안한다. 구체적으로, 서로 다른 투영 간의 공간적 대응 관계를 활용해 기하학 인식 샘플링 위치를 구성하는 교차 투영 변형 가능 어텐션(cross-projection deformable attention, CDA) 모듈을 설계한다. 이 모듈은 변형 가능 어텐션(deformable attention)이 교차 투영 문맥 정보를 집계하도록 유도하며, 투영으로 인한 변형에 대한 강건성을 높인다. 더불어, 구면 위도 사전 정보를 활용해 기하학적 신뢰도 가중치를 구성하고 ERP 특징과 CMP 특징을 적응적으로 균형 있게 결합하는 위도 유도 융합(latitude-guided fusion, LGF) 모듈을 도입한다. 또한 LGF는 접선 투영(Tangent Projection)에서 얻은 왜곡이 감소된 의미적 참조를 통합하여 교차 투영 특징 정제와 공간 정렬을 달성한다. ERP, CMP, 접선 투영을 기반으로 한 3-브랜치 인코딩 아키텍처를 구성함으로써, TDFNet은 전역적 공간 연속성, 국소적 기하학적 세부 정보, 그리고 세밀한 경계 정보를 동시에 보존한다.

arXiv:2401.06159

FRED: Towards a Full Rotation-Equivariance in Aerial Image Object Detection

Rotation-equivariance is an essential yet challenging property in oriented object detection. While general object detectors naturally leverage robustness to spatial shifts due to the translation-equivariance of the conventional CNNs, achieving rotation-equivariance remains an elusive goal. Current detectors deploy various alignment techniques to derive rotation-invariant features, but still rely on high capacity models and heavy data augmentation with all possible rotations. In this paper, we introduce a Fully Rotation-Equivariant Oriented Object Detector (FRED), whose entire process from the image to the bounding box prediction is strictly equivariant. Specifically, we decouple the invariant task (object classification) and the equivariant task (object localization) to achieve end-to-end equivariance. We represent the bounding box as a set of rotation-equivariant vectors to implement rotation-equivariant localization. Moreover, we utilized these rotation-equivariant vectors as offsets in the deformable convolution, thereby enhancing the existing advantages of spatial adaptation. Leveraging full rotation-equivariance, our FRED demonstrates higher robustness to image-level rotation compared to existing methods. Furthermore, we show that FRED is one step closer to non-axis aligned learning through our experiments. Compared to state-of-the-art methods, our proposed method delivers comparable performance on DOTA-v1.0 and outperforms by 1.5 mAP on DOTA-v1.5, all while significantly reducing the model parameters to 16%.

arXiv:1403.3118

Parallel WiSARD object tracker: a ram-based tracking system

This paper proposes the Parallel WiSARD Object Tracker (PWOT), a new object tracker based on the WiSARD weightless neural network that is robust against quantization errors. Object tracking in video is an important and challenging task in many applications. Difficulties can arise due to weather conditions, target trajectory and appearance, occlusions, lighting conditions and noise. Tracking is a high-level application and requires the object location frame by frame in real time. This paper proposes a fast hybrid image segmentation (threshold and edge detection) in YcbCr color model and a parallel RAM based discriminator that improves efficiency when quantization errors occur. The original WiSARD training algorithm was changed to allow the tracking.