PyTorchKR Papers

“Object Detection”

총 13,455편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2110.12261

espiownage: Tracking Transients in Steelpan Drum Strikes Using Surveillance Technology

We present an improvement in the ability to meaningfully track features in high speed videos of Caribbean steelpan drums illuminated by Electronic Speckle Pattern Interferometry (ESPI). This is achieved through the use of up-to-date computer vision libraries for object detection and image segmentation as well as a significant effort toward cleaning the dataset previously used to train systems for this application. Besides improvements on previous metric scores by 10% or more, noteworthy in this project are the introduction of a segmentation-regression map for the entire drum surface yielding interference fringe counts comparable to those obtained via object detection, as well as the accelerated workflow for coordinating the data-cleaning-and-model-training feedback loop for rapid iteration allowing this project to be conducted on a timescale of only 18 days.

arXiv:2403.08016

Aedes aegypti Egg Counting with Neural Networks for Object Detection

Aedes aegypti is still one of the main concerns when it comes to disease vectors. Among the many ways to deal with it, there are important protocols that make use of egg numbers in ovitraps to calculate indices, such as the LIRAa and the Breteau Index, which can provide information on predictable outbursts and epidemics. Also, there are many research lines that require egg numbers, specially when mass production of mosquitoes is needed. Egg counting is a laborious and error-prone task that can be automated via computer vision-based techniques, specially deep learning-based counting with object detection. In this work, we propose a new dataset comprising field and laboratory eggs, along with test results of three neural networks applied to the task: Faster R-CNN, Side-Aware Boundary Localization and FoveaBox.

arXiv:1902.02067공식 코드 · Papers with Code

Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples

This paper demonstrates that Non-Maximum Suppression (NMS), which is commonly used in Object Detection (OD) tasks to filter redundant detection results, is no longer secure. Considering that NMS has been an integral part of OD systems, thwarting the functionality of NMS can result in unexpected or even lethal consequences for such systems. In this paper, an adversarial example attack which triggers malfunctioning of NMS in end-to-end OD models is proposed. The attack, namely \texttt{Daedalus}, compresses the dimensions of detection boxes to evade NMS. As a result, the final detection output contains extremely dense false positives. This can be fatal for many OD applications such as autonomous vehicles and surveillance systems. The attack can be generalised to different end-to-end OD models, such that the attack cripples various OD applications. Furthermore, a way to craft robust adversarial examples is developed by using an ensemble of popular detection models as the substitutes. Considering the pervasive nature of model reusing in real-world OD scenarios, Daedalus examples crafted based on an \textit{ensemble of substitutes} can launch attacks without knowing the parameters of the victim models. Experimental results demonstrate that the attack effectively stops NMS from filtering redundant bounding boxes. As the evaluation results suggest, Daedalus increases the false positive rate in detection results to $99.9\%$ and reduces the mean average precision scores to $0$, while maintaining a low cost of distortion on the original inputs. It is also demonstrated that the attack can be practically launched against real-world OD systems via printed posters.

arXiv:2506.04499

FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices

Existing LiDAR 3D object detection methods predominantely rely on sparse convolutions and/or transformers, which can be challenging to run on resource-constrained edge devices, due to irregular memory access patterns and high computational costs. In this paper, we propose FALO, a hardware-friendly approach to LiDAR 3D detection, which offers both state-of-the-art (SOTA) detection accuracy and fast inference speed. More specifically, given the 3D point cloud and after voxelization, FALO first arranges sparse 3D voxels into a 1D sequence based on their coordinates and proximity. The sequence is then processed by our proposed ConvDotMix blocks, consisting of large-kernel convolutions, Hadamard products, and linear layers. ConvDotMix provides sufficient mixing capability in both spatial and embedding dimensions, and introduces higher-order nonlinear interaction among spatial features. Furthermore, when going through the ConvDotMix layers, we introduce implicit grouping, which balances the tensor dimensions for more efficient inference and takes into account the growing receptive field. All these operations are friendly to run on resource-constrained platforms and proposed FALO can readily deploy on compact, embedded devices. Our extensive evaluation on LiDAR 3D detection benchmarks such as nuScenes and Waymo shows that FALO achieves competitive performance. Meanwhile, FALO is 1.6~9.8x faster than the latest SOTA on mobile Graphics Processing Unit (GPU) and mobile Neural Processing Unit (NPU).

arXiv:2604.11140v2cs.CV

Hyper-FEOD: 세립화된 MoE 기반의 희소 초그래프 강화 프레임-이벤트 객체 검출

Hyper-FEOD: Sparse Hypergraph-Enhanced Frame-Event Object Detection with Fine-Grained MoE

프레임 기반 RGB 카메라와 이벤트 스트림의 통합은 까다로운 동적 조건에서 강건한 객체 검출(object detection)을 수행하기 위한 유망한 패러다임입니다. 그럼에도 불구하고, 복잡한 다중 모달 상호작용을 효과적으로 모델링하고 RGB와 이벤트 데이터 간의 의미론적 이질성을 조화시키는 것은 고정밀도 검출을 위한 만만치 않은 과제로 남아 있습니다. 본 논문에서는 두 가지 핵심 초그래프(hypergraph) 기반 컴포넌트를 통해 교차 모달 표현 학습을 시너지 효과로 강화하는 새로운 고성능 검출 프레임워크인 Hyper-FEOD를 제안합니다. 구체적으로, 먼저 이벤트 활동 단서를 활용하여 움직임에 중요한 희소 토큰을 식별하고 초그래프 모델링을 통해 고차 관계 추론을 수행하는 희소 초그래프 강화 교차 모달 융합(SHCF) 모듈을 설계합니다. 이러한 설계는 모달리티 전반에 걸친 복잡한 고차 의존성과 풍부한 문맥 상관관계를 효과적으로 포착합니다. 둘째로, 서로 다른 시각 영역에서 발생하는 이질적인 의미론적 요구를 처리하도록 맞춤화된 세립화된 전문가 혼합(FG-MoE) 모듈을 개발합니다. 서로 다른 하이퍼에지 연결 패턴을 가진 특화된 초그래프 전문가들을 배치하고 공간 게이팅 메커니즘을 통합함으로써, FG-MoE는 특징을 적응적으로 라우팅하여 대상 영역에서의 정밀한 향상을 가능하게 합니다. 보조 라우터 손실과 결합된 이 제안 프레임워크는 안정적인 엔드투엔드 학습(training)과 최적의 특징 정제를 보장합니다. 널리 채택된 RGB-이벤트 벤치마크에서 수행된 포괄적인 실험은 Hyper-FEOD가 우수한 검출 성능을 제공하며 기존의 최첨단 접근 방식들을 상당한 마진으로 일관되게 능가함을 보여줍니다.

arXiv:2203.08141

Object Manipulation via Visual Target Localization

Object manipulation is a critical skill required for Embodied AI agents interacting with the world around them. Training agents to manipulate objects, poses many challenges. These include occlusion of the target object by the agent's arm, noisy object detection and localization, and the target frequently going out of view as the agent moves around in the scene. We propose Manipulation via Visual Object Location Estimation (m-VOLE), an approach that explores the environment in search for target objects, computes their 3D coordinates once they are located, and then continues to estimate their 3D locations even when the objects are not visible, thus robustly aiding the task of manipulating these objects throughout the episode. Our evaluations show a massive 3x improvement in success rate over a model that has access to the same sensory suite but is trained without the object location estimator, and our analysis shows that our agent is robust to noise in depth perception and agent localization. Importantly, our proposed approach relaxes several assumptions about idealized localization and perception that are commonly employed by recent works in embodied AI -- an important step towards training agents for object manipulation in the real world.

arXiv:2304.06619

Class-Incremental Learning of Plant and Disease Detection: Growing Branches with Knowledge Distillation

This paper investigates the problem of class-incremental object detection for agricultural applications where a model needs to learn new plant species and diseases incrementally without forgetting the previously learned ones. We adapt two public datasets to include new categories over time, simulating a more realistic and dynamic scenario. We then compare three class-incremental learning methods that leverage different forms of knowledge distillation to mitigate catastrophic forgetting. Our experiments show that all three methods suffer from catastrophic forgetting, but the Dynamic Y-KD approach, which additionally uses a dynamic architecture that grows new branches to learn new tasks, outperforms ILOD and Faster-ILOD in most settings both on new and old classes. These results highlight the challenges and opportunities of continual object detection for agricultural applications. In particular, we hypothesize that the large intra-class and small inter-class variability that is typical of plant images exacerbate the difficulty of learning new categories without interfering with previous knowledge. We publicly release our code to encourage future work.

arXiv:2608.20944v1cs.CV

SuppreSensing: 다중모달 객체 검출을 위한 전문가 가이드 특징 재보정 및 불일치 증강

SuppreSensing: Expert-Guided Feature Recalibration and Discrepancy Augmentation for Multimodal Object Detection

원격 탐사에서의 다중모달 객체 검출은 의미적 이질성과 모달리티별 노이즈 간섭으로 인해 어려움에 직면한다. 이를 위해 우리는 공유 정보와 모달리티별 단서를 공동으로 모델링하는 선택적 협력 과정으로 다중모달 융합을 재공식화하는 SuppreSensing을 제안한다. SuppreSensing은 먼저 공유 합의 추출을 입력 적응형 다중 전문가 선택 과정으로 재공식화하여 다중모달 융합의 대칭성 함정(symmetry trap)을 완화하는 전문가 주도 다중모달 특징 재보정(EMFR) 모듈을 설계한다. 이를 보완하기 위해 양방향 불일치 패턴을 모델링하여 특정 모달리티 특징을 강화하고 교차 모달 이질성을 완화하는 모달리티별 속성 증강 전략을 사용한다. 또한 "전문 검사-종합 분석-진단 갱신" 신체검사 패러다이m에 기반한 전문가 주도 맞춤형 특징 정제(ECFP) 모듈을 제안하여, 중복을 반복적으로 걸러내고 과제 관련 의미를 강화한다. DroneVehicle 및 VEDAI 데이터셋에 대한 광범위한 실험에서 SuppreSensing이 최신(state-of-the-art) 검출 성능을 달성함을 보여준다. 자연 장면 데이터셋(FLIR 및 LLVIP)에 대한 교차 도메인 평가는 다양한 환경 조건에서의 우수한 견고성과 일반화 능력을 추가로 검증한다.

arXiv:2103.01353

There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge

Attributes of sound inherent to objects can provide valuable cues to learn rich representations for object detection and tracking. Furthermore, the co-occurrence of audiovisual events in videos can be exploited to localize objects over the image field by solely monitoring the sound in the environment. Thus far, this has only been feasible in scenarios where the camera is static and for single object detection. Moreover, the robustness of these methods has been limited as they primarily rely on RGB images which are highly susceptible to illumination and weather changes. In this work, we present the novel self-supervised MM-DistillNet framework consisting of multiple teachers that leverage diverse modalities including RGB, depth and thermal images, to simultaneously exploit complementary cues and distill knowledge into a single audio student network. We propose the new MTA loss function that facilitates the distillation of information from multimodal teachers in a self-supervised manner. Additionally, we propose a novel self-supervised pretext task for the audio student that enables us to not rely on labor-intensive manual annotations. We introduce a large-scale multimodal dataset with over 113,000 time-synchronized frames of RGB, depth, thermal, and audio modalities. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods while being able to detect multiple objects using only sound during inference and even while moving.

arXiv:2107.14391공식 코드 · Papers with Code

From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection

As an emerging data modal with precise distance sensing, LiDAR point clouds have been placed great expectations on 3D scene understanding. However, point clouds are always sparsely distributed in the 3D space, and with unstructured storage, which makes it difficult to represent them for effective 3D object detection. To this end, in this work, we regard point clouds as hollow-3D data and propose a new architecture, namely Hallucinated Hollow-3D R-CNN ($\text{H}^2$3D R-CNN), to address the problem of 3D object detection. In our approach, we first extract the multi-view features by sequentially projecting the point clouds into the perspective view and the bird-eye view. Then, we hallucinate the 3D representation by a novel bilaterally guided multi-view fusion block. Finally, the 3D objects are detected via a box refinement module with a novel Hierarchical Voxel RoI Pooling operation. The proposed $\text{H}^2$3D R-CNN provides a new angle to take full advantage of complementary information in the perspective view and the bird-eye view with an efficient framework. We evaluate our approach on the public KITTI Dataset and Waymo Open Dataset. Extensive experiments demonstrate the superiority of our method over the state-of-the-art algorithms with respect to both effectiveness and efficiency. The code will be made available at \url{https://github.com/djiajunustc/H-23D_R-CNN}.

논문 ID: document/1041506

TED-Net: Dispersal Attention for Perceiving Interaction Region in Indirectly-Contact HOI Detection

Human-Object Interaction (HOI) detection is a fertile research ground that merits further investigation in computer vision, and plays an important role in image high-level semantic information understanding. To achieve superior object detection performance, existing HOI models predominantly concentrate on the corresponding bounding box information of humans and objects, respectively, and ignore their surrounding information, thus it results in imprecise inference of instance interaction, which is severe for indirectly-contact interaction images (Intersectionover-Union = 0). To address that, a novel Triple stream Enhanced encoder-decoder Dispersal Network (TED-Net), equipped with human, object, and instance interaction decoding streams, is proposed to decouple instances’ relationships. Meanwhile, we design a dispersal attention mechanism to capture indirectly-contact interaction information and an auxiliary discrimination mechanism to improve the ability of instance interaction decoding stream for action category recognition. Experimental results show that the proposed TED-Net achieves the best performance among HOI models using the ResNet-50 backbone on the (big) HICODet dataset and comes third on the (small) V-COCO dataset in leaderboard. Additionally, two indirectly-contact interaction datasets, namely, HICO-Det-IC and V-COCO-IC, are constructed to demonstrate the usefulness and effectiveness of our TED-Net in interacting between indirectly-contact instances, with an average of +3.80 mAP on HICO-Det-IC and +5.46 mAP on V-COCO-IC. Code is available at https://drliuqi.github.io/.

arXiv:2009.10292

PennSyn2Real: Training Object Recognition Models without Human Labeling

Scalable training data generation is a critical problem in deep learning. We propose PennSyn2Real - a photo-realistic synthetic dataset consisting of more than 100,000 4K images of more than 20 types of micro aerial vehicles (MAVs). The dataset can be used to generate arbitrary numbers of training images for high-level computer vision tasks such as MAV detection and classification. Our data generation framework bootstraps chroma-keying, a mature cinematography technique with a motion tracking system, providing artifact-free and curated annotated images where object orientations and lighting are controlled. This framework is easy to set up and can be applied to a broad range of objects, reducing the gap between synthetic and real-world data. We show that synthetic data generated using this framework can be directly used to train CNN models for common object recognition tasks such as detection and segmentation. We demonstrate competitive performance in comparison with training using only real images. Furthermore, bootstrapping the generated synthetic data in few-shot learning can significantly improve the overall performance, reducing the number of required training data samples to achieve the desired accuracy.

arXiv:2202.03527

Integrated Multiscale Domain Adaptive YOLO

The area of domain adaptation has been instrumental in addressing the domain shift problem encountered by many applications. This problem arises due to the difference between the distributions of source data used for training in comparison with target data used during realistic testing scenarios. In this paper, we introduce a novel MultiScale Domain Adaptive YOLO (MS-DAYOLO) framework that employs multiple domain adaptation paths and corresponding domain classifiers at different scales of the recently introduced YOLOv4 object detector. Building on our baseline multiscale DAYOLO framework, we introduce three novel deep learning architectures for a Domain Adaptation Network (DAN) that generates domain-invariant features. In particular, we propose a Progressive Feature Reduction (PFR), a Unified Classifier (UC), and an Integrated architecture. We train and test our proposed DAN architectures in conjunction with YOLOv4 using popular datasets. Our experiments show significant improvements in object detection performance when training YOLOv4 using the proposed MS-DAYOLO architectures and when tested on target data for autonomous driving applications. Moreover, MS-DAYOLO framework achieves an order of magnitude real-time speed improvement relative to Faster R-CNN solutions while providing comparable object detection performance.

arXiv:2307.10685

Pre-train, Adapt and Detect: Multi-Task Adapter Tuning for Camouflaged Object Detection

Camouflaged object detection (COD), aiming to segment camouflaged objects which exhibit similar patterns with the background, is a challenging task. Most existing works are dedicated to establishing specialized modules to identify camouflaged objects with complete and fine details, while the boundary can not be well located for the lack of object-related semantics. In this paper, we propose a novel ``pre-train, adapt and detect" paradigm to detect camouflaged objects. By introducing a large pre-trained model, abundant knowledge learned from massive multi-modal data can be directly transferred to COD. A lightweight parallel adapter is inserted to adjust the features suitable for the downstream COD task. Extensive experiments on four challenging benchmark datasets demonstrate that our method outperforms existing state-of-the-art COD models by large margins. Moreover, we design a multi-task learning scheme for tuning the adapter to exploit the shareable knowledge across different semantic classes. Comprehensive experimental results showed that the generalization ability of our model can be substantially improved with multi-task adapter initialization on source tasks and multi-task adaptation on target tasks.

arXiv:2503.04308

Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Tasks

Datasets for object detection often do not account for enough variety of glasses, due to their transparent and reflective properties. Specifically, open-vocabulary object detectors, widely used in embodied robotic agents, fail to distinguish subclasses of glasses. This scientific gap poses an issue to robotic applications that suffer from accumulating errors between detection, planning, and action execution. The paper introduces a novel method for the acquisition of real-world data from RGB-D sensors that minimizes human effort. We propose an auto-labeling pipeline that generates labels for all the acquired frames based on the depth measurements. We provide a novel real-world glass object dataset that was collected on the Neuro-Inspired COLlaborator (NICOL), a humanoid robot platform. The data set consists of 7850 images recorded from five different cameras. We show that our trained baseline model outperforms state-of-the-art open-vocabulary approaches. In addition, we deploy our baseline model in an embodied agent approach to the NICOL platform, on which it achieves a success rate of 81% in a human-robot bartending scenario.

arXiv:2204.10007공식 코드 · Papers with Code

Fluctuation-based Outlier Detection

Outlier detection is an important topic in machine learning and has been used in a wide range of applications. Outliers are objects that are few in number and deviate from the majority of objects. As a result of these two properties, we show that outliers are susceptible to a mechanism called fluctuation. This article proposes a method called fluctuation-based outlier detection (FBOD) that achieves a low linear time complexity and detects outliers purely based on the concept of fluctuation without employing any distance, density or isolation measure. Fundamentally different from all existing methods. FBOD first converts the Euclidean structure datasets into graphs by using random links, then propagates the feature value according to the connection of the graph. Finally, by comparing the difference between the fluctuation of an object and its neighbors, FBOD determines the object with a larger difference as an outlier. The results of experiments comparing FBOD with seven state-of-the-art algorithms on eight real-world tabular datasets and three video datasets show that FBOD outperforms its competitors in the majority of cases and that FBOD has only 5% of the execution time of the fastest algorithm. The experiment codes are available at: https://github.com/FluctuationOD/Fluctuation-based-Outlier-Detection.

arXiv:2205.05412

An Objective Method for Pedestrian Occlusion Level Classification

Pedestrian detection is among the most safety-critical features of driver assistance systems for autonomous vehicles. One of the most complex detection challenges is that of partial occlusion, where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of current pedestrian detection benchmarks provide annotation for partial occlusion to assess algorithm performance in these scenarios, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. In addition, current occlusion level annotation methods contain a high degree of subjectivity by the human annotator. This can lead to inaccurate or inconsistent reporting of an algorithm's detection performance for partially occluded pedestrians, depending on which benchmark is used. This research presents a novel, objective method for pedestrian occlusion level classification for ground truth annotation. Occlusion level classification is achieved through the identification of visible pedestrian keypoints and through the use of a novel, effective method of 2D body surface area estimation. Experimental results demonstrate that the proposed method reflects the pixel-wise occlusion level of pedestrians in images and is effective for all forms of occlusion, including challenging edge cases such as self-occlusion, truncation and inter-occluding pedestrians.

arXiv:2303.14488공식 코드 · Papers with Code

Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Images

Object detection on drone images with low-latency is an important but challenging task on the resource-constrained unmanned aerial vehicle (UAV) platform. This paper investigates optimizing the detection head based on the sparse convolution, which proves effective in balancing the accuracy and efficiency. Nevertheless, it suffers from inadequate integration of contextual information of tiny objects as well as clumsy control of the mask ratio in the presence of foreground with varying scales. To address the issues above, we propose a novel global context-enhanced adaptive sparse convolutional network (CEASC). It first develops a context-enhanced group normalization (CE-GN) layer, by replacing the statistics based on sparsely sampled features with the global contextual ones, and then designs an adaptive multi-layer masking strategy to generate optimal mask ratios at distinct scales for compact foreground coverage, promoting both the accuracy and efficiency. Extensive experimental results on two major benchmarks, i.e. VisDrone and UAVDT, demonstrate that CEASC remarkably reduces the GFLOPs and accelerates the inference procedure when plugging into the typical state-of-the-art detection frameworks (e.g. RetinaNet and GFL V1) with competitive performance. Code is available at https://github.com/Cuogeihong/CEASC.

arXiv:2511.15343v2cs.CV

견고한 자율 주행을 위한 3-way 오픈셋 탐지

Three-Way Open-Set Detection for Robust Autonomous Navigation

복잡한 장면에서의 자율 주행은 모델이 학습 중에 접하지 못한 시나리오 전반에 걸쳐 신뢰할 수 있는 인식을 필요로 합니다. 이동 경로를 따라가는 동안 자율 주행 프레임워크는 인식하도록 학습된 객체, 한 번도 본 적 없는 장애물, 그리고 객체와 유사하게 보이는 배경 구조를 마주하게 됩니다. 이 세 가지는 각각 다르게 처리되어야 합니다. 이를 해결하기 위해 기존의 오픈셋(open-set) 및 분포 외(out-of-distribution) 탐지기는 객체성(objectness) 임계값으로 신뢰도가 낮은 탐지 결과를 먼저 제거한 뒤, 나머지에 대해서만 새로움(novelty)을 검사합니다. 이처럼 단일 임계값을 강제하면 트레이드오프가 발생하는데, 임계값이 낮으면 탐지된 객체에 배경 잡음이 추가되고, 임계값이 높으면 필요한 새로운 객체가 제거될 수 있습니다. 이에 우리는 오픈셋 탐지를 각 탐지 결과를 알려진 객체, 알려지지 않은 객체, 배경 중 하나로 분류하는 3-way 분류(three-way classification)로 정식화하며, 이는 사전학습된 탐지기의 출력으로부터 사후(post hoc)에 계산됩니다. 우리는 도메인 일반화(domain generalization)와 도메인 적응(domain adaptation)을 위한 방법을 개발하고, 의미론적 도메인 이동과 공변량 도메인 이동이 결합된 경우까지 포함하여 다양한 탐지기 계열과 벤치마크에서 평가합니다. 내비게이션 상황에서 이 프레임워크를 검증하기 위해, 측정된 탐지 성능으로 매개변수화한 시뮬레이션을 수행합니다. 그 결과, 3-way 결정이 이진(binary) 대안보다 더 안전하고 효율적인 임무 수행을 가능하게 함을 보여줍니다.

arXiv:2308.13183공식 코드 · Papers with Code

STRIDE: Street View-based Environmental Feature Detection and Pedestrian Collision Prediction

This paper introduces a novel benchmark to study the impact and relationship of built environment elements on pedestrian collision prediction, intending to enhance environmental awareness in autonomous driving systems to prevent pedestrian injuries actively. We introduce a built environment detection task in large-scale panoramic images and a detection-based pedestrian collision frequency prediction task. We propose a baseline method that incorporates a collision prediction module into a state-of-the-art detection model to tackle both tasks simultaneously. Our experiments demonstrate a significant correlation between object detection of built environment elements and pedestrian collision frequency prediction. Our results are a stepping stone towards understanding the interdependencies between built environment conditions and pedestrian safety.