PyTorchKR Papers

“Object Detection”

총 13,435편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2608.00232v2cs.CV

수술용 증강현실에서의 실시간 시각적 가림 감지

Real-Time Visual Obstruction Detection in Surgical Augmented Reality

수술용 증강현실(AR)은 수술 작업 공간 위에 가상 주석, 도구 신호, 절차적 정보를 중첩하여 상황별 가이던스를 제공할 수 있다. 그러나 가상 콘텐츠는 수술 도구와 같이 작업과 관련된 실제 정보를 가릴 수 있으며, 시간에 민감한 수술 작업 중에 사용자의 인식을 방해할 수 있다. 본 논문에서는 수술용 AR을 위한 시각적 가림 감지(visual obstruction detection)를 조사하고, 비전-언어 모델(VLM) 기반의 수술 객체 인식과 세그멘테이션 기반 가림 추론을 결합한 지연시간 인식(latency-aware) 파이프라인을 제시한다. 추론(inference) 오버헤드를 줄이기 위해, 이 시스템은 세그멘테이션 유도 조기 종료(early exiting)와 어텐션(attention) 기반 시각적 토큰(token) 가지치기(pruning)를 갖춘 계단식 소형-대형 VLM 아키텍처를 채택한다. 소형 VLM은 주요 객체 예측이 세그멘테이션 일관성에 의해 뒷받침될 때 쉬운 프레임을 처리하는 반면, 어려운 프레임은 가지치기된 시각적 토큰과 함께 대형 VLM으로 전달된다. 저자들은 수술 도구 이미지 위에 가상 콘텐츠를 중첩하고 가상 콘텐츠가 작업 관련 도구를 가리는지 여부를 라벨링하여 유사 AR 수술 가림 감지 벤치마크를 구축한다. 평가 결과에 따르면, 제안된 시스템은 평균 479ms의 종단간(end-to-end) 지연시간으로 87.43%의 가림 감지 정확도를 달성하여 클라우드 대형 모델 기준선(baseline) 대비 지연시간을 62.90% 단축한다. 이러한 결과는 수술용 AR을 위한 지연시간 인식 가림 감지의 실현 가능성을 입증하며, 동적 수술 비디오, 다중 객체 장면, 임상적으로 근거가 있는 AR 가이던스 콘텐츠에 대한 향후 연구의 동기를 부여한다.

arXiv:2009.05307

A Density-Aware PointRCNN for 3D Object Detection in Point Clouds

We present an improved version of PointRCNN for 3D object detection, in which a multi-branch backbone network is adopted to handle the non-uniform density of point clouds. An uncertainty-based sampling policy is proposed to deal with the distribution differences of different point clouds. The new model can achieve about 0.8 AP higher performance than the baseline PointRCNN on KITTI val set. In addition, a simplified model using a single scale grouping for each set-abstraction layer can achieve competitive performance with less computational cost.

arXiv:2204.08745

Augmentation of Atmospheric Turbulence Effects on Thermal Adapted Object Detection Models

Atmospheric turbulence has a degrading effect on the image quality of long-range observation systems. As a result of various elements such as temperature, wind velocity, humidity, etc., turbulence is characterized by random fluctuations in the refractive index of the atmosphere. It is a phenomenon that may occur in various imaging spectra such as the visible or the infrared bands. In this paper, we analyze the effects of atmospheric turbulence on object detection performance in thermal imagery. We use a geometric turbulence model to simulate turbulence effects on a medium-scale thermal image set, namely "FLIR ADAS v2". We apply thermal domain adaptation to state-of-the-art object detectors and propose a data augmentation strategy to increase the performance of object detectors which utilizes turbulent images in different severity levels as training data. Our results show that the proposed data augmentation strategy yields an increase in performance for both turbulent and non-turbulent thermal test images.

arXiv:2306.13301공식 코드 · Papers with Code

Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images

Deep learning (DL)-based rib fracture detection has shown promise of playing an important role in preventing mortality and improving patient outcome. Normally, developing DL-based object detection models requires a huge amount of bounding box annotation. However, annotating medical data is time-consuming and expertise-demanding, making obtaining a large amount of fine-grained annotations extremely infeasible. This poses a pressing need {for} developing label-efficient detection models to alleviate radiologists' labeling burden. To tackle this challenge, the literature on object detection has witnessed an increase of weakly-supervised and semi-supervised approaches, yet still lacks a unified framework that leverages various forms of fully-labeled, weakly-labeled, and unlabeled data. In this paper, we present a novel omni-supervised object detection network, ORF-Netv2, to leverage as much available supervision as possible. Specifically, a multi-branch omni-supervised detection head is introduced with each branch trained with a specific type of supervision. A co-training-based dynamic label assignment strategy is then proposed to enable flexible and robust learning from the weakly-labeled and unlabeled data. Extensive evaluation was conducted for the proposed framework with three rib fracture datasets on both chest CT and X-ray. By leveraging all forms of supervision, ORF-Netv2 achieves mAPs of 34.7, 44.7, and 19.4 on the three datasets, respectively, surpassing the baseline detector which uses only box annotations by mAP gains of 3.8, 4.8, and 5.0, respectively. Furthermore, ORF-Netv2 consistently outperforms other competitive label-efficient methods over various scenarios, showing a promising framework for label-efficient fracture detection. The code is available at: https://github.com/zhizhongchai/ORF-Net.

arXiv:2003.12244

One-Shot GAN Generated Fake Face Detection

Fake face detection is a significant challenge for intelligent systems as generative models become more powerful every single day. As the quality of fake faces increases, the trained models become more and more inefficient to detect the novel fake faces, since the corresponding training data is considered outdated. In this case, robust One-Shot learning methods is more compatible with the requirements of changeable training data. In this paper, we propose a universal One-Shot GAN generated fake face detection method which can be used in significantly different areas of anomaly detection. The proposed method is based on extracting out-of-context objects from faces via scene understanding models. To do so, we use state of the art scene understanding and object detection methods as a pre-processing tool to detect the weird objects in the face. Second, we create a bag of words given all the detected out-of-context objects per all training data. This way, we transform each image into a sparse vector where each feature represents the confidence score related to each detected object in the image. Our experiments show that, we can discriminate fake faces from real ones in terms of out-of-context features. It means that, different sets of objects are detected in fake faces comparing to real ones when we analyze them with scene understanding and object detection models. We prove that, the proposed method can outperform previous methods based on our experiments on Style-GAN generated fake faces.

arXiv:1912.01674공식 코드 · Papers with Code

Learning to Separate: Detecting Heavily-Occluded Objects in Urban Scenes

While visual object detection with deep learning has received much attention in the past decade, cases when heavy intra-class occlusions occur have not been studied thoroughly. In this work, we propose a Non-Maximum-Suppression (NMS) algorithm that dramatically improves the detection recall while maintaining high precision in scenes with heavy occlusions. Our NMS algorithm is derived from a novel embedding mechanism, in which the semantic and geometric features of the detected boxes are jointly exploited. The embedding makes it possible to determine whether two heavily-overlapping boxes belong to the same object in the physical world. Our approach is particularly useful for car detection and pedestrian detection in urban scenes where occlusions often happen. We show the effectiveness of our approach by creating a model called SG-Det (short for Semantics and Geometry Detection) and testing SG-Det on two widely-adopted datasets, KITTI and CityPersons for which it achieves state-of-the-art performance.

arXiv:1807.00436

Liver Lesion Detection from Weakly-labeled Multi-phase CT Volumes with a Grouped Single Shot MultiBox Detector

We present a focal liver lesion detection model leveraged by custom-designed multi-phase computed tomography (CT) volumes, which reflects real-world clinical lesion detection practice using a Single Shot MultiBox Detector (SSD). We show that grouped convolutions effectively harness richer information of the multi-phase data for the object detection model, while a naive application of SSD suffers from a generalization gap. We trained and evaluated the modified SSD model and recently proposed variants with our CT dataset of 64 subjects by five-fold cross validation. Our model achieved a 53.3% average precision score and ran in under three seconds per volume, outperforming the original model and state-of-the-art variants. Results show that the one-stage object detection model is a practical solution, which runs in near real-time and can learn an unbiased feature representation from a large-volume real-world detection dataset, which requires less tedious and time consuming construction of the weak phase-level bounding box labels.

arXiv:2011.12790공식 코드 · Papers with Code

Fast Region Proposal Learning for Object Detection for Robotics

Object detection is a fundamental task for robots to operate in unstructured environments. Today, there are several deep learning algorithms that solve this task with remarkable performance. Unfortunately, training such systems requires several hours of GPU time. For robots, to successfully adapt to changes in the environment or learning new objects, it is also important that object detectors can be re-trained in a short amount of time. A recent method [1] proposes an architecture that leverages on the powerful representation of deep learning descriptors, while permitting fast adaptation time. Leveraging on the natural decomposition of the task in (i) regions candidate generation, (ii) feature extraction and (iii) regions classification, this method performs fast adaptation of the detector, by only re-training the classification layer. This shortens training time while maintaining state-of-the-art performance. In this paper, we firstly demonstrate that a further boost in accuracy can be obtained by adapting, in addition, the regions candidate generation on the task at hand. Secondly, we extend the object detection system presented in [1] with the proposed fast learning approach, showing experimental evidence on the improvement provided in terms of speed and accuracy on two different robotics datasets. The code to reproduce the experiments is publicly available on GitHub.

arXiv:2106.05499

AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection

Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. Unfortunately, it has received much less attention than supervised object detection. Models that try to address this task tend to suffer from a shortage of annotated training samples. Moreover, existing methods of feature alignments are not sufficient to learn domain-invariant representations. To address these limitations, we propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training into a unified framework. An intermediate domain image generator is proposed to enhance feature alignments by domain-adversarial training with automatically generated soft domain labels. The synthetic intermediate domain images progressively bridge the domain divergence and augment the annotated source domain training data. A feature pyramid alignment is designed and the corresponding feature discriminator is used to align multi-scale convolutional features of different semantic levels. Last but not least, we introduce a region feature alignment and an instance discriminator to learn domain-invariant features for object proposals. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations. Further extensive experiments verify the effectiveness of each component and demonstrate that the proposed network can learn domain-invariant representations.

arXiv:2108.10809

Reconcile Prediction Consistency for Balanced Object Detection

Classification and regression are two pillars of object detectors. In most CNN-based detectors, these two pillars are optimized independently. Without direct interactions between them, the classification loss and the regression loss can not be optimized synchronously toward the optimal direction in the training phase. This clearly leads to lots of inconsistent predictions with high classification score but low localization accuracy or low classification score but high localization accuracy in the inference phase, especially for the objects of irregular shape and occlusion, which severely hurts the detection performance of existing detectors after NMS. To reconcile prediction consistency for balanced object detection, we propose a Harmonic loss to harmonize the optimization of classification branch and localization branch. The Harmonic loss enables these two branches to supervise and promote each other during training, thereby producing consistent predictions with high co-occurrence of top classification and localization in the inference phase. Furthermore, in order to prevent the localization loss from being dominated by outliers during training phase, a Harmonic IoU loss is proposed to harmonize the weight of the localization loss of different IoU-level samples. Comprehensive experiments on benchmarks PASCAL VOC and MS COCO demonstrate the generality and effectiveness of our model for facilitating existing object detectors to state-of-the-art accuracy.

arXiv:2206.14451공식 코드 · Papers with Code

SRCN3D: Sparse R-CNN 3D for Compact Convolutional Multi-View 3D Object Detection and Tracking

Detection and tracking of moving objects is an essential component in environmental perception for autonomous driving. In the flourishing field of multi-view 3D camera-based detectors, different transformer-based pipelines are designed to learn queries in 3D space from 2D feature maps of perspective views, but the dominant dense BEV query mechanism is computationally inefficient. This paper proposes Sparse R-CNN 3D (SRCN3D), a novel two-stage fully-sparse detector that incorporates sparse queries, sparse attention with box-wise sampling, and sparse prediction. SRCN3D adopts a cascade structure with the twin-track update of both a fixed number of query boxes and latent query features. Our novel sparse feature sampling module only utilizes local 2D region of interest (RoI) features calculated by the projection of 3D query boxes for further box refinement, leading to a fully-convolutional and deployment-friendly pipeline. For multi-object tracking, motion features, query features and RoI features are comprehensively utilized in multi-hypotheses data association. Extensive experiments on nuScenes dataset demonstrate that SRCN3D achieves competitive performance in both 3D object detection and multi-object tracking tasks, while also exhibiting superior efficiency compared to transformer-based methods. Code and models are available at https://github.com/synsin0/SRCN3D.

arXiv:2409.16808v3cs.CV

이기종 엣지 디바이스에서 심층 학습 객체 검출 모델의 종합 평가

A Comprehensive Evaluation of Deep Learning Object Detection Models on Heterogeneous Edge Devices

자율주행차, 지능형 감시, 스마트 시티 시스템과 같은 현대 애플리케이션은 자원이 제한된 엣지 디바이스에서의 객체 검출을 점점 더 많이 요구하고 있다. 그러나 이기종 엣지 디바이스와 다양한 장면 복잡도에서 서로 다른 객체 검출 모델이 어떻게 동작하는지에 대한 이해는 아직 제한적이다. 본 논문에서는 Raspberry Pi 3, 4, 5(Coral TPU 가속기 유/무), AI HAT+를 장착한 Raspberry Pi 5, Jetson Nano, Jetson Orin Nano에서 YOLOv8(Nano, Small, Medium), EfficientDet Lite(Lite0, Lite1, Lite2), SSD(SSD MobileNet V1, SSDLite MobileDet)을 벤치마크한다. 에너지 소비, 추론 시간, 정확도를 평가하고, 입력 이미지의 객체 수에 따라 정확도가 어떻게 변하는지 추가로 분석한다. 결과는 모델-디바이스 조합 전반에 걸쳐 정확도, 지연 시간, 에너지 효율 간의 뚜렷한 트레이드오프를 보여준다. SSD MobileNet V1은 가장 낮은 지연 시간과 에너지 소비를 달성하지만 정확도는 가장 낮고, YOLOv8 Medium은 더 높은 연산 비용으로 가장 높은 정확도를 달성한다. TPU 기반 Raspberry Pi 디바이스는 SSD와 EfficientDet Lite의 효율을 향상시키는 동시에 YOLOv8의 정확도를 낮춘다. Orin Nano는 대부분의 모델 계열에서 가장 유리한 전반적 균형을 제공한다. 객체 수 기반 분석은 단순한 이미지에서는 모델들이 더 유사한 정확도를 달성하는 반면, 장면 복잡도가 증가할수록 정확도 격차가 커진다는 것을 추가로 보여준다.

arXiv:2411.05863공식 코드 · Papers with Code

Exploring the Feasibility of Affordable Sonar Technology: Object Detection in Underwater Environments Using the Ping 360

This study explores the potential of the Ping 360 sonar device, primarily used for navigation, in detecting complex underwater obstacles. The key motivation behind this research is the device's affordability and open-source nature, offering a cost-effective alternative to more expensive imaging sonar systems. The investigation focuses on understanding the behaviour of the Ping 360 in controlled environments and assessing its suitability for object detection, particularly in scenarios where human operators are unavailable for inspecting offshore structures in shallow waters. Through a series of carefully designed experiments, we examined the effects of surface reflections and object shadows in shallow underwater environments. Additionally, we developed a manually annotated sonar image dataset to train a U-Net segmentation model. Our findings indicate that while the Ping 360 sonar demonstrates potential in simpler settings, its performance is limited in more cluttered or reflective environments unless extensive data pre-processing and annotation are applied. To our knowledge, this is the first study to evaluate the Ping 360's capabilities for complex object detection. By investigating the feasibility of low-cost sonar devices, this research provides valuable insights into their limitations and potential for future AI-based interpretation, marking a unique contribution to the field.

arXiv:1506.08347공식 코드 · Papers with Code

Occlusion Coherence: Detecting and Localizing Occluded Faces

The presence of occluders significantly impacts object recognition accuracy. However, occlusion is typically treated as an unstructured source of noise and explicit models for occluders have lagged behind those for object appearance and shape. In this paper we describe a hierarchical deformable part model for face detection and landmark localization that explicitly models part occlusion. The proposed model structure makes it possible to augment positive training data with large numbers of synthetically occluded instances. This allows us to easily incorporate the statistics of occlusion patterns in a discriminatively trained model. We test the model on several benchmarks for landmark localization and detection including challenging new data sets featuring significant occlusion. We find that the addition of an explicit occlusion model yields a detection system that outperforms existing approaches for occluded instances while maintaining competitive accuracy in detection and landmark localization for unoccluded instances.

arXiv:2007.00493

Optimisation of the PointPillars network for 3D object detection in point clouds

In this paper we present our research on the optimisation of a deep neural network for 3D object detection in a point cloud. Techniques like quantisation and pruning available in the Brevitas and PyTorch tools were used. We performed the experiments for the PointPillars network, which offers a reasonable compromise between detection accuracy and calculation complexity. The aim of this work was to propose a variant of the network which we will ultimately implement in an FPGA device. This will allow for real-time LiDAR data processing with low energy consumption. The obtained results indicate that even a significant quantisation from 32-bit floating point to 2-bit integer in the main part of the algorithm, results in 5%-9% decrease of the detection accuracy, while allowing for almost a 16-fold reduction in size of the model.

arXiv:2202.00632

Reducing the Amount of Real World Data for Object Detector Training with Synthetic Data

A number of studies have investigated the training of neural networks with synthetic data for applications in the real world. The aim of this study is to quantify how much real world data can be saved when using a mixed dataset of synthetic and real world data. By modeling the relationship between the number of training examples and detection performance by a simple power law, we find that the need for real world data can be reduced by up to 70% without sacrificing detection performance. The training of object detection networks is especially enhanced by enriching the mixed dataset with classes underrepresented in the real world dataset. The results indicate that mixed datasets with real world data ratios between 5% and 20% reduce the need for real world data the most without reducing the detection performance.

arXiv:2402.18698

Spatial Coherence Loss: All Objects Matter in Salient and Camouflaged Object Detection

Generic object detection is a category-independent task that relies on accurate modeling of objectness. We show that for accurate semantic analysis, the network needs to learn all object-level predictions that appear at any stage of learning, including the pre-defined ground truth (GT) objects and the ambiguous decoy objects that the network misidentifies as foreground. Yet, most relevant models focused mainly on improving the learning of the GT objects. A few methods that consider decoy objects utilize loss functions that only focus on the single-response, i.e., the loss response of a single ambiguous pixel, and thus do not benefit from the wealth of information that an object-level ambiguity learning design can provide. Inspired by the human visual system, which first discerns the boundaries of ambiguous regions before delving into the semantic meaning, we propose a novel loss function, Spatial Coherence Loss (SCLoss), that incorporates the mutual response between adjacent pixels into the widely-used single-response loss functions. We demonstrate that the proposed SCLoss can gradually learn the ambiguous regions by detecting and emphasizing their boundaries in a self-adaptive manner. Through comprehensive experiments, we demonstrate that replacing popular loss functions with SCLoss can improve the performance of current state-of-the-art (SOTA) salient or camouflaged object detection (SOD or COD) models. We also demonstrate that combining SCLoss with other loss functions can further improve performance and result in SOTA outcomes for different applications.

arXiv:1902.09630

Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression

Intersection over Union (IoU) is the most popular evaluation metric used in the object detection benchmarks. However, there is a gap between optimizing the commonly used distance losses for regressing the parameters of a bounding box and maximizing this metric value. The optimal objective for a metric is the metric itself. In the case of axis-aligned 2D bounding boxes, it can be shown that $IoU$ can be directly used as a regression loss. However, $IoU$ has a plateau making it infeasible to optimize in the case of non-overlapping bounding boxes. In this paper, we address the weaknesses of $IoU$ by introducing a generalized version as both a new loss and a new metric. By incorporating this generalized $IoU$ ($GIoU$) as a loss into the state-of-the art object detection frameworks, we show a consistent improvement on their performance using both the standard, $IoU$ based, and new, $GIoU$ based, performance measures on popular object detection benchmarks such as PASCAL VOC and MS COCO.

arXiv:1903.08701

LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving

In this paper, we present LaserNet, a computationally efficient method for 3D object detection from LiDAR data for autonomous driving. The efficiency results from processing LiDAR data in the native range view of the sensor, where the input data is naturally compact. Operating in the range view involves well known challenges for learning, including occlusion and scale variation, but it also provides contextual information based on how the sensor data was captured. Our approach uses a fully convolutional network to predict a multimodal distribution over 3D boxes for each point and then it efficiently fuses these distributions to generate a prediction for each object. Experiments show that modeling each detection as a distribution rather than a single deterministic box leads to better overall detection performance. Benchmark results show that this approach has significantly lower runtime than other recent detectors and that it achieves state-of-the-art performance when compared on a large dataset that has enough data to overcome the challenges of training on the range view.

arXiv:2206.00309공식 코드 · Papers with Code

Label-Efficient Online Continual Object Detection in Streaming Video

Humans can watch a continuous video stream and effortlessly perform continual acquisition and transfer of new knowledge with minimal supervision yet retaining previously learnt experiences. In contrast, existing continual learning (CL) methods require fully annotated labels to effectively learn from individual frames in a video stream. Here, we examine a more realistic and challenging problem$\unicode{x2014}$Label-Efficient Online Continual Object Detection (LEOCOD) in streaming video. We propose a plug-and-play module, Efficient-CLS, that can be easily inserted into and improve existing continual learners for object detection in video streams with reduced data annotation costs and model retraining time. We show that our method has achieved significant improvement with minimal forgetting across all supervision levels on two challenging CL benchmarks for streaming real-world videos. Remarkably, with only 25% annotated video frames, our method still outperforms the base CL learners, which are trained with 100% annotations on all video frames. The data and source code will be publicly available at https://github.com/showlab/Efficient-CLS.