PyTorchKR Papers

“Object Detection”

총 13,455편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2501.16754

SSF-PAN: Semantic Scene Flow-Based Perception for Autonomous Navigation in Traffic Scenarios

Vehicle detection and localization in complex traffic scenarios pose significant challenges due to the interference of moving objects. Traditional methods often rely on outlier exclusions or semantic segmentations, which suffer from low computational efficiency and accuracy. The proposed SSF-PAN can achieve the functionalities of LiDAR point cloud based object detection/localization and SLAM (Simultaneous Localization and Mapping) with high computational efficiency and accuracy, enabling map-free navigation frameworks. The novelty of this work is threefold: 1) developing a neural network which can achieve segmentation among static and dynamic objects within the scene flows with different motion features, that is, semantic scene flow (SSF); 2) developing an iterative framework which can further optimize the quality of input scene flows and output segmentation results; 3) developing a scene flow-based navigation platform which can test the performance of the SSF perception system in the simulation environment. The proposed SSF-PAN method is validated using the SUScape-CARLA and the KITTI datasets, as well as on the CARLA simulator. Experimental results demonstrate that the proposed approach outperforms traditional methods in terms of scene flow computation accuracy, moving object detection accuracy, computational efficiency, and autonomous navigation effectiveness.

arXiv:1602.01728

NeRD: a Neural Response Divergence Approach to Visual Salience Detection

In this paper, a novel approach to visual salience detection via Neural Response Divergence (NeRD) is proposed, where synaptic portions of deep neural networks, previously trained for complex object recognition, are leveraged to compute low level cues that can be used to compute image region distinctiveness. Based on this concept , an efficient visual salience detection framework is proposed using deep convolutional StochasticNets. Experimental results using CSSD and MSRA10k natural image datasets show that the proposed NeRD approach can achieve improved performance when compared to state-of-the-art image saliency approaches, while the attaining low computational complexity necessary for near-real-time computer vision applications.

arXiv:2206.02977

DETR++: Taming Your Multi-Scale Detection Transformer

Convolutional Neural Networks (CNN) have dominated the field of detection ever since the success of AlexNet in ImageNet classification [12]. With the sweeping reform of Transformers [27] in natural language processing, Carion et al. [2] introduce the Transformer-based detection method, i.e., DETR. However, due to the quadratic complexity in the self-attention mechanism in the Transformer, DETR is never able to incorporate multi-scale features as performed in existing CNN-based detectors, leading to inferior results in small object detection. To mitigate this issue and further improve performance of DETR, in this work, we investigate different methods to incorporate multi-scale features and find that a Bi-directional Feature Pyramid (BiFPN) works best with DETR in further raising the detection precision. With this discovery, we propose DETR++, a new architecture that improves detection results by 1.9% AP on MS COCO 2017, 11.5% AP on RICO icon detection, and 9.1% AP on RICO layout extraction over existing baselines.

arXiv:1811.10216

A Survey on Joint Object Detection and Pose Estimation using Monocular Vision

In this survey we present a complete landscape of joint object detection and pose estimation methods that use monocular vision. Descriptions of traditional approaches that involve descriptors or models and various estimation methods have been provided. These descriptors or models include chordiograms, shape-aware deformable parts model, bag of boundaries, distance transform templates, natural 3D markers and facet features whereas the estimation methods include iterative clustering estimation, probabilistic networks and iterative genetic matching. Hybrid approaches that use handcrafted feature extraction followed by estimation by deep learning methods have been outlined. We have investigated and compared, wherever possible, pure deep learning based approaches (single stage and multi stage) for this problem. Comprehensive details of the various accuracy measures and metrics have been illustrated. For the purpose of giving a clear overview, the characteristics of relevant datasets are discussed. The trends that prevailed from the infancy of this problem until now have also been highlighted.

arXiv:2106.13603

Partially fake it till you make it: mixing real and fake thermal images for improved object detection

In this paper we propose a novel data augmentation approach for visual content domains that have scarce training datasets, compositing synthetic 3D objects within real scenes. We show the performance of the proposed system in the context of object detection in thermal videos, a domain where 1) training datasets are very limited compared to visible spectrum datasets and 2) creating full realistic synthetic scenes is extremely cumbersome and expensive due to the difficulty in modeling the thermal properties of the materials of the scene. We compare different augmentation strategies, including state of the art approaches obtained through RL techniques, the injection of simulated data and the employment of a generative model, and study how to best combine our proposed augmentation with these other techniques.Experimental results demonstrate the effectiveness of our approach, and our single-modality detector achieves state-of-the-art results on the FLIR ADAS dataset.

arXiv:2207.05921공식 코드 · Papers with Code

Texture-guided Saliency Distilling for Unsupervised Salient Object Detection

Deep Learning-based Unsupervised Salient Object Detection (USOD) mainly relies on the noisy saliency pseudo labels that have been generated from traditional handcraft methods or pre-trained networks. To cope with the noisy labels problem, a class of methods focus on only easy samples with reliable labels but ignore valuable knowledge in hard samples. In this paper, we propose a novel USOD method to mine rich and accurate saliency knowledge from both easy and hard samples. First, we propose a Confidence-aware Saliency Distilling (CSD) strategy that scores samples conditioned on samples' confidences, which guides the model to distill saliency knowledge from easy samples to hard samples progressively. Second, we propose a Boundary-aware Texture Matching (BTM) strategy to refine the boundaries of noisy labels by matching the textures around the predicted boundary. Extensive experiments on RGB, RGB-D, RGB-T, and video SOD benchmarks prove that our method achieves state-of-the-art USOD performance.

arXiv:2305.15097

Computer Vision for Construction Progress Monitoring: A Real-Time Object Detection Approach

Construction progress monitoring (CPM) is essential for effective project management, ensuring on-time and on-budget delivery. Traditional CPM methods often rely on manual inspection and reporting, which are time-consuming and prone to errors. This paper proposes a novel approach for automated CPM using state-of-the-art object detection algorithms. The proposed method leverages e.g. YOLOv8's real-time capabilities and high accuracy to identify and track construction elements within site images and videos. A dataset was created, consisting of various building elements and annotated with relevant objects for training and validation. The performance of the proposed approach was evaluated using standard metrics, such as precision, recall, and F1-score, demonstrating significant improvement over existing methods. The integration of Computer Vision into CPM provides stakeholders with reliable, efficient, and cost-effective means to monitor project progress, facilitating timely decision-making and ultimately contributing to the successful completion of construction projects.

arXiv:2607.12874v1cs.CV

에이전트형 AI와 호환되는 심층 학습을 위한 메트릭 기반 합성 이미지 데이터 렌더링

Metric-Guided Synthetic Image Data Rendering for Deep Learning compatible with Agentic AI

과학 응용을 위한 심층 학습 컴퓨터 비전은 대규모 데이터셋을 수집하고 주석을 달아야 하는데, 이 과정은 힘들고 비용이 많이 들며 오류가 발생하기 쉽다. 3D 모델링과 렌더링을 통한 합성 데이터 생성은 주석을 프로그래밍 방식으로 생성함으로써 이 과정을 단순화하고 주석 정확도를 높일 수 있다. 그러나 실제 이미지와 합성 이미지 사이의 도메인 격차(domain gap)를 시각적으로 최소화하는 것은 주관적이며 체계적인 정량적 가이드가 부족하다. 우리는 렌더링된 장면의 개선을 안내하는 메트릭을 포함한 Python 패키지 GraNatPy를 제시한다. 렌더링된 데이터셋의 사실성, 다양성, 규모가 정량적으로 증가하는 것이 장면에 대한 시각적 인식 개선 및 객체 검출 모델의 제로샷 성능 향상과 상관관계가 있음을 보인다. 또한 바이러스학적 플라크 분석 사진을 이용해 그래디언트 유사성이 소형 객체 검출 성능에 영향을 미친다는 것을 보였으며, 이는 실제 데이터와 합성 데이터를 혼합하면 개선할 수 있다. 마지막으로 절차적 매개변수 최적화를 자동화하기 위해 절차적 데이터 렌더링을 에이전트형 기술(SynthClaw)로 전환한다.

arXiv:1411.7911

On Rendering Synthetic Images for Training an Object Detector

We propose a novel approach to synthesizing images that are effective for training object detectors. Starting from a small set of real images, our algorithm estimates the rendering parameters required to synthesize similar images given a coarse 3D model of the target object. These parameters can then be reused to generate an unlimited number of training images of the object of interest in arbitrary 3D poses, which can then be used to increase classification performances. A key insight of our approach is that the synthetically generated images should be similar to real images, not in terms of image quality, but rather in terms of features used during the detector training. We show in the context of drone, plane, and car detection that using such synthetically generated images yields significantly better performances than simply perturbing real images or even synthesizing images in such way that they look very realistic, as is often done when only limited amounts of training data are available.

arXiv:2308.09564공식 코드 · Papers with Code

Deep Equilibrium Object Detection

Query-based object detectors directly decode image features into object instances with a set of learnable queries. These query vectors are progressively refined to stable meaningful representations through a sequence of decoder layers, and then used to directly predict object locations and categories with simple FFN heads. In this paper, we present a new query-based object detector (DEQDet) by designing a deep equilibrium decoder. Our DEQ decoder models the query vector refinement as the fixed point solving of an {implicit} layer and is equivalent to applying {infinite} steps of refinement. To be more specific to object decoding, we use a two-step unrolled equilibrium equation to explicitly capture the query vector refinement. Accordingly, we are able to incorporate refinement awareness into the DEQ training with the inexact gradient back-propagation (RAG). In addition, to stabilize the training of our DEQDet and improve its generalization ability, we devise the deep supervision scheme on the optimization path of DEQ with refinement-aware perturbation~(RAP). Our experiments demonstrate DEQDet converges faster, consumes less memory, and achieves better results than the baseline counterpart (AdaMixer). In particular, our DEQDet with ResNet50 backbone and 300 queries achieves the $49.5$ mAP and $33.0$ AP$_s$ on the MS COCO benchmark under $2\times$ training scheme (24 epochs).

arXiv:2410.05721

Mero Nagarikta: Advanced Nepali Citizenship Data Extractor with Deep Learning-Powered Text Detection and OCR

Transforming text-based identity documents, such as Nepali citizenship cards, into a structured digital format poses several challenges due to the distinct characteristics of the Nepali script and minor variations in print alignment and contrast across different cards. This work proposes a robust system using YOLOv8 for accurate text object detection and an OCR algorithm based on Optimized PyTesseract. The system, implemented within the context of a mobile application, allows for the automated extraction of important textual information from both the front and the back side of Nepali citizenship cards, including names, citizenship numbers, and dates of birth. The final YOLOv8 model was accurate, with a mean average precision of 99.1% for text detection on the front and 96.1% on the back. The tested PyTesseract optimized for Nepali characters outperformed the standard OCR regarding flexibility and accuracy, extracting text from images with clean and noisy backgrounds and various contrasts. Using preprocessing steps such as converting the images into grayscale, removing noise from the images, and detecting edges further improved the system's OCR accuracy, even for low-quality photos. This work expands the current body of research in multilingual OCR and document analysis, especially for low-resource languages such as Nepali. It emphasizes the effectiveness of combining the latest object detection framework with OCR models that have been fine-tuned for practical applications.

arXiv:1904.04882

Contextual Attention for Hand Detection in the Wild

We present Hand-CNN, a novel convolutional network architecture for detecting hand masks and predicting hand orientations in unconstrained images. Hand-CNN extends MaskRCNN with a novel attention mechanism to incorporate contextual cues in the detection process. This attention mechanism can be implemented as an efficient network module that captures non-local dependencies between features. This network module can be inserted at different stages of an object detection network, and the entire detector can be trained end-to-end. We also introduce a large-scale annotated hand dataset containing hands in unconstrained images for training and evaluation. We show that Hand-CNN outperforms existing methods on several datasets, including our hand detection benchmark and the publicly available PASCAL VOC human layout challenge. We also conduct ablation studies on hand detection to show the effectiveness of the proposed contextual attention module.

arXiv:2112.08544공식 코드 · Papers with Code

NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge

Claim detection and verification are crucial for news understanding and have emerged as promising technologies for mitigating misinformation and disinformation in the news. However, most existing work has focused on claim sentence analysis while overlooking additional crucial attributes (e.g., the claimer and the main object associated with the claim). In this work, we present NewsClaims, a new benchmark for attribute-aware claim detection in the news domain. We extend the claim detection problem to include extraction of additional attributes related to each claim and release 889 claims annotated over 143 news articles. NewsClaims aims to benchmark claim detection systems in emerging scenarios, comprising unseen topics with little or no training data. To this end, we see that zero-shot and prompt-based baselines show promising performance on this benchmark, while still considerably behind human performance.

arXiv:2203.14911공식 코드 · Papers with Code

Expanding Low-Density Latent Regions for Open-Set Object Detection

Modern object detectors have achieved impressive progress under the close-set setup. However, open-set object detection (OSOD) remains challenging since objects of unknown categories are often misclassified to existing known classes. In this work, we propose to identify unknown objects by separating high/low-density regions in the latent space, based on the consensus that unknown objects are usually distributed in low-density latent regions. As traditional threshold-based methods only maintain limited low-density regions, which cannot cover all unknown objects, we present a novel Open-set Detector (OpenDet) with expanded low-density regions. To this aim, we equip OpenDet with two learners, Contrastive Feature Learner (CFL) and Unknown Probability Learner (UPL). CFL performs instance-level contrastive learning to encourage compact features of known classes, leaving more low-density regions for unknown classes; UPL optimizes unknown probability based on the uncertainty of predictions, which further divides more low-density regions around the cluster of known classes. Thus, unknown objects in low-density regions can be easily identified with the learned unknown probability. Extensive experiments demonstrate that our method can significantly improve the OSOD performance, e.g., OpenDet reduces the Absolute Open-Set Errors by 25%-35% on six OSOD benchmarks. Code is available at: https://github.com/csuhan/opendet2.

arXiv:2611.06282v1cs.CV

MonoLaDN: 단안 3D 객체 검출을 위한 언어 기반 노이즈 제거

MonoLaDN: Language-Guided DeNoising for Monocular 3D Object Detection

단안 3D 객체 검출은 3D 인식을 위한 저비용 접근법을 제공하지만, 깊이를 단일 시점에서 추론해야 하므로 본질적으로 부적절 설정(ill-posed) 문제로 남아 있다. 최근의 트랜스포머 기반 검출기는 이미지 특징과 3D 박스 주석만으로 기하 정보를 학습하므로, 비전-언어 모델(VLM)이 대규모 사전 학습을 통해 획득하는 고수준 의미 사전 지식이 부족하다. VLM은 이러한 사전 지식을 제공하지만, 추론 비용이 크기 때문에 이를 검출기로 직접 배포하는 것은 비현실적이다. 본 논문은 VLM의 기하 추론을 단안 3D 검출기로 전이하면서도 배포 모델은 순수하게 비전 기반으로 유지하는 언어 기반 노이즈 제거 프레임워크인 MonoLaDN을 제안한다. 학습 중에는 고정된 VLM이 이미지와 정답 박스로부터 각 객체의 가림, 깊이, 크기, 각도에 대한 구조화된 추론을 생성한다. 이 추론을 쿼리로 인코딩하고 노이즈로 교란한 뒤, 검출기가 깨끗한 추론을 복원하도록 학습시킨다. 손상된 추론을 복원하는 과정은 검출기의 깊이 유도 특징이 VLM의 기하 사전 지식을 내재화하도록 만들며, 따라서 추론 시 해당 추론 경로를 제거한 이후에도 이 지식이 유지되어 배포 모델에 지연 시간이나 매개변수를 추가하지 않는다. KITTI 3D와 nuScenes에서의 실험 결과, MonoLaDN은 최신 검출기들을 일관되게 능가하며 새로운 도메인에도 잘 일반화됨을 보인다.

arXiv:2611.11607v1cs.CV

단안 3D 객체 검출에서 기하학적 깊이의 재고찰: 투영 일관성 재정식화

Rethinking Geometric Depth in Monocular 3D Object Detection: A Projection-Consistent Reformulation

단안 3D 검출(Mono3D)은 단일 이미지로부터 깊이를 복원하는 데 달려 있으며, 기하학적 정식화 $z \approx f \cdot H / h_{2D}$는 단순성과 투영 기하에 대한 직접적 근거 때문에 널리 채택된다. 본 논문은 이 정식화에 투영으로 인한 체계적 편향이 내재함을 보인다. 2D 경계 상자 높이 $h_{2D}$를 대입하면 객체의 중심이 아니라 보이는 전면 표면의 깊이가 복원되어, 정답 입력 하에서도 지속되는 크기 의존적 과소추정이 발생한다. 우리는 투영 일관성 재정식화에 기반한 통합 프레임워크인 MonoPCA로 이 전면 표면 편향을 해결한다. 핵심인 PCDepth는 측정 가능한 $h_{2D}$를 유지하면서, 원근 기하에서 유도한 방향 및 크기 인식 보정을 통해 중심 깊이와의 간극을 메워, 부적절 문제(ill-posed)인 회귀 목표를 도입하지 않고 편향을 제거한다. PCDepth는 깊이를 예측된 객체 크기 및 방향과 결합하므로 속성 인지가 필수적이 된다. 따라서 비전 파운데이션 모델의 기하 표현을 경량 학생 모델로 전이하여 추론 비용 없이 속성 인지를 강화하는 교사-학생 프레임워크인 GeoAlign을 도입한다. 네 가지 도전적인 벤치마크에서 MonoPCA는 KITTI 리더보드에서 AP$_{3D}$ +1.44%, Omni3D$_{OUT}$에서 +2.80%의 향상을 포함하여 SoTA 성능을 달성한다. 코드는 https://github.com/alanzhangcs/MonoPCA 에서 이용할 수 있다.

arXiv:1909.05770

Recognizing Object Affordances to Support Scene Reasoning for Manipulation Tasks

Affordance information about a scene provides important clues as to what actions may be executed in pursuit of meeting a specified goal state. Thus, integrating affordance-based reasoning into symbolic action plannning pipelines would enhance the flexibility of robot manipulation. Unfortunately, the top performing affordance recognition methods use object category priors to boost the accuracy of affordance detection and segmentation. Object priors limit generalization to unknown object categories. This paper describes an affordance recognition pipeline based on a category-agnostic region proposal network for proposing instance regions of an image across categories. To guide affordance learning in the absence of category priors, the training process includes the auxiliary task of explicitly inferencing existing affordances within a proposal. Secondly, a self-attention mechanism trained to interpret each proposal learns to capture rich contextual dependencies through the region. Visual benchmarking shows that the trained network, called AffContext, reduces the performance gap between object-agnostic and object-informed affordance recognition. AffContext is linked to the Planning Domain Definition Language (PDDL) with an augmented state keeper for action planning across temporally spaced goal-oriented tasks. Manipulation experiments show that AffContext can successfully parse scene content to seed a symbolic planner problem specification, whose execution completes the target task. Additionally, task-oriented grasping for cutting and pounding actions demonstrate the exploitation of multiple affordances for a given object to complete specified tasks.

arXiv:2203.12849공식 코드 · Papers with Code

Complex Scene Image Editing by Scene Graph Comprehension

Conditional diffusion models have demonstrated impressive performance on various tasks like text-guided semantic image editing. Prior work requires image regions to be identified manually by human users or use an object detector that only perform well for object-centric manipulations. For example, if an input image contains multiple objects with the same semantic meaning (such as a group of birds), object detectors may struggle to recognize and localize the target object, let alone accurately manipulate it. To address these challenges, we propose a two-stage method for achieving complex scene image editing by Scene Graph Comprehension (SGC-Net). In the first stage, we train a Region of Interest (RoI) prediction network that uses scene graphs and predict the locations of the target objects. Unlike object detection methods based solely on object category, our method can accurately recognize the target object by comprehending the objects and their semantic relationships within a complex scene. The second stage uses a conditional diffusion model to edit the image based on our RoI predictions. We evaluate the effectiveness of our approach on the CLEVR and Visual Genome datasets. We report an 8 point improvement in SSIM on CLEVR and our edited images were preferred by human users by 9-33% over prior work on Visual Genome, validating the effectiveness of our proposed method. Code is available at github.com/Zhongping-Zhang/SGC_Net.

arXiv:1701.04210

Bandwidth limited object recognition in high resolution imagery

This paper proposes a novel method to optimize bandwidth usage for object detection in critical communication scenarios. We develop two operating models of active information seeking. The first model identifies promising regions in low resolution imagery and progressively requests higher resolution regions on which to perform recognition of higher semantic quality. The second model identifies promising regions in low resolution imagery while simultaneously predicting the approximate location of the object of higher semantic quality. From this general framework, we develop a car recognition system via identification of its license plate and evaluate the performance of both models on a car dataset that we introduce. Results are compared with traditional JPEG compression and demonstrate that our system saves up to one order of magnitude of bandwidth while sacrificing little in terms of recognition performance.

arXiv:2207.07953

Level Set-Based Camera Pose Estimation From Multiple 2D/3D Ellipse-Ellipsoid Correspondences

In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling, demonstrate its nice properties for handling partially visible objects and compare its performance with other common metrics. Finally, we show that the use of a predictive uncertainty on the detected ellipses allows a fair weighting of the contribution of the correspondences which improves the computed pose. The code is released at https://gitlab.inria.fr/tangram/level-set-based-camera-pose-estimation.