PyTorchKR Papers

“Object Detection”

총 13,455편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:1507.07882

Occlusion-Aware Object Localization, Segmentation and Pose Estimation

We present a learning approach for localization and segmentation of objects in an image in a manner that is robust to partial occlusion. Our algorithm produces a bounding box around the full extent of the object and labels pixels in the interior that belong to the object. Like existing segmentation aware detection approaches, we learn an appearance model of the object and consider regions that do not fit this model as potential occlusions. However, in addition to the established use of pairwise potentials for encouraging local consistency, we use higher order potentials which capture information at the level of im- age segments. We also propose an efficient loss function that targets both localization and segmentation performance. Our algorithm achieves 13.52% segmentation error and 0.81 area under the false-positive per image vs. recall curve on average over the challenging CMU Kitchen Occlusion Dataset. This is a 42.44% decrease in segmentation error and a 16.13% increase in localization performance compared to the state-of-the-art. Finally, we show that the visibility labelling produced by our algorithm can make full 3D pose estimation from a single image robust to occlusion.

arXiv:2203.16513공식 코드 · Papers with Code

PromptDet: Towards Open-vocabulary Detection using Uncurated Images

The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations. To achieve that, we make the following four contributions: (i) in pursuit of generalisation, we propose a two-stage open-vocabulary object detector, where the class-agnostic object proposals are classified with a text encoder from pre-trained visual-language model; (ii) To pair the visual latent space (of RPN box proposals) with that of the pre-trained text encoder, we propose the idea of regional prompt learning to align the textual embedding space with regional visual object features; (iii) To scale up the learning procedure towards detecting a wider spectrum of objects, we exploit the available online resource via a novel self-training framework, which allows to train the proposed detector on a large corpus of noisy uncurated web images. Lastly, (iv) to evaluate our proposed detector, termed as PromptDet, we conduct extensive experiments on the challenging LVIS and MS-COCO dataset. PromptDet shows superior performance over existing approaches with fewer additional training images and zero manual annotations whatsoever. Project page with code: https://fcjian.github.io/promptdet.

arXiv:2209.13948공식 코드 · Papers with Code

Obj2Seq: Formatting Objects as Sequences with Class Prompt for Visual Tasks

Visual tasks vary a lot in their output formats and concerned contents, therefore it is hard to process them with an identical structure. One main obstacle lies in the high-dimensional outputs in object-level visual tasks. In this paper, we propose an object-centric vision framework, Obj2Seq. Obj2Seq takes objects as basic units, and regards most object-level visual tasks as sequence generation problems of objects. Therefore, these visual tasks can be decoupled into two steps. First recognize objects of given categories, and then generate a sequence for each of these objects. The definition of the output sequences varies for different tasks, and the model is supervised by matching these sequences with ground-truth targets. Obj2Seq is able to flexibly determine input categories to satisfy customized requirements, and be easily extended to different visual tasks. When experimenting on MS COCO, Obj2Seq achieves 45.7% AP on object detection, 89.0% AP on multi-label classification and 65.0% AP on human pose estimation. These results demonstrate its potential to be generally applied to different visual tasks. Code has been made available at: https://github.com/CASIA-IVA-Lab/Obj2Seq.

arXiv:2406.15565공식 코드 · Papers with Code

Unseen Object Reasoning with Shared Appearance Cues

This paper introduces an innovative approach to open world recognition (OWR), where we leverage knowledge acquired from known objects to address the recognition of previously unseen objects. The traditional method of object modeling relies on supervised learning with strict closed-set assumptions, presupposing that objects encountered during inference are already known at the training phase. However, this assumption proves inadequate for real-world scenarios due to the impracticality of accounting for the immense diversity of objects. Our hypothesis posits that object appearances can be represented as collections of "shareable" mid-level features, arranged in constellations to form object instances. By adopting this framework, we can efficiently dissect and represent both known and unknown objects in terms of their appearance cues. Our paper introduces a straightforward yet elegant method for modeling novel or unseen objects, utilizing established appearance cues and accounting for inherent uncertainties. This representation not only enables the detection of out-of-distribution objects or novel categories among unseen objects but also facilitates a deeper level of reasoning, empowering the identification of the superclass to which an unknown instance belongs. This novel approach holds promise for advancing open world recognition in diverse applications.

arXiv:2308.07918공식 코드 · Papers with Code

Helping Hands: An Object-Aware Ego-Centric Video Recognition Model

We introduce an object-aware decoder for improving the performance of spatio-temporal representations on ego-centric videos. The key idea is to enhance object-awareness during training by tasking the model to predict hand positions, object positions, and the semantic label of the objects using paired captions when available. At inference time the model only requires RGB frames as inputs, and is able to track and ground objects (although it has not been trained explicitly for this). We demonstrate the performance of the object-aware representations learnt by our model, by: (i) evaluating it for strong transfer, i.e. through zero-shot testing, on a number of downstream video-text retrieval and classification benchmarks; and (ii) by using the representations learned as input for long-term video understanding tasks (e.g. Episodic Memory in Ego4D). In all cases the performance improves over the state of the art -- even compared to networks trained with far larger batch sizes. We also show that by using noisy image-level detection as pseudo-labels in training, the model learns to provide better bounding boxes using video consistency, as well as grounding the words in the associated text descriptions. Overall, we show that the model can act as a drop-in replacement for an ego-centric video model to improve performance through visual-text grounding.

arXiv:2511.20274v2cs.CV

SCLARO: 그라운딩된 시나리오 수준 장면 이해를 위한 데이터셋 및 벤치마킹을 위한 ScenarioCLIP

SCLARO: A Dataset for Grounded Scenario-Level Scene Understanding and ScenarioCLIP for Benchmarking

장면 내 행동, 객체, 객체 간 관계를 세 수준 모두에서 그라운딩된 어노테이션으로 동시에 제공하는 대규모 데이터셋이 부재한 문제를 해결하기 위해, 실내·실외·주행 시나리오에 걸친 615,805장 이미지에 전역 행동 캡션, 바운딩 박스, 관계 트리플렛을 포함한 SCLARO 데이터셋을 구축했다. 이를 벤치마킹하기 위해 분리된 인코더와 EMA 기반 지식 증류(distillation)로 세 수준을 동시에 인코딩하는 ScenarioCLIP을 제안하며, 제로샷 검색·객체 검출·씬 그래프 분류 등 다양한 태스크에서 PyramidCLIP 대비 특히 관계 수준과 도메인 외 일반화 성능이 향상됨을 입증했다.

arXiv:2607.25239v1cs.CV

CD-RMOT-Bench: 교차 도메인 지칭 다중 객체 추적 벤치마킹

CD-RMOT-Bench: Benchmarking the Cross-Domain Referring Multi-Object Tracking

지칭 다중 객체 추적(RMOT)은 자연어 표현에 객체 궤적을 그라운딩(grounding)함으로써 추적을 범주 중심 인지에서 언어 유도형 이해로 확장합니다. 최근의 발전에도 불구하고 기존 RMOT 연구는 대부분 도메인 내(in-domain) 환경에서 수행되어, 피할 수 없는 시각적 도메인 변화(domain shift) 상황에서 언어 조건부 추적의 강건성을 규명하지 못했습니다. 본 논문에서는 라벨이 지정된 소스 도메인에서 학습된 RMOT 모델이 시각적 조건이 다른 라벨 없는 타깃 도메인에서 자연어 표현을 안정적으로 따를 수 있는지를 평가하는 새롭고 도전적인 문제인 교차 도메인 지칭 다중 객체 추적(CD-RMOT)을 연구합니다. 체계적인 연구를 지원하기 위해, 실제 선명한 도메인의 지칭 추적 데이터, 정렬된 디지털 트윈(digital-twin) 변형, 실제 악조건 도메인 영상을 결합한 통합 벤치마크인 CD-RMOT-Bench를 구축합니다. CD-RMOT-Bench는 공유된 RMOT 프로토콜 하에서 통제된 날씨/시점 변화 분석과 현실적인 합성-실제(synthetic-real) 전이 평가를 모두 가능하게 합니다. 나아가 시각적 궤적과 지칭 표현을 연결하는 쿼리 공간을 안정화하도록 설계된 쿼리 중심 적응(QCA) 프레임워크를 제공합니다. 광범위한 실험을 통해 도메인 변화가 RMOT 성능을 심각하게 저하시키며, 이러한 실패가 단순한 객체 검출(object detection) 오류뿐만 아니라 더 결정적으로는 표현 조건부 시간적 연관 및 타깃 선택의 불안정성으로 인해 발생함을 밝혀냈습니다. QCA는 강력한 기준선(baseline)을 확립하며, CD-RMOT-Bench는 시각 도메인 전반에 걸친 강건한 언어 유도 추적을 위한 새로운 방향을 제시합니다.

arXiv:2305.20055

Cross-Domain Car Detection Model with Integrated Convolutional Block Attention Mechanism

Car detection, particularly through camera vision, has become a major focus in the field of computer vision and has gained widespread adoption. While current car detection systems are capable of good detection, reliable detection can still be challenging due to factors such as proximity between the car, light intensity, and environmental visibility. To address these issues, we propose cross-domain Car Detection Model with integrated convolutional block Attention mechanism(CDMA) that we apply to car recognition for autonomous driving and other areas. CDMA includes several novelties: 1)Building a complete cross-domain target detection framework. 2)Developing an unpaired target domain picture generation module with an integrated convolutional attention mechanism which specifically emphasizes the car headlights feature. 3)Adopting Generalized Intersection over Union (GIOU) as the loss function of the target detection framework. 4)Designing an object detection model integrated with two-headed Convolutional Block Attention Module(CBAM). 5)Utilizing an effective data enhancement method. To evaluate the model's effectiveness, we performed a reduced will resolution process on the data in the SSLAD dataset and used it as the benchmark dataset for our task. Experimental results show that the performance of the cross-domain car target detection model improves by 40% over the model without our framework, and our improvements have a significant impact on cross-domain car recognition.

arXiv:1806.11223

Active query-driven visual search using probabilistic bisection and convolutional neural networks

We present a novel efficient object detection and localization framework based on the probabilistic bisection algorithm. A Convolutional Neural Network (CNN) is trained and used as a noisy oracle that provides answers to input query images. The responses along with error probability estimates obtained from the CNN are used to update beliefs on the object location along each dimension. We show that querying along each dimension achieves the same lower bound on localization error as the joint query design. Finally, we compare our approach to the traditional sliding window technique on a real world face localization task and show speed improvements by at least an order of magnitude while maintaining accurate localization.

arXiv:2211.09529공식 코드 · Papers with Code

InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges

In this report, we present our champion solutions to five tracks at Ego4D challenge. We leverage our developed InternVideo, a video foundation model, for five Ego4D tasks, including Moment Queries, Natural Language Queries, Future Hand Prediction, State Change Object Detection, and Short-term Object Interaction Anticipation. InternVideo-Ego4D is an effective paradigm to adapt the strong foundation model to the downstream ego-centric video understanding tasks with simple head designs. In these five tasks, the performance of InternVideo-Ego4D comprehensively surpasses the baseline methods and the champions of CVPR2022, demonstrating the powerful representation ability of InternVideo as a video foundation model. Our code will be released at https://github.com/OpenGVLab/ego4d-eccv2022-solutions

arXiv:2407.01894공식 코드 · Papers with Code

Adaptive Modality Balanced Online Knowledge Distillation for Brain-Eye-Computer based Dim Object Detection

Advanced cognition can be extracted from the human brain using brain-computer interfaces. Integrating these interfaces with computer vision techniques, which possess efficient feature extraction capabilities, can achieve more robust and accurate detection of dim targets in aerial images. However, existing target detection methods primarily concentrate on homogeneous data, lacking efficient and versatile processing capabilities for heterogeneous multimodal data. In this paper, we first build a brain-eye-computer based object detection system for aerial images under few-shot conditions. This system detects suspicious targets using region proposal networks, evokes the event-related potential (ERP) signal in electroencephalogram (EEG) through the eye-tracking-based slow serial visual presentation (ESSVP) paradigm, and constructs the EEG-image data pairs with eye movement data. Then, an adaptive modality balanced online knowledge distillation (AMBOKD) method is proposed to recognize dim objects with the EEG-image data. AMBOKD fuses EEG and image features using a multi-head attention module, establishing a new modality with comprehensive features. To enhance the performance and robust capability of the fusion modality, simultaneous training and mutual learning between modalities are enabled by end-to-end online knowledge distillation. During the learning process, an adaptive modality balancing module is proposed to ensure multimodal equilibrium by dynamically adjusting the weights of the importance and the training gradients across various modalities. The effectiveness and superiority of our method are demonstrated by comparing it with existing state-of-the-art methods. Additionally, experiments conducted on public datasets and system validations in real-world scenarios demonstrate the reliability and practicality of the proposed system and the designed method.

arXiv:1303.5613

Network Detection Theory and Performance

Network detection is an important capability in many areas of applied research in which data can be represented as a graph of entities and relationships. Oftentimes the object of interest is a relatively small subgraph in an enormous, potentially uninteresting background. This aspect characterizes network detection as a "big data" problem. Graph partitioning and network discovery have been major research areas over the last ten years, driven by interest in internet search, cyber security, social networks, and criminal or terrorist activities. The specific problem of network discovery is addressed as a special case of graph partitioning in which membership in a small subgraph of interest must be determined. Algebraic graph theory is used as the basis to analyze and compare different network detection methods. A new Bayesian network detection framework is introduced that partitions the graph based on prior information and direct observations. The new approach, called space-time threat propagation, is proved to maximize the probability of detection and is therefore optimum in the Neyman-Pearson sense. This optimality criterion is compared to spectral community detection approaches which divide the global graph into subsets or communities with optimal connectivity properties. We also explore a new generative stochastic model for covert networks and analyze using receiver operating characteristics the detection performance of both classes of optimal detection techniques.

arXiv:2607.03004v1cs.CV

REAL-OW: 저랭크 적응과 이중 단계 객체성 모델링을 활용한 리허설 없는 개방 세계 객체 검출

REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling

개방 세계 객체 검출(OWOD)은 검출기가 이전에 본 적 없는 객체를 미지(unknown)로 식별하고, 기존에 획득한 지식을 보존하면서 이를 점진적으로 알려진 카테고리 집합에 통합할 것을 요구한다. 기존 프레임워크는 파국적 망각(catastrophic forgetting)을 완화하기 위해 예시 리플레이(exemplar replay)에 크게 의존하지만, 일부 실제 응용 환경에서는 원시 데이터 저장이 데이터 접근 제한과 충돌하고 데이터 노출 위험을 초래하며 상당한 메모리 오버헤드를 야기한다. 본 논문에서는 저랭크 적응(LoRA, Low-Rank Adaptation) 기반의 협력적 어댑터 아키텍처를 통해 점진적 지식을 분리하는 새로운 리허설 없는 프레임워크인 REAL-OW를 제안한다. 구체적으로, 백본에 일반 어댑터(GA, General Adapter)를 배치하여 교차 태스크 범용 표현의 중요도 인식 정제를 가능하게 하고, 디코더의 특정 어댑터(SA, Specific Adapter)는 태스크별 전문 지식의 직교 저장을 제공한다. 리허설 없는 제약 조건 하에서 객체성 모델링의 표현 드리프트를 해결하기 위해, 특징 집계와 경계 통합을 교대로 수행하여 객체성 분포를 안정화하고 알려진 카테고리와 미지 카테고리 간의 분리를 유지하는 이중 단계 객체성 모델링(DSOM, Dual-Stage Objectness Modeling)을 도입한다. 나아가 DSOM은 리허설 없는 환경에 내재된 분산된 특징 분포에 맞게 설계된 보정된 가우시안 음 로그 가능도(CG-NLL, Calibrated Gaussian Negative Log-Likelihood) 거리를 통해 지원된다. 광범위한 평가를 통해 REAL-OW가 최첨단 성능을 달성하며, 검출 정밀도와 미지 객체 발견 모두에서 기존 예시 리플레이 방법을 능가함을 입증한다. 우리의 접근 방식은 리허설 없는 OWOD에 대한 새로운 기준선을 확립한다.

arXiv:1510.03608공식 코드 · Papers with Code

Deep convolutional neural networks for pedestrian detection

Pedestrian detection is a popular research topic due to its paramount importance for a number of applications, especially in the fields of automotive, surveillance and robotics. Despite the significant improvements, pedestrian detection is still an open challenge that calls for more and more accurate algorithms. In the last few years, deep learning and in particular convolutional neural networks emerged as the state of the art in terms of accuracy for a number of computer vision tasks such as image classification, object detection and segmentation, often outperforming the previous gold standards by a large margin. In this paper, we propose a pedestrian detection system based on deep learning, adapting a general-purpose convolutional network to the task at hand. By thoroughly analyzing and optimizing each step of the detection pipeline we propose an architecture that outperforms traditional methods, achieving a task accuracy close to that of state-of-the-art approaches, while requiring a low computational time. Finally, we tested the system on an NVIDIA Jetson TK1, a 192-core platform that is envisioned to be a forerunner computational brain of future self-driving cars.

arXiv:2206.12941

Object Detection and Tracking with Autonomous UAV

In this paper, a combat Unmanned Air Vehicle (UAV) is modeled in the simulation environment. The rotary wing UAV is successfully performed various tasks such as locking on the targets, tracking, and sharing the relevant data with surrounding vehicles. Different software technologies such as API communication, ground control station configuration, autonomous movement algorithms, computer vision, and deep learning are employed.

arXiv:2108.01971

Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection

The popularity and promotion of depth maps have brought new vigor and vitality into salient object detection (SOD), and a mass of RGB-D SOD algorithms have been proposed, mainly concentrating on how to better integrate cross-modality features from RGB image and depth map. For the cross-modality interaction in feature encoder, existing methods either indiscriminately treat RGB and depth modalities, or only habitually utilize depth cues as auxiliary information of the RGB branch. Different from them, we reconsider the status of two modalities and propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD, which differentially models the dependence of two modalities according to the feature representations of different layers. To this end, two components are designed to implement the effective cross-modality interaction: 1) the RGB-induced Detail Enhancement (RDE) module leverages RGB modality to enhance the details of the depth features in low-level encoder stage. 2) the Depth-induced Semantic Enhancement (DSE) module transfers the object positioning and internal consistency of depth features to the RGB branch in high-level encoder stage. Furthermore, we also design a Dense Decoding Reconstruction (DDR) structure, which constructs a semantic block by combining multi-level encoder features to upgrade the skip connection in the feature decoding. Extensive experiments on five benchmark datasets demonstrate that our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively. Our code is publicly available at: https://rmcong.github.io/proj_CDINet.html.

arXiv:1803.08414

Buried object detection from B-scan ground penetrating radar data using Faster-RCNN

In this paper, we adapt the Faster-RCNN framework for the detection of underground buried objects (i.e. hyperbola reflections) in B-scan ground penetrating radar (GPR) images. Due to the lack of real data for training, we propose to incorporate more simulated radargrams generated from different configurations using the gprMax toolbox. Our designed CNN is first pre-trained on the grayscale Cifar-10 database. Then, the Faster-RCNN framework based on the pre-trained CNN is trained and fine-tuned on both real and simulated GPR data. Preliminary detection results show that the proposed technique can provide significant improvements compared to classical computer vision methods and hence becomes quite promising to deal with this kind of specific GPR data even with few training samples.

arXiv:1310.8574

Spatial statistics, image analysis and percolation theory

We develop a novel method for detection of signals and reconstruction of images in the presence of random noise. The method uses results from percolation theory. We specifically address the problem of detection of multiple objects of unknown shapes in the case of nonparametric noise. The noise density is unknown and can be heavy-tailed. The objects of interest have unknown varying intensities. No boundary shape constraints are imposed on the objects, only a set of weak bulk conditions is required. We view the object detection problem as a multiple hypothesis testing for discrete statistical inverse problems. We present an algorithm that allows to detect greyscale objects of various shapes in noisy images. We prove results on consistency and algorithmic complexity of our procedures. Applications to cryo-electron microscopy are presented.

arXiv:2204.12841

BBBD: Bounding Box Based Detector for Occlusion Detection and Order Recovery

Occlusion handling is one of the challenges of object detection and segmentation, and scene understanding. Because objects appear differently when they are occluded in varying degree, angle, and locations. Therefore, determining the existence of occlusion between objects and their order in a scene is a fundamental requirement for semantic understanding. Existing works mostly use deep learning based models to retrieve the order of the instances in an image or for occlusion detection. This requires labelled occluded data and it is time consuming. In this paper, we propose a simpler and faster method that can perform both operations without any training and only requires the modal segmentation masks. For occlusion detection, instead of scanning the two objects entirely, we only focus on the intersected area between their bounding boxes. Similarly, we use the segmentation mask inside the same area to recover the depth-ordering. When tested on COCOA dataset, our method achieves +8% and +5% more accuracy than the baselines in order recovery and occlusion detection respectively.

arXiv:1411.5309

End-to-End Integration of a Convolutional Network, Deformable Parts Model and Non-Maximum Suppression

Deformable Parts Models and Convolutional Networks each have achieved notable performance in object detection. Yet these two approaches find their strengths in complementary areas: DPMs are well-versed in object composition, modeling fine-grained spatial relationships between parts; likewise, ConvNets are adept at producing powerful image features, having been discriminatively trained directly on the pixels. In this paper, we propose a new model that combines these two approaches, obtaining the advantages of each. We train this model using a new structured loss function that considers all bounding boxes within an image, rather than isolated object instances. This enables the non-maximal suppression (NMS) operation, previously treated as a separate post-processing stage, to be integrated into the model. This allows for discriminative training of our combined Convnet + DPM + NMS model in end-to-end fashion. We evaluate our system on PASCAL VOC 2007 and 2011 datasets, achieving competitive results on both benchmarks.