PyTorchKR Papers

“Object Detection”

총 13,455편
태그·기간 등 상세 필터

카테고리 탐색

arXiv:2104.14360공식 코드 · Papers with Code

Video Salient Object Detection via Adaptive Local-Global Refinement

Video salient object detection (VSOD) is an important task in many vision applications. Reliable VSOD requires to simultaneously exploit the information from both the spatial domain and the temporal domain. Most of the existing algorithms merely utilize simple fusion strategies, such as addition and concatenation, to merge the information from different domains. Despite their simplicity, such fusion strategies may introduce feature redundancy, and also fail to fully exploit the relationship between multi-level features extracted from both spatial and temporal domains. In this paper, we suggest an adaptive local-global refinement framework for VSOD. Different from previous approaches, we propose a local refinement architecture and a global one to refine the simply fused features with different scopes, which can fully explore the local dependence and the global dependence of multi-level features. In addition, to emphasize the effective information and suppress the useless one, an adaptive weighting mechanism is designed based on graph convolutional neural network (GCN). We show that our weighting methodology can further exploit the feature correlations, thus driving the network to learn more discriminative feature representation. Extensive experimental results on public video datasets demonstrate the superiority of our method over the existing ones.

arXiv:2106.15796

Monocular 3D Object Detection: An Extrinsic Parameter Free Approach

Monocular 3D object detection is an important task in autonomous driving. It can be easily intractable where there exists ego-car pose change w.r.t. ground plane. This is common due to the slight fluctuation of road smoothness and slope. Due to the lack of insight in industrial application, existing methods on open datasets neglect the camera pose information, which inevitably results in the detector being susceptible to camera extrinsic parameters. The perturbation of objects is very popular in most autonomous driving cases for industrial products. To this end, we propose a novel method to capture camera pose to formulate the detector free from extrinsic perturbation. Specifically, the proposed framework predicts camera extrinsic parameters by detecting vanishing point and horizon change. A converter is designed to rectify perturbative features in the latent space. By doing so, our 3D detector works independent of the extrinsic parameter variations and produces accurate results in realistic cases, e.g., potholed and uneven roads, where almost all existing monocular detectors fail to handle. Experiments demonstrate our method yields the best performance compared with the other state-of-the-arts by a large margin on both KITTI 3D and nuScenes datasets.

arXiv:2209.09486

Self-supervised 3D Object Detection from Monocular Pseudo-LiDAR

There have been attempts to detect 3D objects by fusion of stereo camera images and LiDAR sensor data or using LiDAR for pre-training and only monocular images for testing, but there have been less attempts to use only monocular image sequences due to low accuracy. In addition, when depth prediction using only monocular images, only scale-inconsistent depth can be predicted, which is the reason why researchers are reluctant to use monocular images alone. Therefore, we propose a method for predicting absolute depth and detecting 3D objects using only monocular image sequences by enabling end-to-end learning of detection networks and depth prediction networks. As a result, the proposed method surpasses other existing methods in performance on the KITTI 3D dataset. Even when monocular image and 3D LiDAR are used together during training in an attempt to improve performance, ours exhibit is the best performance compared to other methods using the same input. In addition, end-to-end learning not only improves depth prediction performance, but also enables absolute depth prediction, because our network utilizes the fact that the size of a 3D object such as a car is determined by the approximate size.

arXiv:2304.00218

Mask Hierarchical Features For Self-Supervised Learning

This paper shows that Masking the Deep hierarchical features is an efficient self-supervised method, denoted as MaskDeep. MaskDeep treats each patch in the representation space as an independent instance. We mask part of patches in the representation space and then utilize sparse visible patches to reconstruct high semantic image representation. The intuition of MaskDeep lies in the fact that models can reason from sparse visible patches semantic to the global semantic of the image. We further propose three designs in our framework: 1) a Hierarchical Deep-Masking module to concern the hierarchical property of patch representations, 2) a multi-group strategy to improve the efficiency without any extra computing consumption of the encoder and 3) a multi-target strategy to provide more description of the global semantic. Our MaskDeep brings decent improvements. Trained on ResNet50 with 200 epochs, MaskDeep achieves state-of-the-art results of 71.2% Top1 accuracy linear classification on ImageNet. On COCO object detection tasks, MaskDeep outperforms the self-supervised method SoCo, which specifically designed for object detection. When trained with 100 epochs, MaskDeep achieves 69.6% Top1 accuracy, which surpasses current methods trained with 200 epochs, such as HCSC, by 0.4% .

arXiv:2305.07180

Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition

Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches, which may not sufficiently facilitate meaningful object-specific semantic understanding, leading to a reliance on apparent background correlations. Moreover, they primarily rely on high-dimensional local descriptors to construct complex embedding space, potentially limiting the generalization. To address the above challenges, this article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition. RSaD introduces additional saliency-aware supervision via saliency detection to guide the model toward focusing on the intrinsic discriminative regions. Specifically, RSaD utilizes the saliency detection model to emphasize the critical regions of each sub-category, providing additional object-specific information for fine-grained prediction. RSaD transfers such information with two symmetric branches in a mutual learning paradigm. Furthermore, RSaD exploits inter-regional relationships to enhance the informativeness of the representation and subsequently summarize the highlighted details into contextual embeddings to facilitate the effective transfer, enabling quick generalization to novel sub-categories. The proposed approach is empirically evaluated on three widely used benchmarks, demonstrating its superior performance.

arXiv:2406.17654공식 코드 · Papers with Code

MDHA: Multi-Scale Deformable Transformer with Hybrid Anchors for Multi-View 3D Object Detection

Multi-view 3D object detection is a crucial component of autonomous driving systems. Contemporary query-based methods primarily depend either on dataset-specific initialization of 3D anchors, introducing bias, or utilize dense attention mechanisms, which are computationally inefficient and unscalable. To overcome these issues, we present MDHA, a novel sparse query-based framework, which constructs adaptive 3D output proposals using hybrid anchors from multi-view, multi-scale image input. Fixed 2D anchors are combined with depth predictions to form 2.5D anchors, which are projected to obtain 3D proposals. To ensure high efficiency, our proposed Anchor Encoder performs sparse refinement and selects the top-$k$ anchors and features. Moreover, while existing multi-view attention mechanisms rely on projecting reference points to multiple images, our novel Circular Deformable Attention mechanism only projects to a single image but allows reference points to seamlessly attend to adjacent images, improving efficiency without compromising on performance. On the nuScenes val set, it achieves 46.4\% mAP and 55.0\% NDS with a ResNet101 backbone. MDHA significantly outperforms the baseline where anchor proposals are modelled as learnable embeddings. Code is available at https://github.com/NaomiEX/MDHA.

arXiv:2607.17340v1cs.CV

도메인 점진적 객체 검출을 위한 직교 지식 갱신

Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection

도메인 점진적 객체 검출(DIOD)은 모델이 기존 지식을 보존하면서 새로운 도메인에 지속적으로 적응해야 한다. 최근 매개변수 효율적 미세조정은 유망한 접근법을 제공하며, 사전학습된 모델을 고정하고 소수의 학습 가능한 매개변수를 하류 작업에 주입한다. 그러나 이러한 방법은 중요한 과거 지식을 덮어쓸 위험이 있어 도메인 간 간섭과 성능 저하를 유발한다. 이 문제를 해결하기 위해 우리는 DIOD를 위한 단순하면서도 효과적인 프레임워크인 직교 지식 갱신(Orthogonal Knowledge Refreshing, OKR)을 제안한다. OKR은 각 도메인마다 전용 저랭크(low-rank) 분기를 통해 독립적인 도메인별 부분공간을 점진적으로 구성하고, 이를 원활하게 융합하여 전체적인 의사결정을 수행함으로써 추론 시 도메인 선택 없이 충돌 없는 용량 확장을 가능하게 한다. 융합 과정에서의 지식 간섭을 최소화하기 위해, 우리는 새 도메인의 그래디언트 업데이트를 융합된 역사적 부분공간의 직교 여공간(orthogonal complement)에 투영하는 그래디언트 기반 직교 갱신 전략을 제시하여 망각 없는 지속적 적응을 지원한다. 또한 도메인 간 의미적 분절을 완화하기 위해 토폴로지 인식(topology-aware) 일관성을 강제하여 기존 도메인과 새 도메인의 의미 구조를 정렬한다. 광범위한 실험을 통해 OKR의 우수성이 검증되었으며, Pascal VOC 및 BDD100K 시리즈에서 각각 최고의 exemplar-free 방법보다 mAP +5.6% 및 +6.5%의 큰 폭으로 성능을 능가한다.

arXiv:2311.01429

Efficient Vision Transformer for Accurate Traffic Sign Detection

This research paper addresses the challenges associated with traffic sign detection in self-driving vehicles and driver assistance systems. The development of reliable and highly accurate algorithms is crucial for the widespread adoption of traffic sign recognition and detection (TSRD) in diverse real-life scenarios. However, this task is complicated by suboptimal traffic images affected by factors such as camera movement, adverse weather conditions, and inadequate lighting. This study specifically focuses on traffic sign detection methods and introduces the application of the Transformer model, particularly the Vision Transformer variants, to tackle this task. The Transformer's attention mechanism, originally designed for natural language processing, offers improved parallel efficiency. Vision Transformers have demonstrated success in various domains, including autonomous driving, object detection, healthcare, and defense-related applications. To enhance the efficiency of the Transformer model, the research proposes a novel strategy that integrates a locality inductive bias and a transformer module. This includes the introduction of the Efficient Convolution Block and the Local Transformer Block, which effectively capture short-term and long-term dependency information, thereby improving both detection speed and accuracy. Experimental evaluations demonstrate the significant advancements achieved by this approach, particularly when applied to the GTSDB dataset.

arXiv:2401.13214

AMANet: Advancing SAR Ship Detection with Adaptive Multi-Hierarchical Attention Network

Recently, methods based on deep learning have been successfully applied to ship detection for synthetic aperture radar (SAR) images. Despite the development of numerous ship detection methodologies, detecting small and coastal ships remains a significant challenge due to the limited features and clutter in coastal environments. For that, a novel adaptive multi-hierarchical attention module (AMAM) is proposed to learn multi-scale features and adaptively aggregate salient features from various feature layers, even in complex environments. Specifically, we first fuse information from adjacent feature layers to enhance the detection of smaller targets, thereby achieving multi-scale feature enhancement. Then, to filter out the adverse effects of complex backgrounds, we dissect the previously fused multi-level features on the channel, individually excavate the salient regions, and adaptively amalgamate features originating from different channels. Thirdly, we present a novel adaptive multi-hierarchical attention network (AMANet) by embedding the AMAM between the backbone network and the feature pyramid network (FPN). Besides, the AMAM can be readily inserted between different frameworks to improve object detection. Lastly, extensive experiments on two large-scale SAR ship detection datasets demonstrate that our AMANet method is superior to state-of-the-art methods.

arXiv:1910.13317

Distributed and Consistent Multi-Image Feature Matching via QuickMatch

In this work we consider the multi-image object matching problem, extend a centralized solution of the problem to a distributed solution, and present an experimental application of the centralized solution. Multi-image feature matching is a keystone of many applications, including simultaneous localization and mapping, homography, object detection, and structure from motion. We first review the QuickMatch algorithm for multi-image feature matching. We then present a scheme for distributing sets of features across computational units (agents) that largely preserves feature match quality and minimizes communication between agents (avoiding, in particular, the need of flooding all data to all agents). Finally, we show how QuickMatch performs on an object matching test with low quality images. The centralized QuickMatch algorithm is compared to other standard matching algorithms, while the Distributed QuickMatch algorithm is compared to the centralized algorithm in terms of preservation of match consistency. The presented experiment shows that QuickMatch matches features across a large number of images and features in larger numbers and more accurately than standard techniques.

arXiv:2611.22852v1cs.RO

산업 환경용 이동형 매니퓰레이터 로봇 기술 보고서

Technical Report of Mobile Manipulator Robot for Industrial Environments

이 논문은 샤히드 베헤슈티 대학교 전기공학과 로봇공학·지능형 자동화 연구실에서 개발한 Auriga Robotics의 이동형 매니퓰레이터 로봇 설계와 구현을 소개한다. 이 플랫폼을 @Work Robot이라고 부른다. 산업 과제를 대상으로 하며, 반복 작업이나 위험 작업 환경에서 효율을 높이는 데 초점을 맞춘다. 메카넘 휠 네 개로 모든 방향으로 이동하며, 3D 프린팅한 그리퍼를 장착한 5자유도 매니퓰레이터 팔을 사용해 물체를 다루고 주행한다. 전자 장치는 ESP32 마이크로컨트롤러 기반의 맞춤형 PCB와 고수준 처리 및 실시간 시스템 제어를 담당하는 NVIDIA Jetson Orin NX로 구성된다. 소프트웨어에서는 Hector SLAM이 지도 작성과 위치 추정을, A*가 경로 계획을, YOLO가 실시간 객체 검출을 담당한다. 각 모듈은 센서 데이터를 융합해 안정적으로 주행하고 충돌을 회피한다. 하드웨어와 소프트웨어를 효과적으로 통합한 결과, 산업 현장과 유사한 환경에서 자율 주행, 물체 조작, 과제 수행 성능이 일관됐다.

arXiv:1906.00705

An Adaptive Training-less System for Anomaly Detection in Crowd Scenes

Anomaly detection in crowd videos has become a popular area of research for the computer vision community. Several existing methods generally perform a prior training about the scene with or without the use of labeled data. However, it is difficult to always guarantee the availability of prior data, especially, for scenarios like remote area surveillance. To address such challenge, we propose an adaptive training-less system capable of detecting anomaly on-the-fly while dynamically estimating and adjusting response based on certain parameters. This makes our system both training-less and adaptive in nature. Our pipeline consists of three main components, namely, adaptive 3D-DCT model for multi-object detection-based association, local motion structure description through saliency modulated optic flow, and anomaly detection based on earth movers distance (EMD). The proposed model, despite being training-free, is found to achieve comparable performance with several state-of-the-art methods on the publicly available UCSD, UMN, CHUK-Avenue and ShanghaiTech datasets.

arXiv:2101.09799공식 코드 · Papers with Code

Online Memory Leak Detection in the Cloud-based Infrastructures

A memory leak in an application deployed on the cloud can affect the availability and reliability of the application. Therefore, to identify and ultimately resolve it quickly is highly important. However, in the production environment running on the cloud, memory leak detection is a challenge without the knowledge of the application or its internal object allocation details. This paper addresses this challenge of online detection of memory leaks in cloud-based infrastructure without having any internal application knowledge by introducing a novel machine learning based algorithm Precog. This algorithm solely uses one metric i.e the system's memory utilization on which the application is deployed for the detection of a memory leak. The developed algorithm's accuracy was tested on 60 virtual machines manually labeled memory utilization data provided by our industry partner Huawei Munich Research Center and it was found that the proposed algorithm achieves the accuracy score of 85\% with less than half a second prediction time per virtual machine.

arXiv:2203.15362

Domain Invariant Siamese Attention Mask for Small Object Change Detection via Everyday Indoor Robot Navigation

The problem of image change detection via everyday indoor robot navigation is explored from a novel perspective of the self-attention technique. Detecting semantically non-distinctive and visually small changes remains a key challenge in the robotics community. Intuitively, these small non-distinctive changes may be better handled by the recent paradigm of the attention mechanism, which is the basic idea of this work. However, existing self-attention models require significant retraining cost per domain, so it is not directly applicable to robotics applications. We propose a new self-attention technique with an ability of unsupervised on-the-fly domain adaptation, which introduces an attention mask into the intermediate layer of an image change detection model, without modifying the input and output layers of the model. Experiments, in which an indoor robot aims to detect visually small changes in everyday navigation, demonstrate that our attention technique significantly boosts the state-of-the-art image change detection model.

arXiv:2005.11549

Self-supervised Robust Object Detectors from Partially Labelled Datasets

In the object detection task, merging various datasets from similar contexts but with different sets of Objects of Interest (OoI) is an inexpensive way (in terms of labor cost) for crafting a large-scale dataset covering a wide range of objects. Moreover, merging datasets allows us to train one integrated object detector, instead of training several ones, which in turn resulting in the reduction of computational and time costs. However, merging the datasets from similar contexts causes samples with partial labeling as each constituent dataset is originally annotated for its own set of OoI and ignores to annotate those objects that are become interested after merging the datasets. With the goal of training \emph{one integrated robust object detector with high generalization performance}, we propose a training framework to overcome missing-label challenge of the merged datasets. More specifically, we propose a computationally efficient self-supervised framework to create on-the-fly pseudo-labels for the unlabeled positive instances in the merged dataset in order to train the object detector jointly on both ground truth and pseudo labels. We evaluate our proposed framework for training Yolo on a simulated merged dataset with missing rate $\approx\!48\%$ using VOC2012 and VOC2007. We empirically show that generalization performance of Yolo trained on both ground truth and the pseudo-labels created by our method is on average $4\%$ higher than the ones trained only with the ground truth labels of the merged dataset.

arXiv:2111.08276공식 코드 · Papers with Code

Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts

Most existing methods in vision language pre-training rely on object-centric features extracted through object detection and make fine-grained alignments between the extracted features and texts. It is challenging for these methods to learn relations among multiple objects. To this end, we propose a new method called X-VLM to perform `multi-grained vision language pre-training.' The key to learning multi-grained alignments is to locate visual concepts in the image given the associated texts, and in the meantime align the texts with the visual concepts, where the alignments are in multi-granularity. Experimental results show that X-VLM effectively leverages the learned multi-grained alignments to many downstream vision language tasks and consistently outperforms state-of-the-art methods.

arXiv:2112.07671

Ghost Image Processing

In computational ghost imaging the object is illuminated with a sequence of known patterns, and the scattered light is collected using a detector that has no spatial resolution. Using those patterns and the total intensity measurement from the detector, one can reconstruct the desired image. Here we study how the reconstructed image is modified if the patterns used for the reconstruction are not the same as the illumination patterns, and show that one can choose how to illuminate the object, such that the reconstruction process behaves like a spatial filtering operation on the image. The ability to measure directly a processed image, allows one to bypass the post-processing steps, and thus avoid any noise amplification they imply. As a simple example we show the case of an edge-detection filter.

arXiv:2309.04887공식 코드 · Papers with Code

SortedAP: Rethinking evaluation metrics for instance segmentation

Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensitive to the change of masks or false predictions. For certain metrics, the score can change drastically in a narrow range which could provide a misleading indication of the quality gap between results. Therefore, we propose a new metric called sortedAP, which strictly decreases with both object- and pixel-level imperfections and has an uninterrupted penalization scale over the entire domain. We provide the evaluation toolkit and experiment code at https://www.github.com/looooongChen/sortedAP.

arXiv:2407.01558

Visual grounding for desktop graphical user interfaces

Most instance perception and image understanding solutions focus mainly on natural images. However, applications for synthetic images, and more specifically, images of Graphical User Interfaces (GUI) remain limited. This hinders the development of autonomous computer-vision-powered Artificial Intelligence (AI) agents. In this work, we present Instruction Visual Grounding or IVG, a multi-modal solution for object identification in a GUI. More precisely, given a natural language instruction and GUI screen, IVG locates the coordinates of the element on the screen where the instruction would be executed. To this end, we develop two methods. The first method is a three-part architecture that relies on a combination of a Large Language Model (LLM) and an object detection model. The second approach uses a multi-modal foundation model.

arXiv:1810.06055

A Simple Change Comparison Method for Image Sequences Based on Uncertainty Coefficient

For identification of change information in image sequences, most studies focus on change detection in one image sequence, while few studies have considered the change level comparison between two different image sequences. Moreover, most studies require the detection of image information in details, for example, object detection. Based on Uncertainty Coefficient(UC), this paper proposes an innovative method CCUC for change comparison between two image sequences. The proposed method is computationally efficient and simple to implement. The change comparison stems from video monitoring system. The limited number of provided screens and a large number of monitoring cameras require the videos or image sequences ordered by change level. We demonstrate this new method by applying it on two publicly available image sequences. The results are able to show the method can distinguish the different change level for sequences.