PyTorchKR Papers

최신 논문

총 3,152편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1805.10799공식 코드 · Papers with Code

Interactive Text2Pickup Network for Natural Language based Human-Robot Collaboration

In this paper, we propose the Interactive Text2Pickup (IT2P) network for human-robot collaboration which enables an effective interaction with a human user despite the ambiguity in user's commands. We focus on the task where a robot is expected to pick up an object instructed by a human, and to interact with the human when the given instruction is vague. The proposed network understands the command from the human user and estimates the position of the desired object first. To handle the inherent ambiguity in human language commands, a suitable question which can resolve the ambiguity is generated. The user's answer to the question is combined with the initial command and given back to the network, resulting in more accurate estimation. The experiment results show that given unambiguous commands, the proposed method can estimate the position of the requested object with an accuracy of 98.49% based on our test dataset. Given ambiguous language commands, we show that the accuracy of the pick up task increases by 1.94 times after incorporating the information obtained from the interaction.

arXiv:1803.05262공식 코드 · Papers with Code

Learning to Play General Video-Games via an Object Embedding Network

Deep reinforcement learning (DRL) has proven to be an effective tool for creating general video-game AI. However most current DRL video-game agents learn end-to-end from the video-output of the game, which is superfluous for many applications and creates a number of additional problems. More importantly, directly working on pixel-based raw video data is substantially distinct from what a human player does.In this paper, we present a novel method which enables DRL agents to learn directly from object information. This is obtained via use of an object embedding network (OEN) that compresses a set of object feature vectors of different lengths into a single fixed-length unified feature vector representing the current game-state and fulfills the DRL simultaneously. We evaluate our OEN-based DRL agent by comparing to several state-of-the-art approaches on a selection of games from the GVG-AI Competition. Experimental results suggest that our object-based DRL agent yields performance comparable to that of those approaches used in our comparative study.

arXiv:1805.11227공식 코드 · Papers with Code

Getting to Know Low-light Images with The Exclusively Dark Dataset

Low-light is an inescapable element of our daily surroundings that greatly affects the efficiency of our vision. Research works on low-light has seen a steady growth, particularly in the field of image enhancement, but there is still a lack of a go-to database as benchmark. Besides, research fields that may assist us in low-light environments, such as object detection, has glossed over this aspect even though breakthroughs-after-breakthroughs had been achieved in recent years, most noticeably from the lack of low-light data (less than 2% of the total images) in successful public benchmark dataset such as PASCAL VOC, ImageNet, and Microsoft COCO. Thus, we propose the Exclusively Dark dataset to elevate this data drought, consisting exclusively of ten different types of low-light images (i.e. low, ambient, object, single, weak, strong, screen, window, shadow and twilight) captured in visible light only with image and object level annotations. Moreover, we share insightful findings in regards to the effects of low-light on the object detection task by analyzing visualizations of both hand-crafted and learned features. Most importantly, we found that the effects of low-light reaches far deeper into the features than can be solved by simple "illumination invariance'". It is our hope that this analysis and the Exclusively Dark dataset can encourage the growth in low-light domain researches on different fields. The Exclusively Dark dataset with its annotation is available at https://github.com/cs-chan/Exclusively-Dark-Image-Dataset

arXiv:1806.00557공식 코드 · Papers with Code

CubeSLAM: Monocular 3D Object SLAM

We present a method for single image 3D cuboid object detection and multi-view object SLAM in both static and dynamic environments, and demonstrate that the two parts can improve each other. Firstly for single image object detection, we generate high-quality cuboid proposals from 2D bounding boxes and vanishing points sampling. The proposals are further scored and selected based on the alignment with image edges. Secondly, multi-view bundle adjustment with new object measurements is proposed to jointly optimize poses of cameras, objects and points. Objects can provide long-range geometric and scale constraints to improve camera pose estimation and reduce monocular drift. Instead of treating dynamic regions as outliers, we utilize object representation and motion model constraints to improve the camera pose estimation. The 3D detection experiments on SUN RGBD and KITTI show better accuracy and robustness over existing approaches. On the public TUM, KITTI odometry and our own collected datasets, our SLAM method achieves the state-of-the-art monocular camera pose estimation and at the same time, improves the 3D object detection accuracy.

arXiv:1806.01673공식 코드 · Papers with Code

Recurrent Convolutional Fusion for RGB-D Object Recognition

Providing machines with the ability to recognize objects like humans has always been one of the primary goals of machine vision. The introduction of RGB-D cameras has paved the way for a significant leap forward in this direction thanks to the rich information provided by these sensors. However, the machine vision community still lacks an effective method to synergically use the RGB and depth data to improve object recognition. In order to take a step in this direction, we introduce a novel end-to-end architecture for RGB-D object recognition called recurrent convolutional fusion (RCFusion). Our method generates compact and highly discriminative multi-modal features by combining complementary RGB and depth information representing different levels of abstraction. Extensive experiments on two popular datasets, RGB-D Object Dataset and JHUIT-50, show that RCFusion significantly outperforms state-of-the-art approaches in both the object categorization and instance recognition tasks.

arXiv:1801.02108공식 코드 · Papers with Code

SBNet: Sparse Blocks Network for Fast Inference

Conventional deep convolutional neural networks (CNNs) apply convolution operators uniformly in space across all feature maps for hundreds of layers - this incurs a high computational cost for real-time applications. For many problems such as object detection and semantic segmentation, we are able to obtain a low-cost computation mask, either from a priori problem knowledge, or from a low-resolution segmentation network. We show that such computation masks can be used to reduce computation in the high-resolution main network. Variants of sparse activation CNNs have previously been explored on small-scale tasks and showed no degradation in terms of object classification accuracy, but often measured gains in terms of theoretical FLOPs without realizing a practical speed-up when compared to highly optimized dense convolution implementations. In this work, we leverage the sparsity structure of computation masks and propose a novel tiling-based sparse convolution algorithm. We verified the effectiveness of our sparse CNN on LiDAR-based 3D object detection, and we report significant wall-clock speed-ups compared to dense convolution without noticeable loss of accuracy.

arXiv:1806.03772공식 코드 · Papers with Code

DOOBNet: Deep Object Occlusion Boundary Detection from an Image

Object occlusion boundary detection is a fundamental and crucial research problem in computer vision. This is challenging to solve as encountering the extreme boundary/non-boundary class imbalance during training an object occlusion boundary detector. In this paper, we propose to address this class imbalance by up-weighting the loss contribution of false negative and false positive examples with our novel Attention Loss function. We also propose a unified end-to-end multi-task deep object occlusion boundary detection network (DOOBNet) by sharing convolutional features to simultaneously predict object boundary and occlusion orientation. DOOBNet adopts an encoder-decoder structure with skip connection in order to automatically learn multi-scale and multi-level features. We significantly surpass the state-of-the-art on the PIOD dataset (ODS F-score of .702) and the BSDS ownership dataset (ODS F-score of .555), as well as improving the detecting speed to as 0.037s per image on the PIOD dataset.

arXiv:1803.08494공식 코드 · Papers with Code

Group Normalization

FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

arXiv:1711.11575공식 코드 · Papers with Code

Relation Networks for Object Detection

Although it is well believed for years that modeling relations between objects would help object recognition, there has not been evidence that the idea is working in the deep learning era. All state-of-the-art object detection systems still rely on recognizing object instances individually, without exploiting their relations during learning. This work proposes an object relation module. It processes a set of objects simultaneously through interaction between their appearance feature and geometry, thus allowing modeling of their relations. It is lightweight and in-place. It does not require additional supervision and is easy to embed in existing networks. It is shown effective on improving object recognition and duplicate removal steps in the modern object detection pipeline. It verifies the efficacy of modeling object relations in CNN based detection. It gives rise to the first fully end-to-end object detector.

arXiv:1806.06198공식 코드 · Papers with Code

Part-Aware Fine-grained Object Categorization using Weakly Supervised Part Detection Network

Fine-grained object categorization aims for distinguishing objects of subordinate categories that belong to the same entry-level object category. The task is challenging due to the facts that (1) training images with ground-truth labels are difficult to obtain, and (2) variations among different subordinate categories are subtle. It is well established that characterizing features of different subordinate categories are located on local parts of object instances. In fact, careful part annotations are available in many fine-grained categorization datasets. However, manually annotating object parts requires expertise, which is also difficult to generalize to new fine-grained categorization tasks. In this work, we propose a Weakly Supervised Part Detection Network (PartNet) that is able to detect discriminative local parts for use of fine-grained categorization. A vanilla PartNet builds on top of a base subnetwork two parallel streams of upper network layers, which respectively compute scores of classification probabilities (over subordinate categories) and detection probabilities (over a specified number of discriminative part detectors) for local regions of interest (RoIs). The image-level prediction is obtained by aggregating element-wise products of these region-level probabilities. To generate a diverse set of RoIs as inputs of PartNet, we propose a simple Discretized Part Proposals module (DPP) that directly targets for proposing candidates of discriminative local parts, with no bridging via object-level proposals. Experiments on the benchmark CUB-200-2011 and Oxford Flower 102 datasets show the efficacy of our proposed method for both discriminative part detection and fine-grained categorization. In particular, we achieve the new state-of-the-art performance on CUB-200-2011 dataset when ground-truth part annotations are not available.

arXiv:1806.06157공식 코드 · Papers with Code

Object Level Visual Reasoning in Videos

Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context. The next open challenges in activity recognition require a level of understanding that pushes beyond this and call for models with capabilities for fine distinction and detailed comprehension of interactions between actors and objects in a scene. We propose a model capable of learning to reason about semantically meaningful spatiotemporal interactions in videos. The key to our approach is a choice of performing this reasoning at the object level through the integration of state of the art object detection networks. This allows the model to learn detailed spatial interactions that exist at a semantic, object-interaction relevant level. We evaluate our method on three standard datasets (Twenty-BN Something-Something, VLOG and EPIC Kitchens) and achieve state of the art results on all of them. Finally, we show visualizations of the interactions learned by the model, which illustrate object classes and their interactions corresponding to different activity classes.

arXiv:2404.10034v3cs.CV공식 코드 · Papers with Code

약지도 객체 위치 추정 평가를 위한 현실적인 프로토콜

A Realistic Protocol for Evaluation of Weakly Supervised Object Localization

약지도 객체 위치 추정(Weakly Supervised Object Localization, WSOL)은 전역 클래스 수준 레이블만을 사용하여 분류 및 위치 추정(LOC)을 위한 심층 학습 모델을 학습할 수 있게 한다. 학습 중 경계 상자(bounding box, bbox) 지도 정보의 부재는 하이퍼파라미터 튜닝, 모델 선택, 평가 측면에서 문헌상 여러 과제를 야기한다. WSOL 방법들은 모델 선택을 위해 경계 상자 어노테이션이 포함된 검증 세트에, 그리고 위치 추정 맵으로부터 경계 상자를 생성하기 위한 임계값 추정에 경계 상자 어노테이션이 포함된 테스트 세트에 의존한다. 그러나 이 접근 방식은 실제 환경에서 해당 어노테이션을 일반적으로 구할 수 없다는 점에서 WSOL 설정과 부합하지 않는다. 초기 실증 분석에서는 모델 선택과 임계값 추정을 각각 클래스 레이블과 이미지 자체에만 의존할 경우, 수동 경계 상자 어노테이션을 사용하는 경우에 비해 위치 추정 성능이 크게 저하됨을 확인하였다. 이는 최적의 모델 성능을 위해 경계 상자 레이블을 활용하는 것의 중요성을 강조한다. 본 논문에서는 수동 경계 상자 어노테이션 없이 위치 추정 정보를 제공하는 새로운 WSOL 평가 프로토콜을 제안한다. 구체적으로, Selective Search, CLIP, RPN과 같은 사전학습된 기성 영역 제안(region proposal) 방법으로부터 노이즈가 포함된 의사 경계 상자(pseudo-box)를 생성하여 모델 선택에 활용하였다. 이러한 경계 상자는 위치 추정 맵에서 임계값을 추정하는 데에도 사용되어 테스트 세트 경계 상자 어노테이션의 필요성을 우회한다. 도전적인 자연 이미지 및 의료 이미지 데이터셋에서 여러 WSOL 방법으로 수행한 실험 결과, 제안하는 의사 경계 상자를 검증에 활용하면 모델 선택과 임계값 추정이 용이해지며, 검증 세트에서 GT 경계 상자로 선택하고 테스트 세트에서 임계값을 추정한 모델과 비슷한 위치 추정 성능을 달성함을 보였다. 또한 클래스 수준 레이블로 선택한 후 위치 추정 맵만으로 동적 임계값을 적용한 모델보다 우수한 성능을 나타냈다.