Visual object recognition is one of the most important perception functions for a wide range of intelligent machines. A conventional recognition process begins with forming a clear optical image of the object, followed by its computer analysis. In contrast, it is possible to carry out recognition without imaging by using coherent illumination and directly analyzing the optical interference pattern of the scattered light as captured by an image sensor. Here we show that such direct visual recognition can overcome traditional limitations of imaging optics to realize excellent recognition without focusing, beyond diffraction limit, or in the absence of direct line-of-sight.
The random matrix model is popular in extended object tracking, due to its relative simplicity and versatility. In this model, the extended object state consists of a kinematic vector for the position and motion parameters (velocity, etc), and an extent matrix. Two versions of the model can be found in literature, one where the state density is modelled by a conditional density, and one where the state density is modelled by a factorized density. In this paper, we present closed form Bayesian smoothing expression for both the conditional and the factorised model. In a simulation study, we compare the performance of different versions of the smoother.
In this paper, we present an algorithm for effectively reconstructing an object from a set of its tomographic projections without any knowledge of the viewing directions or any prior structural information, in the presence of pathological amounts of noise, unknown shifts in the projections, and outliers among the projections. The outliers are mainly in the form of a number of projections of a completely different object, as compared to the object of interest. We introduce a novel approach of first processing the projections, then obtaining an initial estimate for the orientations and the shifts, and then define a refinement procedure to obtain the final reconstruction. Even in the presence of high noise variance (up to $50\%$ of the average value of the (noiseless) projections) and presence of outliers, we are able to successfully reconstruct the object. We also provide interesting empirical comparisons of our method with the sparsity based optimization procedures that have been used earlier for image reconstruction tasks.
Object detection in streaming images is a major step in different detection-based applications, such as object tracking, action recognition, robot navigation, and visual surveillance applications. In mostcases, image quality is noisy and biased, and as a result, the data distributions are disturbed and imbalanced. Most object detection approaches, such as the faster region-based convolutional neural network (Faster RCNN), Single Shot Multibox Detector with 300x300 inputs (SSD300), and You Only Look Once version 2 (YOLOv2), rely on simple sampling without considering distortions and noise under real-world changing environments, despite poor object labeling. In this paper, we propose an Incremental active semi-supervised learning (IASSL) technology for unseen object detection. It combines batch-based active learning (AL) and bin-based semi-supervised learning (SSL) to leverage the strong points of AL's exploration and SSL's exploitation capabilities. A collaborative sampling method is also adopted to measure the uncertainty and diversity of AL and the confidence in SSL. Batch-based AL allows us to select more informative, confident, and representative samples with low cost. Bin-based SSL divides streaming image samples into several bins, and each bin repeatedly transfers the discriminative knowledge of convolutional neural network (CNN) deep learning to the next bin until the performance criterion is reached. IASSL can overcome noisy and biased labels in unknown, cluttered data distributions. We obtain superior performance, compared to state-of-the-art technologies such as Faster RCNN, SSD300, and YOLOv2.
Accurate and robust tracking of surrounding road participants plays an important role in autonomous driving. However, there is usually no prior knowledge of the number of tracking targets due to object emergence, object disappearance and false alarms. To overcome this challenge, we propose a generic vehicle tracking framework based on modified mixture particle filter, which can make the number of tracking targets adaptive to real-time observations and track all the vehicles within sensor range simultaneously in a uniform architecture without explicit data association. Each object corresponds to a mixture component whose distribution is non-parametric and approximated by particle hypotheses. Most tracking approaches employ vehicle kinematic models as the prediction model. However, it is hard for these models to make proper predictions when sensor measurements are lost or become low quality due to partial or complete occlusions. Moreover, these models are incapable of forecasting sudden maneuvers. To address these problems, we propose to incorporate learning-based behavioral models instead of pure vehicle kinematic models to realize prediction in the prior update of recursive Bayesian state estimation. Two typical driving scenarios including lane keeping and lane change are demonstrated to verify the effectiveness and accuracy of the proposed framework as well as the advantages of employing learning-based models.
Recovering the image of an object from its phaseless speckle pattern is difficult. Let alone the transmission matrix is unknown in multiple scattering media imaging. Double phase retrieval is a recently proposed efficient method which recovers the unknown object from its phaseless measurements by two steps with phase retrieval. In this paper, we combine the two steps in double phase retrieval and construct an end-to-end neural network called TCNN(Transforming Convolutional Neural Network) which directly learns the relationship between the phaseless measurements and the object. TCNN contains a special layer called transform layer which aims to be a bridge between different transform domains. Tested by the empirical data provided in\cite{Metzler2017Coherent}, images can be recovered by TCNN with comparable quality compared with state-of-the-art methods. Not only the transmission matrix needn't to be calculated but also the time to recover the object can be hugely reduced once the parameters of TCNN are stable.
The most unique advantage of multipass SAR interferometry (InSAR) is the retrieval of long term geophysical parameters, e.g. linear deformation rates, over large areas. Recently, an object-based multipass InSAR framework has been proposed in [1], as an alternative to the typical single-pixel methods, e.g. Persistent Scatterer Interferometry (PSI), or pixel-cluster-based methods, e.g. SqueeSAR. This enables the exploitation of inherent properties of InSAR phase stacks on an object level. As a followon, this paper investigates the inherent low rank property of such phase tensors, and proposes a Robust Multipass InSAR technique via Object-based low rank tensor decomposition (RoMIO). We demonstrate that the filtered InSAR phase stacks can improve the accuracy of geophysical parameters estimated via conventional multipass InSAR techniques, e.g. PSI, by a factor of ten to thirty in typical settings. The proposed method is particularly effective against outliers, such as pixels with unmodeled phases. These merits in turn can effectively reduce the number of images required for a reliable estimation. The promising performance of the proposed method is demonstrated using high-resolution TerraSAR-X image stacks.
We address the task of ranking objects (such as people, blogs, or verticals) that, unlike documents, do not have direct term-based representations. To be able to match them against keyword queries, evidence needs to be amassed from documents that are associated with the given object. We present two design patterns, i.e., general reusable retrieval strategies, which are able to encompass most existing approaches from the past. One strategy combines evidence on the term level (early fusion), while the other does it on the document level (late fusion). We demonstrate the generality of these patterns by applying them to three different object retrieval tasks: expert finding, blog distillation, and vertical ranking.
Deep convolutional neural networks (CNNs) trained for object classification have a number of striking similarities with the primate ventral visual stream. In particular, activity in early, intermediate, and late layers is closely related to activity in V1, V4, and the inferotemporal cortex (IT). This study further compares activity in late layers of object-classification CNNs to activity patterns reported in the IT electrophysiology literature. There are a number of close similarities, including the distributions of population response sparseness across stimuli, and the distribution of size tuning bandwidth. Statisics of scale invariance, responses to clutter and occlusion, and orientation tuning are less similar. Statistics of object selectivity are quite different. These results agree with recent studies that highlight strong parallels between object-categorization CNNs and the ventral stream, and also highlight differences that could perhaps be reduced in future CNNs.
Dreaming is generally thought to be generated by spontaneous brain activity during sleep with patterns common to waking experience. This view is supported by a recent study demonstrating that dreamed objects can be predicted from brain activity during sleep using statistical decoders trained with stimulus-induced brain activity. However, it remains unclear whether and how visual image features associated with dreamed objects are represented in the brain. In this study, we used a deep neural network (DNN) model for object recognition as a proxy for hierarchical visual feature representation, and DNN features for dreamed objects were analyzed with brain decoding of fMRI data collected during dreaming. The decoders were first trained with stimulus-induced brain activity labeled with the feature values of the stimulus image from multiple DNN layers. The decoders were then used to decode DNN features from the dream fMRI data, and the decoded features were compared with the averaged features of each object category calculated from a large-scale image database. We found that the feature values decoded from the dream fMRI data positively correlated with those associated with dreamed object categories at mid- to high-level DNN layers. Using the decoded features, the dreamed object category could be identified at above-chance levels by matching them to the averaged features for candidate categories. The results suggest that dreaming recruits hierarchical visual feature representations associated with objects, which may support phenomenal aspects of dream experience.
In 3-D reconstruction problems, the image data obtained from cryo electron microscopy is the projection of many heterogeneous instances of the object under study (e.g., a virus). When the object is heterogeneous but has an overall symmetry, it is natural to describe the object as stochastic with symmetrical statistics. This paper presents a maximum likelihood reconstruction approach which allows each object to lack symmetry while constraining the {\it statistics} of the ensemble of objects to have symmetry. This algorithm is demonstrated on bacteriophage HK97 and is contrasted with an existing algorithm in which each object, while still heterogeneous, has the symmetry. Reconstruction results show that the proposed algorithm eliminates long-standing distortions in previous heterogeneity calculations associated with symmetry axes, and provides estimates that make more biologically sense than the estimates of existing algorithms.
Following widely used in visual recognition concept of relative attributes, the article establishes definition of the relative PCA attributes for a class of objects defined by vectors of their parameters. A new rating model (RELARM) is built using relative PCA attribute ranking functions for rating object description and k-means clustering algorithm. Rating assignment of each rating object to a rating category is derived as a result of cluster centers projection on the specially selected rating vector. Empirical study has shown a high level of approximation to the existing S & P, Moody's and Fitch ratings.
Hackl, Koster-Hale & Varvoutis (2012; Journal of Semantics, 29, 145-206; HKV) provide data that suggested that in a null context, antecedent-contained-deletion (ACD) relative clause structures modifying a quantified object noun phrase are easier to process than those modifying a definite object NP. HKV argue that this pattern of results supports a quantifier-raising (QR) analysis of both ACD structures and quantified NPs in object position: under the account that they advocate, both ACD resolution and quantified NPs in object position require movement of the object NP to a higher syntactic position. The processing advantage for quantified object NPs in ACD is hypothesized to derive from the fact that - at the point where ACD resolution must take place - the quantified NP has already undergone QR whereas this is not the case for definite NPs. Here, we report attempted replications of their self-paced reading Experiments 1 and 2. We do not replicate the critical interactions in any of the words immediately following the disambiguating verb in either experiment. Putting these observations together with the observation that it was only post-hoc analysis decisions that were responsible for HKV's observed effects in the first place (Gibson et al., submitted), we conclude that the experiments reported by HKV should not be viewed as providing evidence for the ACD quantifier raising processing effect.
In this paper we set out to solve the task of 6-DOF 3D object detection from 2D images, where the only supervision is a geometric representation of the objects we aim to find. In doing so, we remove the need for 6-DOF labels (i.e., position, orientation etc.), allowing our network to be trained on unlabeled images in a self-supervised manner. We achieve this through a neural network which learns an explicit scene parameterization which is subsequently passed into a differentiable renderer. We analyze why analysis-by-synthesis-like losses for supervision of 3D scene structure using differentiable rendering is not practical, as it almost always gets stuck in local minima of visual ambiguities. This can be overcome by a novel form of training, where an additional network is employed to steer the optimization itself to explore the entire parameter space i.e., to be curious, and hence, to resolve those ambiguities and find workable minima.
Recently, many researchers have attempted to improve deep learning-based object detection models, both in terms of accuracy and operational speeds. However, frequently, there is a trade-off between speed and accuracy of such models, which encumbers their use in practical applications such as autonomous navigation. In this paper, we explore a meta-cognitive learning strategy for object detection to improve generalization ability while at the same time maintaining detection speed. The meta-cognitive method selectively samples the object instances in the training dataset to reduce overfitting. We use YOLO v3 Tiny as a base model for the work and evaluate the performance using the MS COCO dataset. The experimental results indicate an improvement in absolute precision of 2.6% (minimum), and 4.4% (maximum), with no overhead to inference time.
We present a novel explicit shape representation for instance segmentation. Based on how to model the object shape, current instance segmentation systems can be divided into two categories, implicit and explicit models. The implicit methods, which represent the object mask/contour by intractable network parameters, and produce it through pixel-wise classification, are predominant. However, the explicit methods, which parameterize the shape with simple and explainable models, are less explored. Since the operations to generate the final shape are light-weighted, the explicit methods have a clear speed advantage over implicit methods, which is crucial for real-world applications. The proposed USD-Seg adopts a linear model, sparse coding with dictionary, for object shapes. First, it learns a dictionary from a large collection of shape datasets, making any shape being able to be decomposed into a linear combination through the dictionary. Hence the name "Universal Shape Dictionary". Then it adds a simple shape vector regression head to ordinary object detector, giving the detector segmentation ability with minimal overhead. For quantitative evaluation, we use both average precision (AP) and the proposed Efficiency of AP (AP$_E$) metric, which intends to also measure the computational consumption of the framework to cater to the requirements of real-world applications. We report experimental results on the challenging COCO dataset, in which our single model on a single Titan Xp GPU achieves 35.8 AP and 27.8 AP$_E$ at 65 fps with YOLOv4 as base detector, 34.1 AP and 28.6 AP$_E$ at 12 fps with FCOS as base detector.
Blastomere instance segmentation is important for analyzing embryos' abnormality. To measure the accurate shapes and sizes of blastomeres, their amodal segmentation is necessary. Amodal instance segmentation aims to recover the complete silhouette of an object even when the object is not fully visible. For each detected object, previous methods directly regress the target mask from input features. However, images of an object under different amounts of occlusion should have the same amodal mask output, which makes it harder to train the regression model. To alleviate the problem, we propose to classify input features into intermediate shape codes and recover complete object shapes from them. First, we pre-train the Vector Quantized Variational Autoencoder (VQ-VAE) model to learn these discrete shape codes from ground truth amodal masks. Then, we incorporate the VQ-VAE model into the amodal instance segmentation pipeline with an additional refinement module. We also detect an occlusion map to integrate occlusion information with a backbone feature. As such, our network faithfully detects bounding boxes of amodal objects. On an internal embryo cell image benchmark, the proposed method outperforms previous state-of-the-art methods. To show generalizability, we show segmentation results on the public KINS natural image benchmark. To examine the learned shape codes and model design choices, we perform ablation studies on a synthetic dataset of simple overlaid shapes. Our method would enable accurate measurement of blastomeres in in vitro fertilization (IVF) clinics, which potentially can increase IVF success rate.
Modeling the hand-object (HO) interaction not only requires estimation of the HO pose, but also pays attention to the contact due to their interaction. Significant progress has been made in estimating hand and object separately with deep learning methods, simultaneous HO pose estimation and contact modeling has not yet been fully explored. In this paper, we present an explicit contact representation namely Contact Potential Field (CPF), and a learning-fitting hybrid framework namely MIHO to Modeling the Interaction of Hand and Object. In CPF, we treat each contacting HO vertex pair as a spring-mass system. Hence the whole system forms a potential field with minimal elastic energy at the grasp position. Extensive experiments on the two commonly used benchmarks have demonstrated that our method can achieve state-of-the-art in several reconstruction metrics, and allow us to produce more physically plausible HO pose even when the ground-truth exhibits severe interpenetration or disjointedness. Our code is available at https://github.com/lixiny/CPF.
Feature pyramid network (FPN) is a critical component in modern object detection frameworks. The performance gain in most of the existing FPN variants is mainly attributed to the increase of computational burden. An attempt to enhance the FPN is enriching the spatial information by expanding the receptive fields, which is promising to largely improve the detection accuracy. In this paper, we first investigate how expanding the receptive fields affect the accuracy and computational costs of FPN. We explore a baseline model called inception FPN in which each lateral connection contains convolution filters with different kernel sizes. Moreover, we point out that not all objects need such a complicated calculation and propose a new dynamic FPN (DyFPN). The output features of DyFPN will be calculated by using the adaptively selected branch according to a dynamic gating operation. Therefore, the proposed method can provide a more efficient dynamic inference for achieving a better trade-off between accuracy and computational cost. Extensive experiments conducted on MS-COCO benchmark demonstrate that the proposed DyFPN significantly improves performance with the optimal allocation of computation resources. For instance, replacing inception FPN with DyFPN reduces about 40% of its FLOPs while maintaining similar high performance.
Although the number of camera-based sensors mounted on vehicles has recently increased dramatically, robust and accurate object velocity detection is difficult. Additionally, it is still common to use radar as a fusion system. We have developed a method to accurately detect the velocity of object using a camera, based on a large-scale dataset collected over 20 years by the automotive manufacturer, SUBARU. The proposed method consists of three methods: an High Dynamic Range (HDR) detection method that fuses multiple stereo disparity images, a fusion method that combines the results of monocular and stereo recognitions, and a new velocity calculation method. The evaluation was carried out using measurement devices and a test course that can quantitatively reproduce severe environment by mounting the developed stereo camera on an actual vehicle.