Interest Point Detection

The COCO (Common Objects in Context) dataset is a large-scale object detection, segmentation, and captioning dataset. It is designed to encourage research on a wide variety of object categories and is commonly used for benchmarking computer vision models. It is an essential dataset for researchers and developers working on object detection, segmentation, and pose estimation tasks.

Planar object tracking is an actively studied problem in vision-based robotic applications. While several benchmarks have been constructed for evaluating state-of-theart algorithms, there is a lack of video sequences captured in the wild rather than in constrained laboratory environment. In this paper, we present a carefully designed planar object tracking benchmark containing 210 videos of 30 planar objects sampled in the natural environment. In particular, for each object, we shoot seven videos involving various challenging factors, namely scale change, rotation, perspective distortion, motion blur, occlusion, out-of-view, and unconstrained. The ground truth is carefully annotated semi-manually to ensure the quality. Moreover, eleven state-of-the-art algorithms are evaluated on the benchmark using two evaluation metrics, with detailed analysis provided for the evaluation results. We expect the proposed benchmark to benefit future studies on planar object tracking.

Synthetic COCO (S-COCO) is a synthetically created dataset for homography estimation learning. It was introduced by DeTone et al., where the source and target images are generated by duplicating the same COCO image. The source patch $I_S$ is generated by randomly cropping a source candidate at position $p$ with a size of 128 ×128 pixels. Then the patch’s corners are randomly perturbed vertically and horizontally by values within the range [−$\rho$,$\rho$] and the four correspondences define a homography $H_{ST}$ . The inverse of this homography $H_{TS} = (H_{ST} )^{-1}$ is applied to the target candidate and from the resulted warped image a target patch $I_T$ is cropped at the same location p. Both $I_S$ and $I_T$ are the input data with the homography $H_{ST}$ as ground truth.

YUD+ is a dataset containing additional Vanishing Point Labels for the York Urban Database. Source: YUD+

NYU-VP is a new dataset for multi-model fitting, vanishing point (VP) estimation in this case. Each image is annotated with up to eight vanishing points, and pre-extracted line segments are provided which act as data points for a robust estimator. Due to its size, the dataset is the first to allow for supervised learning of a multi-model fitting task. Source: CONSAC: Robust Multi-Model Fitting by Conditional Sample Consensus

Photometrically Distorted Synthetic COCO (PDS-COCO) dataset is a synthetically created dataset for homography estimation learning. The idea is exactly the same as in the Synthetic COCO (S-COCO) dataset with SSD-like image distortion added at the beginning of the whole procedure: the first step involves adjusting the brightness of the image using randomly picked value $\delta_b \in \mathcal{U}(-32, 32)$. Next, contrast, saturation and hue noise is applied with the following values: $\delta_c \in \mathcal{U}(0.5, 1.5)$, $\delta_s \in \mathcal{U}(0.5, 1.5)$ and $\delta_h \in \mathcal{U}(-18, 18)$. Finally, the color channels of the image are randomly swapped with a probability of $0.5$. Such a photometric distortion procedure is applied to the original image independently to create source and target candidates.

In order to collect thermal aerial data, we used FLIR's Boson thermal imager (8.7 mm focal length, 640p resolution, and $50^\circ$ horizontal field of view)\footnote{\url{https://www.flir.es/products/boson/}}. The collected images are nadir at approx. 1m/px spatial resolution. We performed six flights from 9:00 PM to 4:00 AM and label this dataset as \textbf{Boson-nighttime}, accordingly. To create a single map, we first run a structure-from-motion (SfM) algorithm to reconstruct the thermal map from multiple views. Subsequently, orthorectification is performed by aligning the photometric satellite maps with thermal maps at the same spatial resolution. The ground area covered by Boson-nighttime measures $33~\text{km}{^2}$ in total. The most prevalent map feature is the desert, with small portions of farms, roads, and buildings. The Bing satellite map\footnote{\url{https://www.bing.com/maps/aerial}} is cropped in the corresponding area as our satellite reference map. We tile the thermal map into $512\times512$~px thermal image crops with a stride of $35$ px. Each thermal image crop pairs with the corresponding satellite image crop. Areas covered by three flights of Boson-nighttime are used for training and validation. The remaining areas, covered by the other three flights are used for testing. The train/validation/test splits for Boson-nighttime are $10256$/$13011$/$26568$ pairs of satellite and thermal image crops, respectively.

A dataset for benchmarking action recognition algorithms in natural environments, while making use of 3D information. The dataset contains around 650 video clips, across 14 classes. In addition, two state of the art action recognition algorithms are extended to make use of the 3D data, and five new interest point detection strategies are also proposed, that extend to the 3D data. Source: Hollywood 3D: Recognizing Actions in 3D Natural Scenes

Consolidates the world cup 2014 (WC14) and time-series world cup (TSWC) datasets and refines their homography annotations.

A large video dataset with dynamic content. Source: Deep Homography Estimation for Dynamic Scenes