Trajectory Prediction

SimAug, or Simulation as Augmentation, is a data augmentation method for trajectory prediction. It augments the representation such that it is robust to the variances in semantic scenes and camera views. First, to deal with the gap between real and synthetic semantic scene, it represents each training trajectory by high-level scene semantic segmentation features, and defends the model from adversarial examples generated by whitebox attack methods. Second, to overcome the changes in camera views, it generates multiple views for the same trajectory, and encourages the model to focus on the “hardest” view to which the model has learned. The classification loss is adopted and the view with the highest loss is favored during training. Finally, the augmented trajectory is computed as a convex combination of the trajectories generated in previous steps. The trajectory prediction model is built on a multi-scale representation and the final model is trained to minimize the empirical vicinal risk over the distribution of augmented trajectories.

Social-STGCNN is a method for human trajectory prediction. Pedestrian trajectories are not only influenced by the pedestrian itself but also by interaction with surrounding objects.

An aspect of Bi-RNNs that could be undesirable is the architecture's symmetry in both time directions. Bi-RNNs are often used in natural language processing, where the order of the words is almost exclusively determined by grammatical rules and not by temporal sequentiality. However, in some cases, the data has a preferred direction in time: the forward direction. Another potential drawback of Bi-RNNs is that their output is simply the concatenation of two naive readings of the input in both directions. In consequence, Bi-RNNs never actually read an input by knowing what happens in the future. Conversely, the idea behind U-RNN, is to first do a backward pass, and then use during the forward pass information about the future. We accumulate information while knowing which part of the information will be useful in the future as it should be relevant to do so if the forward direction is the preferred direction of the data. The backward and forward hidden states $(h^b_t)$ and $(h^f_t)$ are obtained according to these equations: \begin{equation} \begin{aligned} &h_{t-1}^{b}=R N N\left(h_{t}^{b}, e_{t}, W_{b}\right) \ &h_{t+1}^{f}=R N N\left(h_{t}^{f},\left[e_{t}, h_{t}^{b}\right], W_{f}\right) \end{aligned} \end{equation} where $W_b$ and $W_f$ are learnable weights that are shared among pedestrians, and $[\cdot, \cdot]$ denotes concatenation. The last hidden state $h^f_{T_{obs}}$ is then used as the encoding of the sequence.

JLA, or Joint Learning Architecture, is an approach for multiple object tracking and trajectory forecasting. It jointly trains a tracking and trajectory forecasting model, and the trajectory forecasts are used for short-term motion estimates in lieu of linear motion prediction methods such as the Kalman filter. It uses a FairMOT model as the base model because this architecture already performs detection and tracking. A forecasting branch is added to the network and is trained end-to-end. FairMOT consist of a backbone network utilizing Deep Layer Aggregation, an object detection head, and a reID head.