Human Activity Recognition

AdaRNN is an adaptive RNN that learns an adaptive model through two modules: Temporal Distribution Characterization (TDC) and Temporal Distribution Matching (TDM) algorithms. Firstly, to better characterize the distribution information in time-series, TDC splits the training data into $K$ most diverse periods that have a large distribution gap inspired by the principle of maximum entropy. After that, a temporal distribution matching (TDM) algorithm is used to dynamically reduce distribution divergence using a RNN-based model.

Temporal Distribution Characterization, or TDC, is a module used in the AdaRNN architecture to characterize the distributional information in a time series. Based on the principle of maximum entropy, maximizing the utilization of shared knowledge underlying a times series under temporal covariate shift can be done by finding periods which are most dissimilar to each other, which is also considered as the worst case of temporal covariate shift since the cross-period distributions are the most diverse. TDC achieves this goal for splitting the time-series by solving an optimization problem whose objective can be formulated as: $$ \max \_{0<K \leq K\_{0}} \max \_{n\_{1}, \cdots, n\_{K}} \frac{1}{K} \sum_{1 \leq i \neq j \leq K} d\left(\mathcal{D}\_{i}, \mathcal{D}\_{j}\right) $$ $$ \text { s.t. } \forall i, \Delta_{1}<\left|\mathcal{D}\_{i}\right|<\Delta_{2} ; \sum_{i}\left|\mathcal{D}\_{i}\right|=n $$ where $d$ is a distance metric, $\Delta\_{1}$ and $\Delta\_{2}$ are predefined parameters to avoid trivial solutions (e.g., very small values or very large values may fail to capture the distribution information), and $K\_{0}$ is the hyperparameter to avoid over-splitting. The metric $d(\cdot, \cdot)$ above can be any distance function, e.g., Euclidean or Editing distance, or some distribution-based distance / divergence, like MMD [14] and KL-divergence. The learning goal of the optimization problem (1) is to maximize the averaged period-wise distribution distances by searching $K$ and the corresponding periods so that the distributions of each period are as diverse as possible and the learned prediction model has better a more generalization ability.

Temporal Distribution Matching, or TDM, is a module used in the AdaRNN architecture to match the distributions of the discovered periods to build a time series prediction model $\mathcal{M}$ Given the learned time periods, the TDM module is designed to learn the common knowledge shared by different periods via matching their distributions. Thus, the learned model $\mathcal{M}$ is expected to generalize well on unseen test data compared with the methods which only rely on local or statistical information. Within the context of AdaRNN, Temporal Distribution Matching aims to adaptively match the distributions between the RNN cells of two periods while capturing the temporal dependencies. TDM introduces the importance vector $\mathbf{\alpha} \in \mathbb{R}^{\hat{V}}$ to learn the relative importance of $V$ hidden states inside the RNN, where all the hidden states are weighted with a normalized $\alpha$. Note that for each pair of periods, there is an $\mathbf{\alpha}$, and we omit the subscript if there is no confusion. In this way, we can dynamically reduce the distribution divergence of cross-periods. Given a period-pair $\left(\mathcal{D}\_{i}, \mathcal{D}\_{j}\right)$, the loss of temporal distribution matching is formulated as: $$ \mathcal{L}\_{t d m}\left(\mathcal{D}\_{i}, \mathcal{D}\_{j} ; \theta\right)=\sum_{t=1}^{V} \alpha\_{i, j}^{t} d\left(\mathbf{h}\_{i}^{t}, \mathbf{h}\_{j}^{t} ; \theta\right) $$ where $\alpha\_{i, j}^{t}$ denotes the distribution importance between the periods $\mathcal{D}\_{i}$ and $\mathcal{D}\_{j}$ at state $t$. All the hidden states of the RNN can be easily computed by following the standard RNN computation. Denote by $\delta(\cdot)$ the computation of a next hidden state based on a previous state. The state computation can be formulated as $$ \mathbf{h}\_{i}^{t}=\delta\left(\mathbf{x}\_{i}^{t}, \mathbf{h}\_{i}^{t-1}\right) $$ The final objective of temporal distribution matching (one RNN layer) is: $$ \mathcal{L}(\theta, \mathbf{\alpha})=\mathcal{L}\_{\text {pred }}(\theta)+\lambda \frac{2}{K(K-1)} \sum\_{i, j}^{i \neq j} \mathcal{L}\_{t d m}\left(\mathcal{D}\_{i}, \mathcal{D}\_{j} ; \theta, \mathbf{\alpha}\right) $$ where $\lambda$ is a trade-off hyper-parameter. Note that in the second term, we compute the average of the distribution distances of all pairwise periods. For computation, we take a mini-batch of $\mathcal{D}_{i}$ and $\mathcal{D}\_{j}$ to perform forward operation in RNN layers and concatenate all hidden features. Then, we can perform TDM using the above equation.