Learning decent representations from unlabeled time-series data with temporal dynamics is a very challenging task. In this paper, we propose an unsupervised Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC), to learn time-series representation from unlabeled data. First, the raw time-series data are transformed into two different yet correlated views by using weak and strong augmentations. Second, we propose a novel temporal contrasting module to learn robust temporal representations by designing a tough cross-view prediction task. Last, to further learn discriminative representations, we propose a contextual contrasting module built upon the contexts from the temporal contrasting module. It attempts to maximize the similarity among different contexts of the same sample while minimizing similarity among contexts of different samples. Experiments have been carried out on three real-world time-series datasets. The results manifest that training a linear classifier on top of the features learned by our proposed TS-TCC performs comparably with the supervised training. Additionally, our proposed TS-TCC shows high efficiency in few-labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/TS-TCC.
In this paper, we investigate the performance of several sequence prediction techniques on the prediction of future events of human behavior in a smart home, as well as the timestamps of those next events. Prediction techniques in smart home environments have several use cases, such as the real-time identification of abnormal behavior, identifying coachable moments for e-coaching, and a plethora of applications in the area of home automation. We give an overview of several sequence prediction techniques, including techniques that originate from the areas of data mining, process mining, and data compression, and we evaluate the predictive accuracy of those techniques on a collection of publicly available real-life datasets from the smart home environments domain. This contrast our work with existing work on prediction in smart homes, which often evaluate their techniques on a single smart home instead of a larger collection of logs. We found that LSTM neural networks outperform the other prediction methods on the task of predicting the next activity as well as on the task of predicting the timestamp of the next event. However, surprisingly, we found that it is very dependent on the dataset which technique works best for the task of predicting a window of multiple next activities.
We introduce the Action Transformer model for recognizing and localizing human actions in video clips. We repurpose a Transformer-style architecture to aggregate features from the spatiotemporal context around the person whose actions we are trying to classify. We show that by using high-resolution, person-specific, class-agnostic queries, the model spontaneously learns to track individual people and to pick up on semantic context from the actions of others. Additionally its attention mechanism learns to emphasize hands and faces, which are often crucial to discriminate an action - all without explicit supervision other than boxes and class labels. We train and test our Action Transformer network on the Atomic Visual Actions (AVA) dataset, outperforming the state-of-the-art by a significant margin using only raw RGB frames as input.