Atari Games

The Arcade Learning Environment (ALE) is an object-oriented framework that allows researchers to develop AI agents for Atari 2600 games. It is built on top of the Atari 2600 emulator Stella and separates the details of emulation from agent design. Source: https://github.com/mgbellemare/Arcade-Learning-Environment Image Source: https://github.com/muupan/async-rl/blob/master/README.md

The DQN Replay Dataset was collected as follows: We first train a [DQN][nature_dqn] agent, on all 60 [Atari 2600 games][ale] with [sticky actions][stochastic_ale] enabled for 200 million frames (standard protocol) and save all of the experience tuples of (observation, action, reward, next observation) (approximately 50 million) encountered during training. This logged DQN data can be found in the public [GCP bucket][gcp_bucket] gs://atari-replay-datasets which can be downloaded using [gsutil][gsutil]. To install gsutil, follow the instructions [here][gsutil_install]. After installing gsutil, run the command to copy the entire dataset: gsutil -m cp -R gs://atari-replay-datasets/dqn To run the dataset only for a specific Atari 2600 game (e.g., replace GAME_NAME by Pong to download the logged DQN replay datasets for the game of Pong), run the command: gsutil -m cp -R gs://atari-replay-datasets/dqn/[GAME_NAME] This data can be generated by running the online agents using batch_rl/baselines/train.py for 200 million frames (standard protocol). Note that the dataset consists of approximately 50 million experience tuples due to frame skipping (i.e., repeating a selected action for k consecutive frames) of 4. The stickiness parameter is set to 0.25, i.e., there is 25% chance at every time step that the environment will execute the agent's previous action again, instead of the agent's new action. [nature_dqn]: https://www.nature.com/articles/nature14236?wm=book_wap_0005 [gsutil_install]: https://cloud.google.com/storage/docs/gsutil_install#install [gsutil]: https://cloud.google.com/storage/docs/gsutil [batch_rl]: http://tgabel.de/cms/fileadmin/user_upload/documents/Lange_Gabel_EtAl_RL-Book-12.pdf [stochastic_ale]: https://arxiv.org/abs/1709.06009 [ale]: https://github.com/mgbellemare/Arcade-Learning-Environment [gcp_bucket]: https://console.cloud.google.com/storage/browser/atari-replay-datasets [project_page]: https://offline-rl.github.io

Atari-HEAD is a dataset of human actions and eye movements recorded while playing Atari videos games. For every game frame, its corresponding image frame, the human keystroke action, the reaction time to make that action, the gaze positions, and immediate reward returned by the environment were recorded. The gaze data was recorded using an EyeLink 1000 eye tracker at 1000Hz. The human subjects are amateur players who are familiar with the games. The human subjects were only allowed to play for 15 minutes and were required to rest for at least 15 minutes before the next trial. Data was collected from 4 subjects, 16 games, 175 15-minute trials, and a total of 2.97 million frames/demonstrations. Source: https://zenodo.org/record/2587121 Image Source: https://arxiv.org/abs/1903.06754

The Atari Grand Challenge dataset is a large dataset of human Atari 2600 replays. It consists of replays for 5 different games: * Space Invaders (445 episodes, 2M frames) * Q*bert (659 episodes, 1.6M frames) * Ms.Pacman (384 episodes, 1.7M frames) * Video Pinball (211 episodes, 1.5M frames) * Montezuma’s revenge (668 episodes, 2.7M frames) Source: https://arxiv.org/pdf/1705.10998.pdf Image Source: https://arxiv.org/pdf/1705.10998.pdf

The AtariARI (Atari Annotated RAM Interface) is an environment for representation learning. The Atari Arcade Learning Environment (ALE) does not explicitly expose any ground truth state information. However, ALE does expose the RAM state (128 bytes per timestep) which are used by the game programmer to store important state information such as the location of sprites, the state of the clock, or the current room the agent is in. To extract these variables, the dataset creators consulted commented disassemblies (or source code) of Atari 2600 games which were made available by Engelhardt and Jentzsch and CPUWIZ. The dataset creators were able to find and verify important state variables for a total of 22 games. Once this information was acquired, combining it with the ALE interface produced a wrapper that can automatically output a state label for every example frame generated from the game. The dataset creators make this available with an easy-to-use gym wrapper, which returns this information with no change to existing code using gym interfaces. Source: https://arxiv.org/pdf/1906.08226.pdf Image Source: https://github.com/mila-iqia/atari-representation-learning

RL Unplugged is suite of benchmarks for offline reinforcement learning. The RL Unplugged is designed around the following considerations: to facilitate ease of use, we provide the datasets with a unified API which makes it easy for the practitioner to work with all data in the suite once a general pipeline has been established. This is a dataset accompanying the paper RL Unplugged: Benchmarks for Offline Reinforcement Learning. In this suite of benchmarks, we try to focus on the following problems: High dimensional action spaces, for example the locomotion humanoid domains, we have 56 dimensional actions. High dimensional observations. Partial observability, observations have egocentric vision. Difficulty of exploration, using states of the art algorithms and imitation to generate data for difficult environments. Real world challenges.