Video Games

The Arcade Learning Environment (ALE) is an object-oriented framework that allows researchers to develop AI agents for Atari 2600 games. It is built on top of the Atari 2600 emulator Stella and separates the details of emulation from agent design. Source: https://github.com/mgbellemare/Arcade-Learning-Environment Image Source: https://github.com/muupan/async-rl/blob/master/README.md

ViZDoom is an AI research platform based on the classical First Person Shooter game Doom. The most popular game mode is probably the so-called Death Match, where several players join in a maze and fight against each other. After a fixed time, the match ends and all the players are ranked by the FRAG scores defined as kills minus suicides. During the game, each player can access various observations, including the first-person view screen pixels, the corresponding depth-map and segmentation-map (pixel-wise object labels), the bird-view maze map, etc. The valid actions include almost all the keyboard-stroke and mouse-control a human player can take, accounting for moving, turning, jumping, shooting, changing weapon, etc. ViZDoom can run a game either synchronously or asynchronously, indicating whether the game core waits until all players’ actions are collected or runs in a constant frame rate without waiting. Source: Arena: a toolkit for Multi-Agent Reinforcement Learning Image Source: https://github.com/mwydmuch/ViZDoom

The DQN Replay Dataset was collected as follows: We first train a [DQN][nature_dqn] agent, on all 60 [Atari 2600 games][ale] with [sticky actions][stochastic_ale] enabled for 200 million frames (standard protocol) and save all of the experience tuples of (observation, action, reward, next observation) (approximately 50 million) encountered during training. This logged DQN data can be found in the public [GCP bucket][gcp_bucket] gs://atari-replay-datasets which can be downloaded using [gsutil][gsutil]. To install gsutil, follow the instructions [here][gsutil_install]. After installing gsutil, run the command to copy the entire dataset: gsutil -m cp -R gs://atari-replay-datasets/dqn To run the dataset only for a specific Atari 2600 game (e.g., replace GAME_NAME by Pong to download the logged DQN replay datasets for the game of Pong), run the command: gsutil -m cp -R gs://atari-replay-datasets/dqn/[GAME_NAME] This data can be generated by running the online agents using batch_rl/baselines/train.py for 200 million frames (standard protocol). Note that the dataset consists of approximately 50 million experience tuples due to frame skipping (i.e., repeating a selected action for k consecutive frames) of 4. The stickiness parameter is set to 0.25, i.e., there is 25% chance at every time step that the environment will execute the agent's previous action again, instead of the agent's new action. [nature_dqn]: https://www.nature.com/articles/nature14236?wm=book_wap_0005 [gsutil_install]: https://cloud.google.com/storage/docs/gsutil_install#install [gsutil]: https://cloud.google.com/storage/docs/gsutil [batch_rl]: http://tgabel.de/cms/fileadmin/user_upload/documents/Lange_Gabel_EtAl_RL-Book-12.pdf [stochastic_ale]: https://arxiv.org/abs/1709.06009 [ale]: https://github.com/mgbellemare/Arcade-Learning-Environment [gcp_bucket]: https://console.cloud.google.com/storage/browser/atari-replay-datasets [project_page]: https://offline-rl.github.io

The StarCraft II Learning Environment (S2LE) is a reinforcement learning environment based on the game StarCraft II. The environment consists of three sub-components: a Linux StarCraft II binary, the StarCraft II API and PySC2. The StarCraft II API allows programmatic control of StarCraft II. It can be used to start a game, get observations, take actions, and review replays. PyC2 is a Python environment that wraps the StarCraft II API to ease the interaction between Python reinforcement learning agents and StarCraft II. It defines an action and observation specification, and includes a random agent and a handful of rule-based agents as examples. It also includes some mini-games as challenges and visualization tools to understand what the agent can see and do. Source: https://github.com/deepmind/pysc2 Image Source: https://github.com/deepmind/pysc2

LANI is a 3D navigation environment and corpus, where an agent navigates between landmarks. Lani contains 27,965 crowd-sourced instructions for navigation in an open environment. Each datapoint includes an instruction, a human-annotated ground-truth demonstration trajectory, and an environment with various landmarks and lakes. The dataset train/dev/test split is 19,758/4,135/4,072. Each environment specification defines placement of 6–13 landmarks within a square grass field of size 50m×50m. Source: Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction Image Source: https://arxiv.org/pdf/1809.00786.pdf

Mario AI was a benchmark environment for reinforcement learning. The gameplay in Mario AI, as in the original Nintendo’s version, consists in moving the controlled character, namely Mario, through two-dimensional levels, which are viewed sideways. Mario can walk and run to the right and left, jump, and (depending on which state he is in) shoot fireballs. Gravity acts on Mario, making it necessary to jump over cliffs to get past them. Mario can be in one of three states: Small, Big (can kill enemies by jumping onto them), and Fire (can shoot fireballs). Source: https://github.com/zerg000000/mario-ai Image Source: http://julian.togelius.com/Karakovskiy2012The.pdf

The StarCraft Multi-Agent Challenges+ requires agents to learn completion of multi-stage tasks and usage of environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. This challenge, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack.

Atari-HEAD is a dataset of human actions and eye movements recorded while playing Atari videos games. For every game frame, its corresponding image frame, the human keystroke action, the reaction time to make that action, the gaze positions, and immediate reward returned by the environment were recorded. The gaze data was recorded using an EyeLink 1000 eye tracker at 1000Hz. The human subjects are amateur players who are familiar with the games. The human subjects were only allowed to play for 15 minutes and were required to rest for at least 15 minutes before the next trial. Data was collected from 4 subjects, 16 games, 175 15-minute trials, and a total of 2.97 million frames/demonstrations. Source: https://zenodo.org/record/2587121 Image Source: https://arxiv.org/abs/1903.06754

The Atari Grand Challenge dataset is a large dataset of human Atari 2600 replays. It consists of replays for 5 different games: * Space Invaders (445 episodes, 2M frames) * Q*bert (659 episodes, 1.6M frames) * Ms.Pacman (384 episodes, 1.7M frames) * Video Pinball (211 episodes, 1.5M frames) * Montezuma’s revenge (668 episodes, 2.7M frames) Source: https://arxiv.org/pdf/1705.10998.pdf Image Source: https://arxiv.org/pdf/1705.10998.pdf

The AtariARI (Atari Annotated RAM Interface) is an environment for representation learning. The Atari Arcade Learning Environment (ALE) does not explicitly expose any ground truth state information. However, ALE does expose the RAM state (128 bytes per timestep) which are used by the game programmer to store important state information such as the location of sprites, the state of the clock, or the current room the agent is in. To extract these variables, the dataset creators consulted commented disassemblies (or source code) of Atari 2600 games which were made available by Engelhardt and Jentzsch and CPUWIZ. The dataset creators were able to find and verify important state variables for a total of 22 games. Once this information was acquired, combining it with the ALE interface produced a wrapper that can automatically output a state label for every example frame generated from the game. The dataset creators make this available with an easy-to-use gym wrapper, which returns this information with no change to existing code using gym interfaces. Source: https://arxiv.org/pdf/1906.08226.pdf Image Source: https://github.com/mila-iqia/atari-representation-learning

The eSports Sensors dataset contains sensor data collected from 10 players in 22 matches in League of Legends. The sensor data collected includes: * Hand/head/chair movements. * Heart rate. * Muscle activity. * Gaze movement on the monitor. * Galvanic skin response(GSR). * Electroencephalography (EEG). * Mouse and keyboard activity. * Facial skin temperature. * Environmental data. The data were collected for one team of 5 people simultaneously. In-game logs and meta information for each match are also provided for each match. Source: https://github.com/smerdov/eSports_Sensors_Dataset

MSC is a dataset for Macro-Management in StarCraft 2 based on the platfrom SC2LE. It consists of well-designed feature vectors, pre-defined high-level actions and final result of each match. It contains 36,619 high quality replays, which are unbroken and played by relatively professional players. Source: https://github.com/wuhuikai/MSC

RL Unplugged is suite of benchmarks for offline reinforcement learning. The RL Unplugged is designed around the following considerations: to facilitate ease of use, we provide the datasets with a unified API which makes it easy for the practitioner to work with all data in the suite once a general pipeline has been established. This is a dataset accompanying the paper RL Unplugged: Benchmarks for Offline Reinforcement Learning. In this suite of benchmarks, we try to focus on the following problems: High dimensional action spaces, for example the locomotion humanoid domains, we have 56 dimensional actions. High dimensional observations. Partial observability, observations have egocentric vision. Difficulty of exploration, using states of the art algorithms and imitation to generate data for difficult environments. Real world challenges.

StarData is a StarCraft: Brood War replay dataset, with 65,646 games. The full dataset after compression is 365 GB, 1535 million frames, and 496 million player actions. The entire frame data was dumped out at 8 frames per second. Source: https://github.com/TorchCraft/StarData Image Source: https://www.youtube.com/watch?app=desktop&v=vBjgww8jDgw

To evaluate our proposed strategy of asynchronous communication for LLMs, we run games of Mafia with human players, incorporating an LLM-based agent as an additional player, within an asynchronous chat environment. The data includes all players messages and votes including timestamps, game management messages (e.g., announcements of the beginning and end of phases), in addition to records related to the agent, such as the prompts that were provided at each timestamp. While prior datasets for Mafia exist, ours is the first to integrate an LLM agent, allowing analysis of human-LLM interactions. Detailed analysis of the data is available in our paper: Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia Games

SC2EGSet: StarCraft II Esport Game State Dataset Pre-processed data that was generated from the SC2ReSet: StarCraft II Esports Replaypack Set Data Modeling Our aplication programing interface (API) implementation supports downloading, unpacking, loading and data access features. Please refer to: https://github.com/Kaszanas/SC2EGSet_Dataset License Information This dataset is licensed under the following license: Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)

Raw StarCraft II data is subject to processing under the Blizzard end user license agreement (EULA), and in special cases Blizzard AI and Machine Learning License may be applied. Please refer to the materials listed below. The dataset contains data in MPQ format (.SC2Replay) that can be processed with multiple open-source libraries. 1. https://www.blizzard.com/en-us/legal/fba4d00f-c7e4-4883-b8b9-1b4500a402ea/blizzard-end-user-license-agreement 2. https://blzdistsc2-a.akamaihd.net/AI_AND_MACHINE_LEARNING_LICENSE.html