Continuous Control

OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. It includes environment such as Algorithmic, Atari, Box2D, Classic Control, MuJoCo, Robotics, and Toy Text. Source: https://github.com/openai/gym Image Source: https://medium.com/@tuzzer/cart-pole-balancing-with-q-learning-b54c6068d947

D4RL is a collection of environments for offline reinforcement learning. These environments include Maze2D, AntMaze, Adroit, Gym, Flow, FrankKitchen and CARLA. Source: https://sites.google.com/view/d4rl/home Image Source: https://sites.google.com/view/d4rl/home

Datasets drive vision progress, yet existing driving datasets are impoverished in terms of visual content and supported tasks to study multitask learning for autonomous driving. Researchers are usually constrained to study a small set of problems on one dataset, while real-world computer vision applications require performing tasks of various complexities. We construct BDD100K, the largest driving video dataset with 100K videos and 10 tasks to evaluate the exciting progress of image recognition algorithms on autonomous driving. The dataset possesses geographic, environmental, and weather diversity, which is useful for training models that are less likely to be surprised by new conditions. Based on this diverse dataset, we build a benchmark for heterogeneous multitask learning and study how to solve the tasks together. Our experiments show that special training strategies are needed for existing models to perform such heterogeneous tasks. BDD100K opens the door for future studies in this important venue. More detail is at the dataset home page.

The DeepMind Control Suite (DMCS) is a set of simulated continuous control environments with a standardized structure and interpretable rewards. The tasks are written and powered by the MuJoCo physics engine, making them easy to identify. Control Suite tasks include Pendulum, Acrobot, Cart-pole, Cart-k-pole, Ball in cup, Point-mass, Reacher, Finger, Hooper, Fish, Cheetah, Walker, Manipulator, Manipulator extra, Stacker, Swimmer, Humanoid, Humanoid_CMU and LQR. Source: Unsupervised Learning of Object Structure and Dynamics from Videos Image Source: https://arxiv.org/abs/1801.00690

Omniverse Isaac Gym is a GPU-based physics simulation platform developed by NVIDIA. This open-source toolkit implements various Reinforcement Learning benchmarks, simulating real-world robotic applications. (Image Credit: NVIDIA-Omniverse)

LANI is a 3D navigation environment and corpus, where an agent navigates between landmarks. Lani contains 27,965 crowd-sourced instructions for navigation in an open environment. Each datapoint includes an instruction, a human-annotated ground-truth demonstration trajectory, and an environment with various landmarks and lakes. The dataset train/dev/test split is 19,758/4,135/4,072. Each environment specification defines placement of 6–13 landmarks within a square grass field of size 50m×50m. Source: Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction Image Source: https://arxiv.org/pdf/1809.00786.pdf

MO-Gymnasium is an open source Python library for developing and comparing multi-objective reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. Essentially, the environments follow the standard Gymnasium API, but return vectorized rewards as numpy arrays.

The MoCapAct dataset contains training data and models for humanoid locomotion research. It consists of expert policies that are trained to track individual clip snippets and HDF5 files of noisy rollouts collected from each expert, including proprioceptive observations and actions.

PyBullet is an easy to use Python module for physics simulation, robotics and deep reinforcement learning based on the Bullet Physics SDK. With PyBullet you can load articulated bodies from URDF, SDF and other file formats. PyBullet provides forward dynamics simulation, inverse dynamics computation, forward and inverse kinematics and collision detection and ray intersection queries. Aside from physics simulation, PyBullet supports to rendering, with a CPU renderer and OpenGL visualization and support for virtual reality headsets.

RL Unplugged is suite of benchmarks for offline reinforcement learning. The RL Unplugged is designed around the following considerations: to facilitate ease of use, we provide the datasets with a unified API which makes it easy for the practitioner to work with all data in the suite once a general pipeline has been established. This is a dataset accompanying the paper RL Unplugged: Benchmarks for Offline Reinforcement Learning. In this suite of benchmarks, we try to focus on the following problems: High dimensional action spaces, for example the locomotion humanoid domains, we have 56 dimensional actions. High dimensional observations. Partial observability, observations have egocentric vision. Difficulty of exploration, using states of the art algorithms and imitation to generate data for difficult environments. Real world challenges.

The Udacity dataset is mainly composed of video frames taken from urban roads. It provides a total number of 404,916 video frames for training and 5,614 video frames for testing. This dataset is challenging due to severe lighting changes, sharp road curves and busy traffic. Source: Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks Image Source: https://www.researchgate.net/figure/Sample-from-the-Udacity-dataset-with-the-original-ground-truth-bounding-boxes-Note-that_fig3_345652980

A benchmark suite of continuous control tasks, including classic tasks like cart-pole swing-up, tasks with very high state and action dimensionality such as 3D humanoid locomotion, tasks with partial observations, and tasks with hierarchical structure. Source: Benchmarking Deep Reinforcement Learning for Continuous Control