dopamine README

The project's README from github.com/google/dopamine. As one markdown file: README.md.

Dopamine

Getting Started | Docs | Baseline Results | Changelist



Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grokked codebase in which users can freely experiment with wild ideas (speculative research).

Our design principles are:

Dopamine supports the following agents, implemented with jax:

For more information on the available agents, see the docs.

Many of these agents also have a tensorflow (legacy) implementation, though newly added agents are likely to be jax-only.

This is not an official Google product.

Getting Started

We provide docker containers for using Dopamine. Instructions can be found here.

Alternatively, Dopamine can be installed from source (preferred) or installed with pip. For either of these methods, continue reading at prerequisites.

Prerequisites

Dopamine supports Atari environments and Mujoco environments. Install the environments you intend to use before you install Dopamine:

Atari

  1. These should now come packaged with ale_py.
  2. You may need to manually run some steps to properly install baselines, see instructions.

Mujoco

  1. Install Mujoco and get a license here.
  2. Run pip install mujoco-py (we recommend using a virtual environment).

Installing from Source

The most common way to use Dopamine is to install it from source and modify the source code directly:

git clone https://github.com/google/dopamine

After cloning, install dependencies:

pip install -r dopamine/requirements.txt

Dopamine supports tensorflow (legacy) and jax (actively maintained) agents. View the Tensorflow documentation for more information on installing tensorflow.

Note: We recommend using a virtual environment when working with Dopamine.

Installing with Pip

Note: We strongly recommend installing from source for most users.

Installing with pip is simple, but Dopamine is designed to be modified directly. We recommend installing from source for writing your own experiments.

pip install dopamine-rl

Running tests

You can test whether the installation was successful by running the following from the dopamine root directory.

export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_test

Next Steps

View the docs for more information on training agents.

We supply baselines for each Dopamine agent.

We also provide a set of Colaboratory notebooks which demonstrate how to use Dopamine.

References

Bellemare et al., The Arcade Learning Environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 2013.

Machado et al., Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents, Journal of Artificial Intelligence Research, 2018.

Hessel et al., Rainbow: Combining Improvements in Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2018.

Mnih et al., Human-level Control through Deep Reinforcement Learning. Nature, 2015.

Schaul et al., Prioritized Experience Replay. Proceedings of the International Conference on Learning Representations, 2016.

Haarnoja et al., Soft Actor-Critic Algorithms and Applications, arXiv preprint arXiv:1812.05905, 2018.

Schulman et al., Proximal Policy Optimization Algorithms.

Giving credit

If you use Dopamine in your work, we ask that you cite our white paper. Here is an example BibTeX entry:

@article{castro18dopamine,
  author    = {Pablo Samuel Castro and
               Subhodeep Moitra and
               Carles Gelada and
               Saurabh Kumar and
               Marc G. Bellemare},
  title     = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},
  year      = {2018},
  url       = {http://arxiv.org/abs/1812.06110},
  archivePrefix = {arXiv}
}

dopamine (google/dopamine)

Searching Database…

10.9K ·GitHub
Dopamine
What is this?Dopamine is a research framework for fast prototyping of reinforcement learning algorithms — a deliberately small, easily grokked codebase meant for trying out speculative research ideas.
So it's a framework I code against, not a finished app I just click through?Exactly — it's a Python framework designed to be modified directly: you install it from source, edit the code and run your own experiments. The README says the pip install is simple but not the intended way to work with it.
How do I get it running?Clone the repo and install the dependencies, ideally in a virtual environment — installing from source is the preferred route. A pip package and ready-made Docker containers also exist, and bundled tests verify the install worked.
What do I need to set up before installing?The training environments you plan to use: Atari (now packaged with ale-py) and/or Mujoco, which requires a license plus mujoco-py. On the backend side you pick between jax and the legacy tensorflow implementation.
Any install pitfalls I should watch out for?Mujoco won't work without a license, and the Atari setup may need manual steps to install baselines properly. The README also strongly recommends a virtual environment and installing from source rather than pip.
Which reinforcement learning agents are implemented?Six battle-tested ones: DQN, C51, Rainbow, IQN, SAC and PPO, implemented with jax. Many of them also have legacy tensorflow implementations, though newly added agents are likely to be jax-only.
Can I reproduce the published benchmark results?Yes. The experimental setup follows the recommendations of Machado et al. (2018) specifically to facilitate reproducibility, and the project supplies baseline results for every included agent.
What materials help a newcomer get up to speed?Besides the docs, the project provides a set of Colaboratory notebooks demonstrating how to use Dopamine, plus baseline results you can compare your own runs against.
Is Dopamine an official Google product?No — the README explicitly states it is not an official Google product. It is published under Google's GitHub org, though, and the authors ask that you cite their white paper if you use it in your work.
What are similar projects?dqn-adaptive-cpu-scheduler, cleanrl, baselines, OpenMythos, FluxVLA, Motus2, AirSim, tianshou
Which topics is it in?rl, deep-reinforcement-learning, jax, ml, tensorflow, simulation, training, algorithms, google, reinforcement-learning, jupyter notebook, benchmark
Creator
google
Size
28.2 MB
Stars
10.9K
Updated
CommitMar 24Releasev2Sep 26
Age
8y
Commits · 90 Days