dopamine README

The project's README from github.com/google/dopamine. As one markdown file: README.md.

Dopamine

Getting Started | Docs | Baseline Results | Changelist



Dopamine is a research framework for fast prototyping of reinforcement learning algorithms. It aims to fill the need for a small, easily grokked codebase in which users can freely experiment with wild ideas (speculative research).

Our design principles are:

Dopamine supports the following agents, implemented with jax:

For more information on the available agents, see the docs.

Many of these agents also have a tensorflow (legacy) implementation, though newly added agents are likely to be jax-only.

This is not an official Google product.

Getting Started

We provide docker containers for using Dopamine. Instructions can be found here.

Alternatively, Dopamine can be installed from source (preferred) or installed with pip. For either of these methods, continue reading at prerequisites.

Prerequisites

Dopamine supports Atari environments and Mujoco environments. Install the environments you intend to use before you install Dopamine:

Atari

  1. These should now come packaged with ale_py.
  2. You may need to manually run some steps to properly install baselines, see instructions.

Mujoco

  1. Install Mujoco and get a license here.
  2. Run pip install mujoco-py (we recommend using a virtual environment).

Installing from Source

The most common way to use Dopamine is to install it from source and modify the source code directly:

git clone https://github.com/google/dopamine

After cloning, install dependencies:

pip install -r dopamine/requirements.txt

Dopamine supports tensorflow (legacy) and jax (actively maintained) agents. View the Tensorflow documentation for more information on installing tensorflow.

Note: We recommend using a virtual environment when working with Dopamine.

Installing with Pip

Note: We strongly recommend installing from source for most users.

Installing with pip is simple, but Dopamine is designed to be modified directly. We recommend installing from source for writing your own experiments.

pip install dopamine-rl

Running tests

You can test whether the installation was successful by running the following from the dopamine root directory.

export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_test

Next Steps

View the docs for more information on training agents.

We supply baselines for each Dopamine agent.

We also provide a set of Colaboratory notebooks which demonstrate how to use Dopamine.

References

Bellemare et al., The Arcade Learning Environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 2013.

Machado et al., Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents, Journal of Artificial Intelligence Research, 2018.

Hessel et al., Rainbow: Combining Improvements in Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2018.

Mnih et al., Human-level Control through Deep Reinforcement Learning. Nature, 2015.

Schaul et al., Prioritized Experience Replay. Proceedings of the International Conference on Learning Representations, 2016.

Haarnoja et al., Soft Actor-Critic Algorithms and Applications, arXiv preprint arXiv:1812.05905, 2018.

Schulman et al., Proximal Policy Optimization Algorithms.

Giving credit

If you use Dopamine in your work, we ask that you cite our white paper. Here is an example BibTeX entry:

@article{castro18dopamine,
  author    = {Pablo Samuel Castro and
               Subhodeep Moitra and
               Carles Gelada and
               Saurabh Kumar and
               Marc G. Bellemare},
  title     = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},
  year      = {2018},
  url       = {http://arxiv.org/abs/1812.06110},
  archivePrefix = {arXiv}
}

dopamine (google/dopamine)

Searching Database…

10.9K ·GitHub
Dopamine
这是什么东西?Dopamine 是一个用于快速原型开发强化学习算法的研究框架——代码库刻意保持精简、易于读懂,方便尝试各种探索性的研究想法。
所以它是一个需要我写代码调用的框架,而不是一个点开就能用的成品应用?没错——它是一个 Python 框架,设计上就是让人直接修改的:你从源码安装、修改代码、运行自己的实验。README 说明虽然 pip install 很简单,但这并不是它推荐的使用方式。
我怎么把它跑起来?克隆仓库并安装依赖,最好放在虚拟环境里——从源码安装是推荐的方式。也有 pip 包和现成的 Docker 容器可用,自带的测试可以验证安装是否成功。
安装之前我需要先准备什么?你打算使用的训练环境:Atari(现在随 ale-py 一起打包)和/或 Mujoco,后者需要许可证外加 mujoco-py。后端方面,你可以在 jax 和旧版的 tensorflow 实现之间选择。
安装上有什么坑需要我注意的吗?没有许可证的话 Mujoco 无法使用,Atari 环境的安装可能需要手动步骤才能正确装好 baselines。README 还强烈建议使用虚拟环境,并从源码安装而不是用 pip。
里面实现了哪些强化学习智能体?有六个经过实战检验的:DQN、C51、Rainbow、IQN、SAC 和 PPO,都是用 jax 实现的。其中不少也有旧版的 tensorflow 实现,不过新加入的智能体很可能只有 jax 版本。
我能复现论文里发表的基准测试结果吗?可以。实验设置专门遵循 Machado 等人(2018)的建议以便于复现,而且项目为每个内置的智能体都提供了基准结果。
有哪些材料能帮助新手快速上手?除了文档之外,项目还提供了一套 Colaboratory 笔记本,演示如何使用 Dopamine,另外还有基准结果可以用来对比你自己跑出的结果。
Dopamine 是 Google 的官方产品吗?不是——README 明确说明它不是 Google 的官方产品。不过它发布在 Google 的 GitHub 组织下,而且作者请求如果你在工作中用到它,请引用他们的白皮书。
What are similar projects?dqn-adaptive-cpu-scheduler, cleanrl, baselines, OpenMythos, FluxVLA, Motus2, AirSim, tianshou
Which topics is it in?rl, deep-reinforcement-learning, jax, ml, tensorflow, simulation, training, algorithms, google, reinforcement-learning, jupyter notebook, benchmark
Creator
google
Size
28.2 MB
Stars
10.9K
Updated
CommitMar 24Releasev2Sep 26
Age
8y
Commits · 90 Days