FinRL README

The project's README from github.com/AI4Finance-Foundation/FinRL. As one markdown file: README.md.

image

FinRL: Financial Reinforcement Learning → FinRL-X

Downloads Downloads Join Discord Python 3.6 PyPI Documentation Status License X LinkedIn

[!IMPORTANT] FinRL-X is the next-generation evolution of FinRL, designed for AI-native, modular, and production-oriented quantitative trading.

  • This repository (FinRL) preserves the original end-to-end educational and research framework.
  • For the latest architecture, live trading deployment, and production-focused development, please use FinRL-X / FinRL-Trading.

FinRL® is widely recognized as the first open-source framework for financial reinforcement learning. This repository contains the original FinRL library for education, benchmarking, and research prototyping.

For the next-generation AI-native and production-oriented trading stack, please visit FinRL-X / FinRL-Trading.

FinRL Ecosystem Roadmap

Generation Positioning Target Users Repository Description
FinRL-Meta Market Environments Practitioners FinRL-Meta Gym-style financial market environments and benchmarks
FinRL Classic End-to-End Framework Learners, Developers, Researchers FinRL Original train-test-trade pipeline for financial reinforcement learning
ElegantRL Algorithm Layer Researchers and Experts ElegantRL Lightweight and elegant DRL algorithms
FinRL-X Next Generation / Production Professional traders, institutions, hedge funds FinRL-Trading AI-native modular infrastructure for deployment-aware quantitative trading

Recommended for new users: Start with FinRL-X / FinRL-Trading if you are building modern or production-oriented trading systems.

🔄 FinRL-X vs. FinRL: What Changed

Capability FinRL (Stage 1.0) FinRL-X (Stage 3.0)
Paradigm Deep Reinforcement Learning AI-Native (ML + DRL + LLM-ready)
Architecture Three-layer coupled monolith Fully decoupled modular layers
Strategies DRL agents (A2C, DDPG, PPO, SAC, TD3) ML selection + DRL timing + extensible base
Data Layer 14 manually-wired processors Auto-select: Yahoo Finance → FMP → WRDS
Backtesting Custom hand-rolled evaluation loops Professional bt library engine
Live Trading Basic Alpaca support Full multi-account integration + risk controls
Configuration config.py + config_tickers.py Type-safe Pydantic + .env multi-env
Risk Management Gym environment constraints only Order · portfolio · strategy-level controls
Target Users Researchers & students Quants, institutions, production deployments
Paper arXiv:2011.09607 arXiv:2603.21330

FinGPT: an open-source project for financial large language models, designed for research and real-world FinTech applications.

Visitors Discord

Outline

Project Contributors

FinRL® is an open-source financial reinforcement learning framework developed by contributors from the AI4Finance community and maintained by the AI4Finance Foundation.

Key contributors include:

Overview

FinRL is the original open-source framework for financial reinforcement learning, organized around three core layers:

For a trading task, an agent interacts with a market environment and learns sequential decision-making policies.

This repository focuses on the classic FinRL workflow for education, experimentation, and research prototyping.

For the next-generation production-oriented stack, including modular deployment and AI-native trading infrastructure, please visit FinRL-X / FinRL-Trading.

Videos FinRL at AI4Finance Youtube Channel.

FinRL Stock Trading 2026 Tutorial

This tutorial demonstrates the original FinRL workflow for educational and research purposes. For the latest production-oriented pipeline, please use FinRL-X / FinRL-Trading.

Step 1: Clone the Repository

git clone https://github.com/AI4Finance-Foundation/FinRL.git
cd FinRL

Step 2: Create and Activate Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3: Install FinRL

pip install -e .

Step 4: Run the Scripts

1. Data Download & Preprocessing

python examples/FinRL_StockTrading_2026_1_data.py

This script downloads DOW 30 stock data from Yahoo Finance, adds technical indicators (MACD, RSI, etc.), VIX, and turbulence index, then splits the data into training set (2014–2025) and trading set (2026-01-01 to 2026-03-20), saving them as train_data.csv and trade_data.csv.

2. Train DRL Agents

python examples/FinRL_StockTrading_2026_2_train.py

This script trains 5 DRL agents (A2C, DDPG, PPO, TD3, SAC) using Stable Baselines 3 on the training data. Trained models are saved to the trained_models/ directory.

3. Backtest

python examples/FinRL_StockTrading_2026_3_Backtest.py

This script loads the trained agents, runs them on the trading data, and compares their performance against two baselines: Mean Variance Optimization (MVO) and the DJIA index. Results are printed to the console and a plot is saved as backtest_result.png.

File Structure

The main folder finrl has three subfolders applications, agents, meta. We employ a train-test-trade pipeline with three files: train.py, test.py, and trade.py.

FinRL
├── finrl (main folder)
│   ├── applications
│   	├── Stock_NeurIPS2018
│   	├── imitation_learning
│   	├── cryptocurrency_trading
│   	├── high_frequency_trading
│   	├── portfolio_allocation
│   	└── stock_trading
│   ├── agents
│   	├── elegantrl
│   	├── rllib
│   	└── stablebaseline3
│   ├── meta
│   	├── data_processors
│   	├── env_cryptocurrency_trading
│   	├── env_portfolio_allocation
│   	├── env_stock_trading
│   	├── preprocessor
│   	├── data_processor.py
│       ├── meta_config_tickers.py
│   	└── meta_config.py
│   ├── config.py
│   ├── config_tickers.py
│   ├── main.py
│   ├── plot.py
│   ├── train.py
│   ├── test.py
│   └── trade.py
│
├── examples
├── unit_tests (unit tests to verify codes on env & data)
│   ├── environments
│   	└── test_env_cashpenalty.py
│   └── downloaders
│   	├── test_yahoodownload.py
│   	└── test_alpaca_downloader.py
├── setup.py
├── requirements.txt
└── README.md

Supported Data Sources

Data Source Type Range and Frequency Request Limits Raw Data Preprocessed Data
Akshare CN Securities 2015-now, 1day Account-specific OHLCV Prices&Indicators
Alpaca US Stocks, ETFs 2015-now, 1min Account-specific OHLCV Prices&Indicators
Baostock CN Securities 1990-12-19-now, 5min Account-specific OHLCV Prices&Indicators
Binance Cryptocurrency API-specific, 1s, 1min API-specific Tick-level daily aggregated trades, OHLCV Prices&Indicators
CCXT Cryptocurrency API-specific, 1min API-specific OHLCV Prices&Indicators
EODhistoricaldata US Securities Frequency-specific, 1min API-specific OHLCV Prices&Indicators
IEXCloud NMS US securities 1970-now, 1 day 100 per second per IP OHLCV Prices&Indicators
JoinQuant CN Securities 2005-now, 1min 3 requests each time OHLCV Prices&Indicators
QuantConnect US Securities 1998-now, 1s NA OHLCV Prices&Indicators
RiceQuant CN Securities 2005-now, 1ms Account-specific OHLCV Prices&Indicators
Sinopac Taiwan securities 2023-04-13~now, 1min Account-specific OHLCV Prices&Indicators
Tushare CN Securities, A-share -now, 1 min Account-specific OHLCV Prices&Indicators
WRDS US Securities 2003-now, 1ms 5 requests each time Intraday Trades Prices&Indicators
YahooFinance US Securities Frequency-specific, 1min 2,000/hour OHLCV Prices&Indicators

OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price

Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.

Installation

Status Update

Version History [click to expand]
  • 2022-06-25 0.3.5: Formal release of FinRL, neo_finrl is changed to FinRL-Meta with related files in directory: meta.
  • 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
  • 2020-12-14 Upgraded to Pytorch with stable-baselines3; Removed TensorFlow 1.x support; TensorFlow 2.0 support was under development at the time.
  • 2020-11-27 0.1: Beta version with tensorflow 1.5

Tutorials

Publications

Title Conference/Journal Link Citations Year
Dynamic Datasets and Market Environments for Financial Reinforcement Learning Machine Learning - Springer Nature paper code 51 2024
FinRL-Meta: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning NeurIPS 2022 paper code 136 2022
FinRL: Deep reinforcement learning framework to automate trading in quantitative finance ACM International Conference on AI in Finance (ICAIF) paper 212 2021
FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance NeurIPS 2020 Deep RL Workshop paper 275 2020
Deep reinforcement learning for automated stock trading: An ensemble strategy ACM International Conference on AI in Finance (ICAIF) paper code 426 2020
Practical deep reinforcement learning approach for stock trading NeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services paper code 303 2018

News

Citing FinRL

For the next-generation AI-native modular trading infrastructure, see FinRL-X / FinRL-Trading.

@inproceedings{yang2026finrlx,
  title     = {FinRL-X: An AI-Native Modular Infrastructure for Quantitative Trading},
  author    = {Yang, Hongy

FinRL (AI4Finance-Foundation/FinRL)

Searching Database…

16.7K ·GitHub
FinRL
这是什么？FinRL 是最初的开源金融强化学习框架，可以训练深度强化学习智能体（A2C、DDPG、PPO、TD3、SAC）来交易股票和其他资产。它围绕训练-测试-交易的流程构建，分为市场环境、DRL 智能体和金融应用三大部分。
这是一个现成可用的交易机器人，还是一个需要自己搭建的框架？它是一个框架：你需要通过类似 train.py、test.py、trade.py 的脚本，自己组装市场环境、DRL 智能体和金融应用，而不是开箱即用的机器人。
我该怎么让它跑起来？克隆仓库，创建并激活虚拟环境，安装 FinRL，然后运行提供的脚本来下载和预处理股票数据、训练五个 DRL 智能体，并对它们进行回测。
获取数据需要券商的 API 密钥吗？还是开箱即用？默认教程会从雅虎财经拉取道琼斯 30 指数的数据，不需要账户。但大多数其他支持的数据源，比如 Alpaca、Tushare、Binance 或 WRDS，都与具体账户绑定，需要各自的 API 密钥。
我应该用这个，还是用 FinRL-X？FinRL-X 是正在积极开发的、面向生产环境的后继项目，支持多账户实盘交易和风险控制。这个经典的 FinRL 仓库已经冻结，定位为最初的教学和研究框架，因此不要指望这里会有新的生产级功能。
它能在 Windows 上运行，和 Mac、Linux 一样吗？可以，README 提供了 macOS、Ubuntu 和 Windows 10 的安装说明。
它自带哪些强化学习算法？开箱即用地训练五个 DRL 智能体，基于 Stable Baselines 3 实现：A2C、DDPG、PPO、TD3 和 SAC。
FinRL、FinRL-Meta 和 FinRL-X 有什么区别？FinRL-Meta 提供类 gym 风格的市场环境和基准测试；FinRL（即本仓库）是经典的端到端训练-测试-交易框架，用于学习和研究；FinRL-X 是面向机构的下一代模块化、生产级技术栈。
回测是怎么评估的？回测脚本会让训练好的智能体在预留的交易数据上运行，并将其收益与均值方差优化（Mean Variance Optimization）以及道琼斯指数进行比较，同时保存一张结果图表。
除了美股，它还能交易哪些市场？它支持 14 个以上的数据源，涵盖美国、中国大陆和中国台湾的证券，以及通过 Binance 和 CCXT 获取的加密货币，每个数据源都提供 OHLCV 以及经过预处理的价格和指标数据。
What are similar projects?qlib, FinGPT, abu, Qbot, machine-learning-for-trading, anti-gambling-trader-tw, CNEquity, awesome-quant
Which topics is it in?tensorflow2, deep-reinforcement-learning, algorithmic-trading, stock-market, quantitative-finance, fintech, finance, reinforcement-learning, jupyter notebook, pytorch, benchmark, framework
Creator
AI4Finance-Foundation
Size
82.5 MB
Stars
16.7K
Updated
CommitOct 13Releasev0.3.8Mar 20
Age
6y
Commits · 90 Days