FinRL README

The project's README from github.com/AI4Finance-Foundation/FinRL. As one markdown file: README.md.

image

FinRL: Financial Reinforcement Learning → FinRL-X

Downloads Downloads Join Discord Python 3.6 PyPI Documentation Status License X LinkedIn

[!IMPORTANT] FinRL-X is the next-generation evolution of FinRL, designed for AI-native, modular, and production-oriented quantitative trading.

  • This repository (FinRL) preserves the original end-to-end educational and research framework.
  • For the latest architecture, live trading deployment, and production-focused development, please use FinRL-X / FinRL-Trading.

FinRL® is widely recognized as the first open-source framework for financial reinforcement learning. This repository contains the original FinRL library for education, benchmarking, and research prototyping.

For the next-generation AI-native and production-oriented trading stack, please visit FinRL-X / FinRL-Trading.

FinRL Ecosystem Roadmap

Generation Positioning Target Users Repository Description
FinRL-Meta Market Environments Practitioners FinRL-Meta Gym-style financial market environments and benchmarks
FinRL Classic End-to-End Framework Learners, Developers, Researchers FinRL Original train-test-trade pipeline for financial reinforcement learning
ElegantRL Algorithm Layer Researchers and Experts ElegantRL Lightweight and elegant DRL algorithms
FinRL-X Next Generation / Production Professional traders, institutions, hedge funds FinRL-Trading AI-native modular infrastructure for deployment-aware quantitative trading

Recommended for new users: Start with FinRL-X / FinRL-Trading if you are building modern or production-oriented trading systems.

🔄 FinRL-X vs. FinRL: What Changed

Capability FinRL (Stage 1.0) FinRL-X (Stage 3.0)
Paradigm Deep Reinforcement Learning AI-Native (ML + DRL + LLM-ready)
Architecture Three-layer coupled monolith Fully decoupled modular layers
Strategies DRL agents (A2C, DDPG, PPO, SAC, TD3) ML selection + DRL timing + extensible base
Data Layer 14 manually-wired processors Auto-select: Yahoo Finance → FMP → WRDS
Backtesting Custom hand-rolled evaluation loops Professional bt library engine
Live Trading Basic Alpaca support Full multi-account integration + risk controls
Configuration config.py + config_tickers.py Type-safe Pydantic + .env multi-env
Risk Management Gym environment constraints only Order · portfolio · strategy-level controls
Target Users Researchers & students Quants, institutions, production deployments
Paper arXiv:2011.09607 arXiv:2603.21330

FinGPT: an open-source project for financial large language models, designed for research and real-world FinTech applications.

Visitors Discord

Outline

Project Contributors

FinRL® is an open-source financial reinforcement learning framework developed by contributors from the AI4Finance community and maintained by the AI4Finance Foundation.

Key contributors include:

Overview

FinRL is the original open-source framework for financial reinforcement learning, organized around three core layers:

For a trading task, an agent interacts with a market environment and learns sequential decision-making policies.

This repository focuses on the classic FinRL workflow for education, experimentation, and research prototyping.

For the next-generation production-oriented stack, including modular deployment and AI-native trading infrastructure, please visit FinRL-X / FinRL-Trading.

Videos FinRL at AI4Finance Youtube Channel.

FinRL Stock Trading 2026 Tutorial

This tutorial demonstrates the original FinRL workflow for educational and research purposes. For the latest production-oriented pipeline, please use FinRL-X / FinRL-Trading.

Step 1: Clone the Repository

git clone https://github.com/AI4Finance-Foundation/FinRL.git
cd FinRL

Step 2: Create and Activate Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3: Install FinRL

pip install -e .

Step 4: Run the Scripts

1. Data Download & Preprocessing

python examples/FinRL_StockTrading_2026_1_data.py

This script downloads DOW 30 stock data from Yahoo Finance, adds technical indicators (MACD, RSI, etc.), VIX, and turbulence index, then splits the data into training set (2014–2025) and trading set (2026-01-01 to 2026-03-20), saving them as train_data.csv and trade_data.csv.

2. Train DRL Agents

python examples/FinRL_StockTrading_2026_2_train.py

This script trains 5 DRL agents (A2C, DDPG, PPO, TD3, SAC) using Stable Baselines 3 on the training data. Trained models are saved to the trained_models/ directory.

3. Backtest

python examples/FinRL_StockTrading_2026_3_Backtest.py

This script loads the trained agents, runs them on the trading data, and compares their performance against two baselines: Mean Variance Optimization (MVO) and the DJIA index. Results are printed to the console and a plot is saved as backtest_result.png.

File Structure

The main folder finrl has three subfolders applications, agents, meta. We employ a train-test-trade pipeline with three files: train.py, test.py, and trade.py.

FinRL
├── finrl (main folder)
│   ├── applications
│   	├── Stock_NeurIPS2018
│   	├── imitation_learning
│   	├── cryptocurrency_trading
│   	├── high_frequency_trading
│   	├── portfolio_allocation
│   	└── stock_trading
│   ├── agents
│   	├── elegantrl
│   	├── rllib
│   	└── stablebaseline3
│   ├── meta
│   	├── data_processors
│   	├── env_cryptocurrency_trading
│   	├── env_portfolio_allocation
│   	├── env_stock_trading
│   	├── preprocessor
│   	├── data_processor.py
│       ├── meta_config_tickers.py
│   	└── meta_config.py
│   ├── config.py
│   ├── config_tickers.py
│   ├── main.py
│   ├── plot.py
│   ├── train.py
│   ├── test.py
│   └── trade.py
│
├── examples
├── unit_tests (unit tests to verify codes on env & data)
│   ├── environments
│   	└── test_env_cashpenalty.py
│   └── downloaders
│   	├── test_yahoodownload.py
│   	└── test_alpaca_downloader.py
├── setup.py
├── requirements.txt
└── README.md

Supported Data Sources

Data Source Type Range and Frequency Request Limits Raw Data Preprocessed Data
Akshare CN Securities 2015-now, 1day Account-specific OHLCV Prices&Indicators
Alpaca US Stocks, ETFs 2015-now, 1min Account-specific OHLCV Prices&Indicators
Baostock CN Securities 1990-12-19-now, 5min Account-specific OHLCV Prices&Indicators
Binance Cryptocurrency API-specific, 1s, 1min API-specific Tick-level daily aggregated trades, OHLCV Prices&Indicators
CCXT Cryptocurrency API-specific, 1min API-specific OHLCV Prices&Indicators
EODhistoricaldata US Securities Frequency-specific, 1min API-specific OHLCV Prices&Indicators
IEXCloud NMS US securities 1970-now, 1 day 100 per second per IP OHLCV Prices&Indicators
JoinQuant CN Securities 2005-now, 1min 3 requests each time OHLCV Prices&Indicators
QuantConnect US Securities 1998-now, 1s NA OHLCV Prices&Indicators
RiceQuant CN Securities 2005-now, 1ms Account-specific OHLCV Prices&Indicators
Sinopac Taiwan securities 2023-04-13~now, 1min Account-specific OHLCV Prices&Indicators
Tushare CN Securities, A-share -now, 1 min Account-specific OHLCV Prices&Indicators
WRDS US Securities 2003-now, 1ms 5 requests each time Intraday Trades Prices&Indicators
YahooFinance US Securities Frequency-specific, 1min 2,000/hour OHLCV Prices&Indicators

OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price

Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.

Installation

Status Update

Version History [click to expand]
  • 2022-06-25 0.3.5: Formal release of FinRL, neo_finrl is changed to FinRL-Meta with related files in directory: meta.
  • 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
  • 2020-12-14 Upgraded to Pytorch with stable-baselines3; Removed TensorFlow 1.x support; TensorFlow 2.0 support was under development at the time.
  • 2020-11-27 0.1: Beta version with tensorflow 1.5

Tutorials

Publications

Title Conference/Journal Link Citations Year
Dynamic Datasets and Market Environments for Financial Reinforcement Learning Machine Learning - Springer Nature paper code 51 2024
FinRL-Meta: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning NeurIPS 2022 paper code 136 2022
FinRL: Deep reinforcement learning framework to automate trading in quantitative finance ACM International Conference on AI in Finance (ICAIF) paper 212 2021
FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance NeurIPS 2020 Deep RL Workshop paper 275 2020
Deep reinforcement learning for automated stock trading: An ensemble strategy ACM International Conference on AI in Finance (ICAIF) paper code 426 2020
Practical deep reinforcement learning approach for stock trading NeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services paper code 303 2018

News

Citing FinRL

For the next-generation AI-native modular trading infrastructure, see FinRL-X / FinRL-Trading.

@inproceedings{yang2026finrlx,
  title     = {FinRL-X: An AI-Native Modular Infrastructure for Quantitative Trading},
  author    = {Yang, Hongy

FinRL (AI4Finance-Foundation/FinRL)

Searching Database…

16.7K ·GitHub
What is this?FinRL is the original open-source framework for financial reinforcement learning, letting you train deep RL agents (A2C, DDPG, PPO, TD3, SAC) to trade stocks and other assets. It's built around a train-test-trade pipeline split into market environments, DRL agents, and financial applications.
Is this a ready-to-run trading bot, or a framework I build on?It's a framework: you assemble market environments, DRL agents, and financial applications yourself through train.py, test.py and trade.py style scripts, rather than getting a plug-and-play bot.
How do I get it running?Clone the repository, create and activate a virtual environment, install FinRL, then run the provided scripts to download/preprocess stock data, train the five DRL agents, and backtest them.
Do I need broker API keys for data, or does it work out of the box?The default tutorial pulls DOW 30 data from Yahoo Finance with no account needed, but most other supported sources like Alpaca, Tushare, Binance, or WRDS are account-specific and require their own API keys.
Should I use this or FinRL-X?FinRL-X is the actively developed, production-oriented successor with live multi-account trading and risk controls; this classic FinRL repo is frozen as the original educational and research framework, so don't expect new production features here.
Does it run on Windows as well as Mac and Linux?Yes, the README provides install instructions for macOS, Ubuntu, and Windows 10.
Which RL algorithms come with it?It trains five DRL agents out of the box using Stable Baselines 3: A2C, DDPG, PPO, TD3, and SAC.
What's the difference between FinRL, FinRL-Meta, and FinRL-X?FinRL-Meta supplies the gym-style market environments and benchmarks, FinRL (this repo) is the classic end-to-end train-test-trade framework for learning and research, and FinRL-X is the next-gen, modular, production-oriented stack for institutions.
How is backtesting evaluated?The backtest script runs the trained agents on held-out trading data and compares their returns against Mean Variance Optimization and the DJIA index, saving a result plot.
What markets can it trade beyond US stocks?It supports 14+ data sources covering US, CN, and Taiwan securities as well as cryptocurrencies through Binance and CCXT, each delivering OHLCV and preprocessed price/indicator data.
Size
82.5 MB
Stars
16.7K
Age
6y
Commits · 90 Days