openlake README

项目的 README,来自 github.com/openlake-project/openlake。合并为一个 markdown 文件:README.md。

OpenLake OpenLake

Fast, easy and efficient storage for LLM Inference and Training

| Blog | Documentation | Comparision | Discord/X | Website |

License Rust Discord Web

🔥 Distributed storage for GPU workloads. Built on Rust on io_uring, OpenLake is a state of the art storage engine delivering million+ iops within 1ms.


Updates

Older updates
  • [2026/07] Introducing OpenLake: open source storage that saturates the GPUs: 8× throughput and 600 µs reads (blog).

Why OpenLake?

OpenLake is a storage engine for AI infrastructure. With OpenLake you get high throughput for small I/O and cache like performance while being fully persistent and durable. Keep GPUs fed during training and inference reducing idle time and getting more from your accelerators.

OpenLake is fast with:

Learn more: Blogs → | Benchmarks → | KV Offload | Object Store


66× speedup on time to first token first token when cached. (128K context window)

GPU nodes contribute to create an OpenLake cluster. The inference engine writes KV once and reads it back in milliseconds (using the host RAM and disk), saving prefill for long and repeated prompts.

Quickstart:

1) Setup KV Pool on GPU nodes:

Drop OpenLake into your existing setup. No code changes:

a. Install the connector and start the store:

pip install openlake-vllm
openlaked

b. Run vLLM with OpenLake enabled:

export PYTHONHASHSEED=0
vllm serve <model_name> --kv-transfer-config '{"kv_connector":"OpenLakeConnector","kv_connector_module_path":"openlake_client.openlake_connector","kv_role":"kv_both","kv_connector_extra_config":{"openlake_nodes":["127.0.0.1:9400"],"openlake_device":"local"}}'

Note: By default OpenLake offloads to the same host. To enable OpenLake across your GPU fleet, please start openlaked with a --config.

For Kubernetes clusters, use the Helm KV deployment guide to place one OpenLake instance on each selected node and generate the ordered vLLM peer configuration.

OpenLake enabled vs disabled:

openlake-video

OpenLake and vLLM serving Gemma4-31B on H100 (256K context window)

Multi host GPU Cluster (IB)

Run OpenLake on existing GPU cluster. Run OpenLake with (kv_rdma.toml) and node's self_id: (0, 1, 2...):

openlaked --config kv_rdma_0.toml   # gpu 1: ids = 0
openlaked --config kv_rdma_1.toml   # gpu 2: ids = 1

Point your vLLM workers at the unified cluster, in id order:

cat > /tmp/openlake-kv.json <<'EOF'
{
  "kv_connector": "OpenLakeConnector",
  "kv_connector_module_path": "openlake_client.openlake_connector",
  "kv_role": "kv_both",
  "kv_connector_extra_config": {
    "openlake_nodes": ["10.0.0.1:9400", "10.0.0.2:9400"],
    "openlake_device": "mlx5_ib0"
  }
}
EOF

export PYTHONHASHSEED=0
vllm serve <model_name> --kv-transfer-config "$(cat /tmp/openlake-kv.json)"

A prefix computed on one GPU host is served to any other from the shared pool.

2) PB scale object store for GPU Fleet

Build from source and have an S3 compatible store running in four steps. Install the dependencies and build the OpenLake binary locally.

a. Clone and build:

sudo apt-get install -y build-essential pkg-config clang cmake libhwloc-dev libudev-dev curl git awscli
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

git clone https://github.com/openlake-project/openlake.git && cd openlake
cargo build --release --bin openlaked

b. Start the store (single node, default config):

mkdir -p data/d0 data/d1 data/d2 data/d3
./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
Talk to it with any S3 client:
export AWS_ACCESS_KEY_ID=openlakeadmin
export AWS_SECRET_ACCESS_KEY=openlakeadmin
export AWS_DEFAULT_REGION=us-east-1

aws --endpoint-url http://127.0.0.1:9000 s3 mb s3://demo
aws --endpoint-url http://127.0.0.1:9000 s3 cp ./checkpoint.safetensors s3://demo/
aws --endpoint-url http://127.0.0.1:9000 s3 ls s3://demo/

Both ListObjects versions support encoding-type=url for UTF-8 percent-encoded keys, prefixes, and delimiters in the XML response. Without this option, key names retain their existing XML representation. ListObjectsV2 now echoes StartAfter and returns XML-safe, versioned continuation tokens; treat tokens as opaque and pass them back unchanged, even when changing the encoding option. Legacy raw-key tokens remain accepted except for the reserved openlake-list-v1: token prefix (restart those listings after upgrading).

Build from source:

To build from the source, please follow the platform specific build guides:

Ubuntu / Debian

Produce binaries for your deployment or test code changes.

# Install system dependencies
sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates build-essential pkg-config clang cmake libhwloc-dev libudev-dev curl git

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

# Clone and build OpenLake
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --release --locked -p openlake_server --bin openlaked

# Create the local storage directories
mkdir -p data/d0 data/d1 data/d2 data/d3

# Start OpenLake in TCP mode. Please switch the config path for RDMA.
RUST_LOG=info ./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
macOS (development)

Install Homebrew first if brew is unavailable.

# Install system dependencies
xcode-select --install
brew install cmake pkg-config

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

# Clone and build OpenLake
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --release --locked -p openlake_server --bin openlaked

# Create local storage directories
mkdir -p data/d0 data/d1 data/d2 data/d3

# Start OpenLake in TCP mode. (macOS does not provide the Linux RDMA interfaces)
RUST_LOG=info ./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
Windows (WSL2)

Build OpenLake with WSL2. See the full Windows development environment guide.


# Run in an administrator PowerShell terminal:
wsl --install -d Ubuntu
wsl

# Then run inside the Ubuntu shell:
# Install system dependencies
sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates build-essential pkg-config clang cmake libhwloc-dev libudev-dev curl git

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

# Clone and build OpenLake
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --release --locked -p openlake_server --bin openlaked

# Create local storage directories
mkdir -p data/d0 data/d1 data/d2 data/d3

# Start OpenLake in TCP mode
RUST_LOG=info ./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml

If you would like support for a different build system, please open an issue on GitHub.

Architecture

OpenLake keeps the path from storage to GPU memory short, predictable, and low latency.

Explore the architecture and user docs, inspect the included configurations, or read the OpenLake engineering blog for deeper technical discussions.

Contributing

We welcome and value any contributions and collaborations. Please check out Contributing to OpenLake for how to get involved.

Get in touch

License

Apache 2.0.

openlake (openlake-project/openlake)

正在搜索数据库…

这是什么?OpenLake 是面向 AI 基础设施的存储引擎。它能在训练和推理过程中持续为 GPU 供数,小数据 I/O 吞吐量高,速度接近缓存,同时保持完全持久化和可靠存储。
它是缓存还是存储服务器?它是一台存储服务器。它对外提供兼容 S3 的对象存储,还可以充当大语言模型(LLM)的 KV 缓存池,但与普通缓存不同,它是持久化且可靠的。
怎么把它跑起来?克隆代码仓库,从源码编译 OpenLake 二进制文件,然后用默认的单节点配置启动存储服务。之后任何 S3 客户端都可以与它通信。若要使用 vLLM,请安装连接器并启动存储服务,然后在启用 OpenLake 的情况下运行 vLLM。
编译前需要准备什么?请按平台指南安装编译依赖,指南涵盖 Ubuntu 或 Debian、macOS(开发用途)以及通过 WSL2 运行的 Windows。核心 I/O 路径依赖 Linux io_uring。
集群部署有什么需要注意的地方吗?OpenLake 默认只卸载到同一台主机上,因此要在整个 GPU 集群中共享,需要使用 --config 参数启动 openlaked。在 Kubernetes 上,请参考 Helm KV 部署指南,在每个选定节点上部署一个实例。
它可以在哪些操作系统上编译?Ubuntu 和 Debian 有专门的编译指南,macOS 支持开发用途,Windows 则通过 WSL2 运行。文档中没有提到原生 Windows 编译方式。
使用它需要 GPU 吗?S3 存储服务不需要 GPU,它在默认配置下运行于单个节点上。针对 vLLM 的 KV 缓存卸载以及 GPU 集群相关功能,则面向 GPU 主机和 GPU 集群。
它为什么比普通 S3 存储更快?它在 Linux io_uring 上为每个 CPU 核心运行一个绑定的异步运行时,并避免核心之间的工作窃取。README 中还提到,可实现超过百万的 IOPS,且延迟在 1 毫秒以内,并支持零拷贝传输到 GPU 显存。
KV 缓存卸载能把推理提速多少?README 中提到,在前缀已缓存、上下文窗口为 128K 的情况下,首个 token 出现的时间可缩短至原来的 1/66,即提速 66 倍。提速来自复用 KV 缓存,而不必对重复的提示词反复进行预填充计算。
它和其他产品做过基准测试吗?README 称 OpenLake 在 MLPerf Storage v3.0 2026 对象存储检查点保存基准测试中位居第一,领先 NVIDIA 与 Nebius。这些数据应视为作者自己的说法。
大小
4.7 MB
星标
3.1K
时长
6个月
提交 · 90 天