torch_tpu README

项目的 README,来自 github.com/google-pytorch/torch_tpu。合并为一个 markdown 文件:README.md。

TorchTPU: PyTorch backend for Google Cloud TPUs

Description

This project provides a PyTorch backend for Google's Tensor Processing Units (TPUs), enabling users to run PyTorch models and operations on TPU devices. It includes custom ATen kernels for various PyTorch operations, a compilation cache for optimizing execution, and examples for distributed training and model inference.

Features

User Guide and Interactive Notebooks

A user guide is available at https://torchtpu.readthedocs.io/. The markdown sources for this documentation are available in the docs/ directory. Jupyter notebook versions of these documents are available in the docs/notebooks/ folder.

The notebooks can be opened in Colab. You will need to be running this from a TPU VM, a v6e GCE instance is easiest.

Installation

The same instructions are available in the user guide linked above.

TorchTPU's Python version support policy matches PyTorch's Release Compatibility Matrix. Currently, Python 3.11–3.15 is supported (including free-threaded 3.14t and 3.15t builds, with Python 3.15 requiring PyTorch >= 2.15). See the Python Version Support Policy for more details.

First create a Python virtual environment:

python -m venv .venv
source .venv/bin/activate

or, with uv

uv venv .venv
source .venv/bin/activate

Install via pip

Install the latest public release from PyPI (for uv, prepend uv to all pip commands):

pip install torch_tpu --extra-index-url https://download.pytorch.org/whl/cpu

Build from source

Wheels can be built via:

bazel build --config=wheel_common //ci/wheel:torch_tpu_wheel

All OSS Bazel builds default to optimized mode (-c opt) automatically via .bazelrc. Unoptimized builds (-c fastbuild, -c dbg) can degrade performance and are not used for wheel builds. If you are developing or debugging C++ code and explicitly want an unoptimized wheel, pass --//:allow_unoptimized_wheel=True:

bazel build -c dbg --config=wheel_common //ci/wheel:torch_tpu_wheel \
    --//:allow_unoptimized_wheel=True

--config=wheel_common is required, and the build fails at build time without it. The config sets --//:wheel_build=True, which routes the shared XLA/MLIR backend through the torch_version-reset transition so that pywrap factors it into a single libxla_base.so; built without the flag, every per-version common ships its own copy of the backend and the resulting wheel aborts on import torch with duplicate static registrations.

wheel_common also turns on the Remote Build Execution (RBE) remote cache and remote executor. If you do not have RBE credentials, append --config=no_rbe to strip those flags back out:

bazel build --config=wheel_common //ci/wheel:torch_tpu_wheel --config=no_rbe

The order matters here: --config=no_rbe has to come after --config=wheel_common, because the later config is the one that wins for the remote-cache and remote-executor flags they both set.

Install wheel via:

cd <path_to_repo>
python -m venv .venv; source .venv/bin/activate
# The extra-index-url provides the CPU version of torch rather than the default CUDA build.
pip install ../bazel-bin/ci/wheel/*.whl --extra-index-url https://download.pytorch.org/whl/cpu

Note that installing without --extra-index-url https://download.pytorch.org/whl/cpu will install the CUDA version of torch; pass --extra-index-url as shown above or install the CPU build separately.

Getting Started

Some pointers to get you started:

Bazel setup

This project uses Bazel for building and dependency management.

To install Bazel, we actually install Bazelisk, which is a transparent wrapper that handles downloading and installing Bazel itself.

There are several ways to install Bazelisk. Here are some common ways:

See the Bazelisk README for more install instructions

However you install it, add it's location to PATH.

To verify the install, run bazel info in the repo's root directory:

bazel info

It should print information about the repo without errors.

VSCode clangd Setup

For C++ code navigation, it is recommended to use VSCode with clangd extension. Relying on a compilation database generated from Bazel action graph, clangd is able to enrich the editor with various smart features, including code completions, go-to definitions, etc.

  1. Generate the initial compile_commands.json:

    tools/setup_clangd.py
    

    If the VSCode code CLI is on PATH, the script also installs the clangd extension automatically. Otherwise, install the extension via the editor UI.

  2. Uninstall the Microsoft C++ extension, which interferes with clangd:

    code --uninstall-extension ms-vscode.cpptools
    

It may take a while (> 30 minutes) for the first run to make a full build, but reruns will be incremental and cheap. If you have run bazel build recently, generated files already exist so you can pass --no-build to skip waiting for heavy builds (note that on a fresh workspace, skipping the build may prevent clangd from resolving generated files). See more options in the tools/setup_clangd.py file-level comments.

Alternatively, you can invoke the script via nox (arguments after -- are forwarded):

nox -s refresh_compile_commands -- --no-build
nox -s refresh_compile_commands -- //src/torch_tpu/ops/...

Contributing

We welcome contributions to the PyTorch on TPU project! Please see our CONTRIBUTING.md for guidelines on how to contribute, including signing our Contributor License Agreement (CLA) and code review processes.

Star History

Star History Chart

License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.

torch_tpu (google-pytorch/torch_tpu)

正在搜索数据库…

这是什么?TorchTPU 是面向谷歌张量处理单元(TPU)的 PyTorch 后端,可让你在 TPU 硬件上运行 PyTorch 模型和算子。
这是独立的框架,还是现有 PyTorch 代码的附加组件?它是一个库/后端,而不是独立的框架:你仍然编写普通的 PyTorch 代码,TorchTPU 提供在 TPU 上执行这些代码所需的自定义 ATen 内核、XLA 编译缓存以及 torch.compile 集成。
我该如何安装 TorchTPU?从 PyPI 安装已发布的 wheel 包(当前版本为 0.2.0),可使用 pip 或 uv;如果你需要自定义或未经优化的构建版本,也可以使用 Bazel 从源码构建 wheel 包。
使用前我需要准备什么?需要 Python 3.11+,并能访问真实的 TPU 设备——README 建议在 TPU 虚拟机上运行,其中 v6e GCE 实例是最简单的选择。若从源码构建,还需要通过 Bazelisk 安装 Bazel。
安装时有哪些需要注意的坑?如果安装 wheel 时不加 --extra-index-url,默认会拉取 torch 的 CUDA 版本,而不是兼容 TPU 的版本,因此你需要指定正确的索引 URL,或者另外单独安装 CPU 版本。从源码构建还必须加上 --config=wheelcommon 参数,否则生成的 wheel 包在导入时会因重复的静态注册而崩溃。
哪些模型提供了示例代码?项目包含 Llama、Qwen、ResNet、DLRM、minGPT 和 UNet 的示例,涵盖分布式训练和推理两类工作流程。
它支持分布式训练吗?支持,TorchTPU 支持在多个 TPU 设备上进行分布式训练,包括数据并行和张量并行。
它能与 torch.compile 集成吗?能,它与 torch.compile() 集成,使模型可以在 TPU 上以图模式执行。
编译缓存有什么用?它会缓存已编译的 XLA 计算,因此同一计算图的重复执行会更快,而无需每次都重新编译。
大小
2 GB
星标
857
更新
提交Nov 18
时长
8个月
提交 · 90 天