Setup: A100 on f16, measured total time for a forward+backward passWhat is this?
xFormers is Meta's (Facebook Research) toolbox of hackable, optimized Transformer building blocks that you compose together — reusable pieces for your own models rather than a monolithic framework.
xFormers is Meta's (Facebook Research) toolbox of hackable, optimized Transformer building blocks that you compose together — reusable pieces for your own models rather than a monolithic framework.Is it a standalone app or a library I import into my own code?A Python library: independent, customizable building blocks you can use without boilerplate code. They are domain-agnostic — researchers use them in vision, NLP and more.
A Python library: independent, customizable building blocks you can use without boilerplate code. They are domain-agnostic — researchers use them in vision, NLP and more.What do I need to have installed first?PyTorch 2.10.0 — the stable pip package targets it; other PyTorch versions (including nightlies) require the development binaries or a from-source build.
PyTorch 2.10.0 — the stable pip package targets it; other PyTorch versions (including nightlies) require the development binaries or a from-source build.What usually goes wrong during installation?Source builds are the fragile part: NVCC must match your CUDA runtime and GCC, set TORCHCUDAARCHLIST for your target GPUs, and use MAXJOBS=2 if the build OOMs. On Windows, enable long paths plus git config --global core.longpaths true.
Source builds are the fragile part: NVCC must match your CUDA runtime and GCC, set TORCHCUDAARCHLIST for your target GPUs, and use MAXJOBS=2 if the build OOMs. On Windows, enable long paths plus git config --global core.longpaths true.How is it licensed?A BSD-style license, as found in the LICENSE file; the codebase also includes code from the triton-lang/kernels repo.
A BSD-style license, as found in the LICENSE file; the codebase also includes code from the triton-lang/kernels repo.Do I need an NVIDIA GPU to actually use it?In practice yes — xFormers ships its own CUDA kernels (it dispatches to other libraries when relevant), and the repo's CI also builds for AMD's ROCm.
In practice yes — xFormers ships its own CUDA kernels (it dispatches to other libraries when relevant), and the repo's CI also builds for AMD's ROCm.What's the headline feature?Memory-efficient exact attention, up to 10x faster — and it is exact attention, not an approximation, available through xformers.ops. The published benchmark times a forward+backward pass on an A100 in f16.
Memory-efficient exact attention, up to 10x faster — and it is exact attention, not an approximation, available through xformers.ops. The published benchmark times a forward+backward pass on an A100 in f16.What else is in the toolbox besides attention?Sparse and block-sparse attention, plus a set of fused ops: fused softmax, fused linear layers, fused layer norm, fused dropout(activation(x+bias)) and fused SwiGLU — all framed as optimized blocks beyond the plain PyTorch primitives.
Sparse and block-sparse attention, plus a set of fused ops: fused softmax, fused linear layers, fused layer norm, fused dropout(activation(x+bias)) and fused SwiGLU — all framed as optimized blocks beyond the plain PyTorch primitives.Why reach for it instead of plain PyTorch?It is research-first: you get bleeding-edge components that are not yet available in mainstream libraries like PyTorch, with speed and memory efficiency as stated design goals.
It is research-first: you get bleeding-edge components that are not yet available in mainstream libraries like PyTorch, with speed and memory efficiency as stated design goals.How can I check which kernels my install actually has?Run the installation-check command from the README's 'Testing the installation' section — it reports information about your xFormers installation and which kernels were built and are available.
Run the installation-check command from the README's 'Testing the installation' section — it reports information about your xFormers installation and which kernels were built and are available.What are similar projects?flash-attention, OpenMythos, FlashMLA, DeepGEMM, llama3-from-scratch, Attention-Residuals, ComfyUI-Ref2VA-VSA, MSA
flash-attention, OpenMythos, FlashMLA, DeepGEMM, llama3-from-scratch, Attention-Residuals, ComfyUI-Ref2VA-VSA, MSAWhich topics is it in?meta, performance-optimization, attention, vision, rocm, attention-mechanism, high-performance, gpu-acceleration, transformer, nlp, bsd, cuda
meta, performance-optimization, attention, vision, rocm, attention-mechanism, high-performance, gpu-acceleration, transformer, nlp, bsd, cuda