Global

8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. cpp and MLX.

PyTorch implementation of VisionHOPE, a visual backbone based on self-referential nested learning (SRNL).

We've been very happy to see FlashAttention being widely adopted in such a short time after its release. This page contains a partial list of places where FlashAttention is being used.

An Open-source book with Modern CUDA learning Notes for Beginners - BF16/FP8/FP4, HGEMM, CuTe, Flash-Attention, etc.

DeepGEMM is a unified, high-performance tensor core kernel library that brings together the key computation primitives of modern large language models — GEMMs (FP8, FP4, BF16), fused MoE with overlapped communication (Mega MoE), MQA scoring for the lightning indexer…

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

Taskflow helps you quickly write high-performance task-parallel programs with high programming productivity. It is faster, more expressive, fewer lines of code, and easier for drop-in integration than many existing task programming libraries.

8-27B's published mixed NVFP4/FP8 weight representation unchanged on four Tesla V100-SXM2-16GB. m8n8k4. No FP8→FP4 requantization.

CuPy is a NumPy/SciPy-compatible array library for GPU-accelerated computing with Python. CuPy acts as a drop-in replacement to run existing NumPy/SciPy code on NVIDIA CUDA or AMD ROCm platforms.