hyperflare-inference README

项目的 README,来自 github.com/NVIDIA/hyperflare-inference。合并为一个 markdown 文件:README.md。

Flare — ASCII wordmark above a supermassive solar flare

Coding agents: skip this README and read AGENTS.md directly.

Hyperflare is a local inference server for NVIDIA SM120/SM121 GPUs, built for long conversations, coding agents, and repeated tool use. Its goal is to make the memory, bandwidth, and compute in workstations and compact systems such as DGX Spark go further, while keeping generation responsive.

This is a research prototype, and is made available via open-source to enable technical collaboration and collect early feedback.

The design rests on a few principles:

vLLM covers a much broader range of models, hardware, and deployment patterns. Hyperflare trades that breadth for a focused serving contract around selected models and local workloads. Hyperflare is not, and will never be, a replacement for production inference engines like vLLM.

Think of Hyperflare as a specialized option when its hardware and model coverage fit your machine.

Start with model preparation and serving. For native Windows development and serving prebuilt model cores, follow Windows serving. See the CLI guide for configuration and the core design for the architectural reasoning.

Licensing and contributions

Hyperflare's project code is licensed under Apache 2.0. Contributions require the sign-off described in CONTRIBUTING.md. Third-party attributions are listed in NOTICE; upstream terms remain applicable to third-party code and assets.

The PyTorch-derived image resizing implementation in helios/ retains the PyTorch BSD-style license, also distributed with Heliopause. The DeepSeek recipe assets retain their MIT license. The heliopause/LICENSE and heliopause/NOTICE symlinks refer to the root files for Python package distribution.

Upstream chat templates are fetched with the model checkpoints and annotated locally using bundled insertion patches. Gemma uses pinned Google sources. The custom Kimi segmented renderer is included as project code. Downloaded model assets and exported model cores retain their upstream terms; see frontend preparation.

hyperflare-inference (NVIDIA/hyperflare-inference)

正在搜索数据库…

这是什么?Flare 是一个本地推理服务器,专为 NVIDIA SM120/SM121 GPU 打造,适用于长对话、编程智能体以及反复调用工具的场景。它目前是一个研究原型,以开源形式发布,用于收集早期反馈并促进协作。
它能完全替代 vLLM 这类工具吗?不能。vLLM 支持的模型、硬件和部署方式范围广得多;Flare 则有意放弃这种广度,专注于一部分选定的模型以及本地单机工作负载,提供精简的服务接口。
我要怎么让它跑起来?先从模型准备和部署开始,配置方法参见 CLI 指南;如果想深入了解,核心设计文档会解释架构背后的思路。
我需要特定的 GPU 硬件吗?需要。它面向工作站和紧凑型系统(如 DGX Spark)中的 NVIDIA SM120/SM121 GPU,并不是通用的多厂商推理服务器。
这个项目能直接用于生产环境吗?不能。项目明确将自己定位为研究原型,公开发布是为了收集早期反馈、促进技术协作,而不是一套生产级的服务系统。
编程智能体能直接读这个 README 来使用 Flare 吗?不能直接用。README 会告诉编程智能体跳过它,转而阅读 AGENTS.md,获取安装和使用说明。
为什么不直接用 vLLM 做本地部署?Flare 的目标是在单台工作站的内存、带宽和算力上发挥最大效能,服务于长时间、反复调用工具的会话,并把对话状态(包括循环状态和 KV 缓存)视为可以复用、分支和恢复的东西。相比 vLLM 广泛的生产场景覆盖,这个目标更加聚焦。
Ignition 和 Helios 是什么?Ignition 是基于 Python/PyTorch 的构建阶段,负责编译模型并捕获 CUDA 图,将其打包成运行时包;Helios 是一个小型 Rust 运行时,用于部署这个包,负责调度、缓存存储和请求处理,全程不依赖 Python。
作者
NVIDIA
大小
36 MB
星标
817
时长
29天
提交 · 90 天