What is this?
FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine built to run frontier-scale open-weight models, including 290B+ parameter models, on consumer hardware like gaming PCs.
FreeToken is an edge-native Mixture-of-Experts (MoE) serving engine built to run frontier-scale open-weight models, including 290B+ parameter models, on consumer hardware like gaming PCs.Is this a desktop app or a command-line engine?Both: there's a desktop app with a GUI for Windows and Linux that sets up the engine for you, and a CLI you can install separately for more control.
Both: there's a desktop app with a GUI for Windows and Linux that sets up the engine for you, and a CLI you can install separately for more control.How do I get it running?Download the desktop app for Windows or Linux from flashml.ai, or install the CLI with `uv` (recommended) or pip, or build it from source.
Download the desktop app for Windows or Linux from flashml.ai, or install the CLI with `uv` (recommended) or pip, or build it from source.Do I need Python to set this up?Yes, the CLI is Python-based and installs via `uv` or pip; the desktop app bundles the engine setup for you instead.
Yes, the CLI is Python-based and installs via `uv` or pip; the desktop app bundles the engine setup for you instead.What's the catch with running 290B+ models on a gaming PC?It doesn't fit the whole model in VRAM; instead it uses bandwidth-adaptive CPU-GPU co-execution, prefill streaming, and LRU expert caching, so real-world speed depends on your system's CPU-GPU bandwidth and is limited to the supported RTX GPU families.
It doesn't fit the whole model in VRAM; instead it uses bandwidth-adaptive CPU-GPU co-execution, prefill streaming, and LRU expert caching, so real-world speed depends on your system's CPU-GPU bandwidth and is limited to the supported RTX GPU families.Which operating systems does the desktop app support?The desktop app is available for Windows and Linux; there's no mention of a macOS version.
The desktop app is available for Windows and Linux; there's no mention of a macOS version.Does it need an NVIDIA GPU?Yes, native support targets NVIDIA RTX 30, RTX 40, and RTX 50 series GPUs, combined with CPU and host memory for offloading experts.
Yes, native support targets NVIDIA RTX 30, RTX 40, and RTX 50 series GPUs, combined with CPU and host memory for offloading experts.What models does FreeToken support?It supports frontier open-weight MoE models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2, across quantization formats like MXFP4, NVFP4, FP8, and BF16.
It supports frontier open-weight MoE models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2, across quantization formats like MXFP4, NVFP4, FP8, and BF16.Can I use FreeToken with coding agents like Claude Code or Codex?Yes, it exposes Anthropic/OpenAI-compatible APIs so it plugs into coding and tool-calling agents such as Codex, Claude Code, OpenCode, OpenClaw, and DeepSeek Harness.
Yes, it exposes Anthropic/OpenAI-compatible APIs so it plugs into coding and tool-calling agents such as Codex, Claude Code, OpenCode, OpenClaw, and DeepSeek Harness.What is the FTW format mentioned in the features?FTW is FreeToken's own fast weight format, part of its edge-native runtime designed for efficient MoE model loading and serving.
FTW is FreeToken's own fast weight format, part of its edge-native runtime designed for efficient MoE model loading and serving.What inspired FreeToken's design?It was deeply inspired by mini-sglang and reused design and code from SGLang, vLLM, FlashInfer, flash-linear-attention, LightLLM, and llama.cpp.
It was deeply inspired by mini-sglang and reused design and code from SGLang, vLLM, FlashInfer, flash-linear-attention, LightLLM, and llama.cpp.What are similar projects?sglang, project-maya, TypeLLM, Strata, v100-skinny, vllm, airllm, jeeves
sglang, project-maya, TypeLLM, Strata, v100-skinny, vllm, airllm, jeevesWhich topics is it in?edge, minimax, glm, edge-ai, inference-engine, model-serving, moe, llm-inference, qwen, quantization, transformer, local-llm
edge, minimax, glm, edge-ai, inference-engine, model-serving, moe, llm-inference, qwen, quantization, transformer, local-llm