Strata README

The project's README from github.com/Niko1221/Strata. As one markdown file: README.md.

Strata

English · 简体中文 · 日本語 · Deutsch · Français · Español · Português

Run a 125-billion-parameter AI model on your own gaming PC
NVIDIA or AMD graphics card (12 GB or more) · Windows or Linux · free and open source

A voxel pagoda garden that Strata's model wrote, running in the browser
A voxel pagoda garden, 1 shot prompt running on an RTX 5070 with Strata (IQ3_S, 128K context) · full video (49 s)

Strata runs Qwen3.8-Flash-Next on a normal PC. This is a large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps and coding agents. Nothing leaves your PC.

How fast is it?

We measured it on two ordinary gaming PCs. A token is about ¾ of a word.

NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAMAMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM
Size Writes answers Reads your prompt
Q2_0 94 tokens/s 2,650 tokens/s
IQ2_XS 79 tokens/s 2,090 tokens/s
IQ3_XXS 62 tokens/s 1,750 tokens/s
IQ3_S 53 tokens/s 1,620 tokens/s
Coder 55 tokens/s 2,180 tokens/s
Size Writes answers Reads your prompt
Q2_0 60 tokens/s 1,160 tokens/s
IQ2_XS 52 tokens/s 1,110 tokens/s
Coder 44 tokens/s 1,420 tokens/s

NVIDIA: Q2_0 with engine 0.1.36, the other rows with 0.1.26 (4K answers, 32K prompts). The full tables are in DETAILS.md. A card with more VRAM is faster: an RTX 3090 (24 GB) should write about 100-140 tokens per second. Long chats and other cards: speed of each model, community results.

Buy Me A Coffee
Strata is free. If it runs well on your PC, a coffee keeps the work on it going.

What you need
Graphics card NVIDIA GeForce RTX 20, 30, 40 or 50 series, or AMD Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. It needs 12 GB of VRAM or more.
RAM 32 GB or more. Your RAM decides which model fits. 64 GB runs every size.
Disk About 80 GB free. Use an SSD if you can: the first start is much faster.
System Windows 10 / 11 or Linux, and a current graphics driver from NVIDIA or AMD.

The installer sets up everything else. Two or three cards can share the model (multi-GPU).

Experimental, written and tested by community members on their own machines:

The full list: docs/INSTALL.md.

Install

Let your AI set it up

Do you use an AI coding assistant (Claude Code, Cursor, Codex, GitHub Copilot, ...)? Paste this into it:

Set up Strata on this PC for me: https://github.com/Niko1221/Strata - follow docs/AI_SETUP.md in that repository.

It checks your graphics card, RAM and disk and picks the model that fits. Then it installs and starts it and tells you how to connect your apps. AI tools can also install, start and stop Strata through its MCP server.

Or do it yourself

Download Strata and unzip it (or git clone it). Windows: double-click START-HERE.bat. Linux: run ./setup.sh in the Strata folder.

The steps are the same for NVIDIA and AMD. The installer finds your card and sets up the right engine for it. It asks you a few questions:

Press Enter each time for the recommended answer. Then it downloads the model (about 70 GB) and starts it. If the download stops, run it again: it continues where it left off. Your browser opens the Strata app at http://127.0.0.1:8080.

While the model starts, your PC can be slow or stop responding for 1-3 minutes (longest the first time). Strata loads 35-55 GB into your RAM and locks part of it for the graphics card. This is normal. Wait, and don't close the window. The window shows what Strata is doing.

Next time, run START-HERE.bat (or ./setup.sh) again. It starts right away and downloads nothing twice. Close its window to stop the model. UPDATE.bat (./update.sh) updates Strata without starting it. Updating, Docker, several cards, where the files go and every option: docs/INSTALL.md.

Which model should I pick?

The installer recommends one for your RAM. The same model comes in several sizes, compressed more or less. Smaller sizes are faster. Larger sizes are a bit smarter.

Your RAM Take Why
32 GB Coder it fits 32 GB, and it is made for code (with a 24 GB card, Q2_0 and IQ2_XS run too)
48 GB IQ2_XS (or Q2_0, the fastest) the larger sizes do not fit
64 GB IQ2_XS (recommended), or IQ3_XXS / IQ3_S every size fits; IQ3_S is the best and the slowest
96 GB or more IQ3_S, or Unsloth's UD-IQ4_XS (~4-bit) room for the largest sizes with everything else open

Sizes, downloads and what fits where: docs/MODELS.md. To add another model later, run SETUP.bat (Linux: ./setup.sh --setup).

Using it

The Strata app's Monitor tab next to a coding agent
The Strata app's Monitor (left) while a coding agent writes the pagoda garden from the video (right)

More: where your chats are stored, the API.

Something went wrong?

More problems and their fixes: docs/TROUBLESHOOTING.md. Still stuck? Open an issue and attach strata-<model>.log from the Strata folder. Found a security problem? Report it privately: SECURITY.md.

How does it work?

Models like this one usually run on servers with hundreds of gigabytes of graphics memory. Your graphics card has 12-24 GB. Strata makes the model fit by sharing the work across your whole PC. Think of a kitchen: the things you use all the time stay on the counter, and the rest waits in the pantry.

The model's 24,576 experts: the busiest on the graphics card, all of them in RAM, a lookup table on the SSD

A small helper guesses the next words; the big model checks them all at once and keeps the right ones

The longer explanation: docs/HOW_IT_WORKS.md. Every part and its numbers: the details and the paper.

Credits and license

The model is Qwen3.8-Flash-Next by the Qwen team. It was compressed by ISTA-DASLab, UkisAI (Swift 1.5) and Unsloth. Strata uses parts of llama.cpp / ggml. All credits: docs/HOW_IT_WORKS.md. Strata is open source under the MIT License. A few parts and every model have their own licenses (which ones).

Support Strata

Strata is free and open source. If it is useful to you, you can support its development:

Buy Me A Coffee

Strata (Niko1221/Strata)

Searching Database…

26.4K ·GitHub
Strata
What is this?Strata runs Qwen3.8-Flash-Next, a 125-billion-parameter AI model, on a normal gaming PC with one NVIDIA GPU (12-24GB VRAM) and 64GB of RAM, with a one-click installer for Windows and Linux.
Is this an app or just an inference engine?It is both: a local inference engine built on llama.cpp/ggml, plus a browser-based Strata app with chat, a GPU/CPU/RAM monitor, a terminal client (chat.py), and an OpenAI/Anthropic-compatible API server.
How do I install and run it?On Windows, download and unzip the project, then double-click START-HERE.bat; on Linux run ./setup.sh. It asks a few setup questions (model, size, context length, image support) then downloads everything and opens the Strata app in your browser automatically.
What hardware do I need?An NVIDIA RTX 30/40/50 GPU with 12GB+ VRAM, enough RAM for the model size you pick (up to 64GB for the biggest), about 80GB of free disk space, and Windows 10/11 or Linux; you only need to install the NVIDIA driver yourself.
Does it run smoothly right out of the box?The first start can freeze or slow your PC for 1-3 minutes while 35-55GB loads into RAM, the first model download is around 70GB, and if your PC runs low on RAM the engine can stop mid-answer (you just resend the message and it restarts).
Is an NVIDIA card required, or will AMD/Intel GPUs work?Only NVIDIA RTX 30, 40 or 50 series cards with at least 12GB of VRAM are supported; there is no mention of AMD or Intel GPU support.
What's the difference between the base model, Coder, and Swift 1.5 versions?The base Qwen3.8-Flash-Next is the full model; Coder removes half the experts to focus on coding/tool-use/images with a smaller RAM footprint; Swift 1.5 is a fine-tune that thinks for less time before answering at the same speed and RAM as the original.
Can it see images, not just text?Yes, image input is an optional setting chosen during setup, and you can attach pictures in the chat page or with the /image command in chat.py.
Why is it so fast for such a huge model?It only activates 10 of 24,576 experts per word, splits work across GPU, RAM and SSD, and uses speculative decoding where a small helper guesses several words ahead for the big model to verify at once, giving a 1.6-1.8x speedup.
Can I use Strata from my phone or another PC?Yes, running START-HERE.bat with --setup --host 0.0.0.0 and an API key exposes the server on your network so other devices can connect to it.
What are similar projects?project-maya, deltafin, StartLux-Decision, vllm, decider, slotstream, qwen3.8-Flash-DGX, Qwen3.8-Flash-Next
Which topics is it in?llama-cpp, speculative-decoding, inference-engine, mixture-of-experts, moe, gguf, local-ai, llm-inference, qwen, quantization, multimodal, openai-compatible
Creator
Niko1221
Size
28.5 MB
Stars
26.4K
Updated
CommitOct 16Releasev0.1.45Oct 17
Age
23d
Commits · 90 Days