Global

8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

WanGP by DeepBeepMeep : The best Open Source Generative Models Accessible to the GPU Poor

An agent-first reverse-engineering app for macOS, Windows and Linux.

Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. 32 GB.

8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. cpp and MLX.

A high-throughput and memory-efficient inference and serving engine for LLMs

Find out which open-source Large Language Models (LLMs) your hardware can comfortably run. llmfit inspects your CPU, system RAM, GPU(s), VRAM, and accelerator configuration to recommend models across popular quantizations.

VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations via an end-to-end diffusion autoregressive architecture, bypassing discrete tokenization to achieve highly natural and expressive synthesis.

Open-source local AI audio & music studio for macOS & Windows. TTS, voice cloning, songs with lyrics, and multi-track scores — 100% offline, zero API fees.

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

Local UI to run and train LLMs and diffusion models. 8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

This repository is Operit's Android edition. Operit 2 is a separate second-generation implementation centered on a shared Rust runtime, Flutter clients, and the operit2 CLI/TUI.

1-Flash-1017 (748B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.

MiniCPM-V and MiniCPM-o are multimodal LLM series designed for strong performance and efficient deployment on devices. MiniCPM-V focuses on efficient vision-language understanding across image, video and text inputs.

Koharu introduces a local-first workflow for manga translation, utilizing the power of ML to automate the process. It combines the capabilities of object detection, OCR, inpainting, and LLMs to create a seamless translation experience.

llamafile lets you distribute and run LLMs with a single file.

Note: This series will continue to grow as I write more blogs and create more videos on new topics. Keep learning.

mini cli search engine for your docs, knowledge bases, meeting notes, whatever. Tracking current sota approaches while being all local

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python.

\janas-chat. 8 times as fast with the dense model (Qwen3-4B 346 tok/s against 193), whose whole blocks go to the GPU, through Intel's own Windows driver there.

We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

Run Bonsai 2 27B locally on Mac (Metal), Linux/Windows (CUDA, Vulkan, ROCm), or CPU.

Note that this project is under active development. \ API and command-line option may change frequently.

8 is the large language model series developed by Qwen team, Alibaba Group.

The Open Source Strix Halo AI engine. 56tg tok/s single user with DFlash2

Hy-MT2 is a family of “fast-thinking” multilingual translation models designed for complex real-world scenarios. 8B, 7B, and 30B-A3B (MoE), all of which support translation among 33 languages and effectively follow translation instructions in multiple languages. 5x.