Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
Global
5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
Run frontier MoE models on hardware you already own — pure C, zero deps, experts streamed from disk. Tiny engine, immense model. 🐦
#363 · +82 · 1d ·GitHub
Build AI agents that run 100% on-device. Sub-100ms latency on Qualcomm NPU. Zero cloud dependency.
A free, open source, and extensible speech-to-text application that works completely offline.
#1100 · +24 · 1d ·GitHub
Fastest and only macOS Dictation app with on-device STT and custom trained AI enhancement model. Windows pre-build available! A local Wispr Flow alternative.
A high-performance inference engine for AI models. It allows you to deploy AI directly in your app with zero latency, full data privacy, and no inference costs. Key features:
A gallery that showcases on-device ML/GenAI use cases and allows people to try and use models locally.
Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Cross-platform, customizable ML solutions for live and streaming media.
#1815 · +13 · 1d ·GitHub
The world's first completely free and open-source face recognition SDK for Windows and Linux from Faceplugin
Your AI second brain. Self-hostable. Get answers from the web or your docs.
#2038 · +11 · 1d ·GitHub
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python.
dmg, drag Parrot to Applications, and open it. Turn on Parrot when macOS asks for Accessibility and the microphone. The first start downloads the speech model (about 150 MB).
WebLLM is a high-performance in-browser LLM inference engine that brings language model inference directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU.
#3605 · +4 · 1d ·GitHub
System requirements to run python examples on x86-64 systems:
#4108 · +3 · 1d ·GitHub
Eight SDKs, one C++ core. A capability registry routes every call to the best engine on the device, and the Console manages your fleet from above.
#4135 · +3 · 1d ·GitHub
3 GB model into memory. 35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
#4934 · +2 · 1d ·GitHub
Run a 105 GB AI model on a Mac that can't hold it. 8-Flash-Next (125B mixture of experts) from your SSD and caches the busiest experts in memory, so it runs on Macs with 16 to 64 GB. One native Swift binary on MLX and Metal, no Python, offline.
> This repository is archived. Development and support have ended. > > The code remains available for reference under the terms of the LICENSE.
9 million parameter language model that generates text on an ESP32-S3 microcontroller. 88 tokens per second on a small screen wired to the chip. It fits because most of the model lives in flash instead of RAM, using Per-Layer Embeddings, an idea from Google's Gemma 3n.
#5989 · +1 · 1d ·GitHub
A diagnostic performance logging utility for macOS: logs CPU, memory, GPU, network, and battery from the menu bar to help detect and diagnose performance issues. Free, open source, no telemetry.
#6007 · +1 · 1d ·GitHub
This project has not provided a description yet.
#6243 · +1 · 1d ·GitHub
⬇️ Download firmware · ▶️ Watch the promo · 🤗 Get the model · 📰 Press kit
Decentralized P2P mesh for autonomous AI agents — self-sovereign identity, peer-to-peer chat, and on-device AI that negotiates tasks on your behalf. No central server.
#7840 · +0 · 1d ·GitHub
Local Laya typed decisions on Apple Core ML and Neural Engine. Validated ports, 5 ms short decisions on M3 Max, reproducible speed and energy benchmarks.