xiaozhi-esp32 README

An MCP-based Chatbot

Introduction

👉 Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】

👉 Handcraft your AI girlfriend, beginner's guide【bilibili】

As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.

Recent Updates

The project now requires ESP-IDF v6.0.1 or later. ESP-IDF v6.1 is the recommended SDK. ESP-IDF 5.x is no longer supported. The current matrix contains 171 variants; ESP32-S31 builds require IDF 6.1 or later.
MQTT and BluFi cryptographic code has migrated to PSA Crypto. IDF 6 component splits and third-party dependency compatibility have also been addressed.
Audio pipeline concurrency, MQTT/UDP packet validation, and release-matrix selection have been hardened.
ESP32-P4 Rev1 and Rev3 are both supported on IDF 6 with ESP-SR 2.4.7.

Features Implemented

Wi-Fi, wired Ethernet, USB RNDIS, and ML307/EC801E or NT26 Cat.1 4G networking; supported boards can switch between Wi-Fi and 4G
Offline voice wake-up with ESP-SR, including customizable wake words
Two communication transports: WebSocket and MQTT + UDP
Opus audio streaming with conventional streaming ASR + LLM + TTS pipelines and Realtime end-to-end voice models; AEC-capable hardware supports realtime full-duplex interaction
Speaker recognition, identifies the current speaker 3D Speaker
OLED / LCD displays with emoji and rich expression support, plus camera vision input on supported boards
Battery display and power management
39 interface languages, with localized voice prompts where available and English fallback
ESP32, ESP32-C3, ESP32-C5, ESP32-C6, ESP32-S3, and ESP32-P4 chip platforms
Wi-Fi provisioning through hotspot or BluFi
Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
Customizable wake words, fonts, emojis, and chat backgrounds with online web-based editing (Custom Assets Generator)

Hardware

Breadboard DIY Practice

See the Feishu document tutorial:

👉 "XiaoZhi AI Chatbot Encyclopedia"

Breadboard demo:

Breadboard Demo

Supports 138 Board Directories and 171 Release Variants (Partial List)

Software

Firmware Flashing

For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.

The firmware connects to the official xiaozhi.me server by default. Personal users can register an account to use the Qwen real-time model for free.

👉 Beginner's Firmware Flashing Guide

Development Environment

Cursor or VSCode
Install the ESP-IDF plugin. The minimum SDK is ESP-IDF v6.0.1; ESP-IDF v6.1 is recommended. ESP-IDF 5.x is not supported.
Linux is better than Windows for faster compilation and fewer driver issues
This project uses Google C++ code style, please ensure compliance when submitting code

Developer Documentation

Custom Board Guide - Learn how to create custom boards for XiaoZhi AI
MCP Protocol IoT Control Usage - Learn how to control IoT devices via MCP protocol
MCP Protocol Interaction Flow - Device-side MCP protocol implementation
MQTT + UDP Hybrid Communication Protocol Document
A detailed WebSocket communication protocol document

Large Model Configuration

If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the xiaozhi.me console for configuration.

👉 Backend Operation Video Tutorial (Old Interface)

For server deployment on personal computers, refer to the following open-source projects: