gpt4all README

The project's README from github.com/nomic-ai/gpt4all. As one markdown file: README.md.

GPT4All

Now with support for DeepSeek R1 Distillations

Website • Documentation • Discord • YouTube Tutorial

GPT4All runs large language models (LLMs) privately on everyday desktops & laptops.

No API calls or GPUs required - you can just download the application and get started.

Read about what's new in our blog.

Subscribe to the newsletter

https://github.com/nomic-ai/gpt4all/assets/70534565/513a0f15-4964-4109-89e4-4f9a9011f311

GPT4All is made possible by our compute partner Paperspace.

— Windows Installer —

— Windows ARM Installer —

— macOS Installer —

— Ubuntu Installer —

The Windows and Linux builds require Intel Core i3 2nd Gen / AMD Bulldozer, or better.

The Windows ARM build supports Qualcomm Snapdragon and Microsoft SQ1/SQ2 processors.

The Linux build is x86-64 only (no ARM).

The macOS build requires Monterey 12.6 or newer. Best results with Apple Silicon M-series processors.

See the full System Requirements for more details.



Get it on Flathub
Flathub (community maintained)

Install GPT4All Python

gpt4all gives you access to LLMs with our Python client around llama.cpp implementations.

Nomic contributes to open source software like llama.cpp to make LLMs accessible and efficient for all.

pip install gpt4all
from gpt4all import GPT4All
model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf") # downloads / loads a 4.66GB LLM
with model.chat_session():
    print(model.generate("How can I run LLMs efficiently on my laptop?", max_tokens=1024))

Integrations

:parrot::link: Langchain :card_file_box: Weaviate Vector Database - module docs :telescope: OpenLIT (OTel-native Monitoring) - Docs

Release History

Contributing

GPT4All welcomes contributions, involvement, and discussion from the open source community! Please see CONTRIBUTING.md and follow the issues, bug reports, and PR markdown templates.

Check project discord, with project owners, or through existing issues/PRs to avoid duplicate work. Please make sure to tag all of the above with relevant project identifiers or your contribution could potentially get lost. Example tags: backend, bindings, python-bindings, documentation, etc.

Citation

If you utilize this repository, models or data in a downstream project, please consider citing it with:

@misc{gpt4all,
  author = {Yuvanesh Anand and Zach Nussbaum and Brandon Duderstadt and Benjamin Schmidt and Andriy Mulyar},
  title = {GPT4All: Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/nomic-ai/gpt4all}},
}

gpt4all (nomic-ai/gpt4all)

Searching Database…

77.4K ·GitHub
这是什么?GPT4All 是一款桌面应用,可以在普通的笔记本电脑和台式机上私密地运行大语言模型,无需调用 API,也不需要 GPU。
这是聊天应用,还是可以导入代码的库?两者都是:它既有带安装程序的原生桌面聊天应用,也提供 Python 客户端(gpt4all 包),用于在代码中调用同一套本地大语言模型。
怎样才能让它运行起来?下载适用于你操作系统(Windows、Windows ARM、macOS 或 Ubuntu)的预编译安装程序并运行,或者安装 gpt4all Python 包,在代码中使用它。
所有操作系统版本支持的硬件都一样吗?Linux 版本仅支持 x86-64,不支持 ARM;Windows ARM 版本只能在高通骁龙或微软 SQ1/SQ2 芯片上运行;macOS 版本需要 Monterey 12.6 或更新的系统(在 Apple Silicon 上效果最好)。
这款软件适合哪些人?可以商用吗?它面向所有希望在自己电脑上私密地本地使用大语言模型聊天的人,并且明确是开源的,允许商业使用。
支持哪些平台?提供 Windows、Windows ARM、macOS 和 Ubuntu Linux 的原生安装程序。
我的聊天数据会留在自己的电脑上吗?会的。GPT4All 在你的设备上私密地运行大语言模型,还能通过 LocalDocs 功能让你在本地与自己的文档对话。
它支持哪些模型格式和架构?GPT4All 支持 GGUF 格式的模型,并且已经扩展支持多种模型架构,包括 Mistral 7B 等模型,以及 Rift Coder 等本地代码模型。
可以把它做成 API 服务器吗?可以。它提供基于 Docker 的 API 服务器,能通过与 OpenAI 兼容的 HTTP 端点对外提供本地大语言模型推理服务。
它能和其他 AI 工具集成吗?可以。它可以与 Langchain、Weaviate 向量数据库以及用于 OpenTelemetry 原生监控的 OpenLIT 集成。
为什么它不需要 GPU 也能运行大语言模型?它采用基于 llama.cpp 的实现(该项目也为其做出了贡献),使大语言模型推理足够高效,能在普通 CPU 上运行。
Creator
nomic-ai
Size
43.6 MB
Stars
77.4K
Age
3y
Commits · 90 Days