40.9K ·GitHub
What is this?
ChatGLM-6B is an open bilingual (Chinese-English) dialogue language model with 6.2 billion parameters, based on the General Language Model (GLM) architecture.
ChatGLM-6B is an open bilingual (Chinese-English) dialogue language model with 6.2 billion parameters, based on the General Language Model (GLM) architecture.Is this a standalone app or a model I integrate myself?It's a model and inference codebase you load via Python code (through Hugging Face transformers), plus optional web/CLI demo scripts and an API server you run yourself.
It's a model and inference codebase you load via Python code (through Hugging Face transformers), plus optional web/CLI demo scripts and an API server you run yourself.How do I run it?Install dependencies with pip install -r requirements.txt, then load the model via transformers from_pretrained, or run the provided webdemo.py (Gradio web demo), clidemo.py (CLI demo), or api.py (FastAPI server on port 8000).
Install dependencies with pip install -r requirements.txt, then load the model via transformers from_pretrained, or run the provided webdemo.py (Gradio web demo), clidemo.py (CLI demo), or api.py (FastAPI server on port 8000).Is this model still maintained or recommended to use?This repo is tagged deprecated — the authors have since released ChatGLM2-6B and the newer GLM-4 series with much better performance, longer context, and faster inference, and recommend using those instead.
This repo is tagged deprecated — the authors have since released ChatGLM2-6B and the newer GLM-4 series with much better performance, longer context, and faster inference, and recommend using those instead.Can I use it commercially?Academic research use is fully open, and free commercial use is allowed after filling out a registration questionnaire.
Academic research use is fully open, and free commercial use is allowed after filling out a registration questionnaire.Which operating systems does it support?It runs on Linux, Windows (with TDM-GCC for CPU quantization), and macOS (including Apple Silicon via the MPS backend).
It runs on Linux, Windows (with TDM-GCC for CPU quantization), and macOS (including Apple Silicon via the MPS backend).Can I deploy the API on a server?Yes, the included api.py starts a FastAPI server (default localhost port 8000) that you can call via POST requests, suitable for self-hosting on any server with sufficient GPU/CPU resources.
Yes, the included api.py starts a FastAPI server (default localhost port 8000) that you can call via POST requests, suitable for self-hosting on any server with sufficient GPU/CPU resources.What's the difference between quantization levels?FP16 (no quantization) needs about 13GB GPU memory for inference, INT8 needs about 8GB, and INT4 needs only about 6GB, with some tradeoff in generation quality at INT4.
FP16 (no quantization) needs about 13GB GPU memory for inference, INT8 needs about 8GB, and INT4 needs only about 6GB, with some tradeoff in generation quality at INT4.Is there a fine-tuning option for custom use cases?Yes, the project implements parameter-efficient fine-tuning via P-Tuning v2, which can run with as little as 7GB of GPU memory under INT4 quantization.
Yes, the project implements parameter-efficient fine-tuning via P-Tuning v2, which can run with as little as 7GB of GPU memory under INT4 quantization.What related projects exist around ChatGLM-6B?The community built acceleration projects like lyraChatGLM and ChatGLM-MNN, plus downstream apps such as langchain-ChatGLM for knowledge-base Q&A and various chat UIs like Chuanhu Chat.
The community built acceleration projects like lyraChatGLM and ChatGLM-MNN, plus downstream apps such as langchain-ChatGLM for knowledge-base Q&A and various chat UIs like Chuanhu Chat.What is VisualGLM-6B?It's a related multimodal dialogue model from the same team that adds image understanding, runnable via included clidemovision.py and webdemovision.py scripts with extra dependencies.
It's a related multimodal dialogue model from the same team that adds image understanding, runnable via included clidemovision.py and webdemovision.py scripts with extra dependencies.What are similar projects?ChatGLM2-6B, ChatGLM3, DeepAnalyze, Qwen, MOSS, Langchain-Chatchat, petals, DeepSeek-V3
ChatGLM2-6B, ChatGLM3, DeepAnalyze, Qwen, MOSS, Langchain-Chatchat, petals, DeepSeek-V3Which topics is it in?chatglm, open-weights, glm, bilingual, fastapi, quantization, fine-tuning, large-language-models, chatbot, deprecated, huggingface, local-first
chatglm, open-weights, glm, bilingual, fastapi, quantization, fine-tuning, large-language-models, chatbot, deprecated, huggingface, local-first