75.5K Β·GitHub
What is this?
LlamaFactory unifies efficient fine-tuning of 100+ large language models and vision-language models in one Python toolkit, published as an ACL 2024 paper.
LlamaFactory unifies efficient fine-tuning of 100+ large language models and vision-language models in one Python toolkit, published as an ACL 2024 paper.Is it a command-line tool or does it have an interface?Both are built in: you launch runs from the CLI, or point your browser at the LLaMA Board Web UI and fine-tune without writing any code.
Both are built in: you launch runs from the CLI, or point your browser at the LLaMA Board Web UI and fine-tune without writing any code.How do I get my first fine-tune running?Install the package, prepare a dataset, then start a run from the CLI or the web UI; with no local GPU you can use free Colab or PAI-DSW notebooks instead.
Install the package, prepare a dataset, then start a run from the CLI or the web UI; with no local GPU you can use free Colab or PAI-DSW notebooks instead.Where do the models and datasets come from?They download from Hugging Face, and the README also documents pulling pre-trained models and datasets from the ModelScope Hub and the Modelers Hub.
They download from Hugging Face, and the README also documents pulling pre-trained models and datasets from the ModelScope Hub and the Modelers Hub.What if I am not on NVIDIA hardware?AMD GPUs and Ascend NPUs are supported, but each has its own setup documentation, and the main docs are marked WIP.
AMD GPUs and Ascend NPUs are supported, but each has its own setup documentation, and the main docs are marked WIP.Is it built for hobby projects or enterprise teams?Both: the README lists Amazon, NVIDIA and Aliyun among its users, while blog posts cover small jobs like fine-tuning Llama3 for role-playing.
Both: the README lists Amazon, NVIDIA and Aliyun among its users, while blog posts cover small jobs like fine-tuning Llama3 for role-playing.Does this run only on Linux servers?The README names no operating system requirement: it is a Python package for your own machines, with a documented Docker build route as an option.
The README names no operating system requirement: it is a Python package for your own machines, with a documented Docker build route as an option.How big a GPU do I need?It depends on the method, but the README cites fine-tuning a 70B model on two 24GB GPUs with FSDP+QLoRA, and smaller quantized setups fit consumer cards.
It depends on the method, but the README cites fine-tuning a 70B model on two 24GB GPUs with FSDP+QLoRA, and smaller quantized setups fit consumer cards.Which models can it fine-tune?LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3, Qwen3-VL, DeepSeek, Gemma, GLM and Phi are listed, and fresh releases get fast support: Qwen3 and Gemma 3 on day 0, Llama 4 on day 1.
LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3, Qwen3-VL, DeepSeek, Gemma, GLM and Phi are listed, and fresh releases get fast support: Qwen3 and Gemma 3 on day 0, Llama 4 on day 1.What training methods does it cover?Continuous pre-training, supervised fine-tuning, reward modeling and preference methods like PPO, DPO, KTO and ORPO, plus LoRA and algorithms such as GaLore, DoRA, LongLoRA and LLaMA Pro.
Continuous pre-training, supervised fine-tuning, reward modeling and preference methods like PPO, DPO, KTO and ORPO, plus LoRA and algorithms such as GaLore, DoRA, LongLoRA and LLaMA Pro.Can I serve the model after training?Yes: checkpoints can be exported to an OpenAI-style API, a Gradio UI or the CLI, sped up with vLLM or SGLang workers, and even saved as an Ollama modelfile.
Yes: checkpoints can be exported to an OpenAI-style API, a Gradio UI or the CLI, sped up with vLLM or SGLang workers, and even saved as an Ollama modelfile.