dolly README

The project's README from github.com/databrickslabs/dolly. As one markdown file: README.md.

Dolly

Databricks’ Dolly is an instruction-following large language model trained on the Databricks machine learning platform that is licensed for commercial use. Based on pythia-12b, Dolly is trained on ~15k instruction/response fine tuning records databricks-dolly-15k generated by Databricks employees in capability domains from the InstructGPT paper, including brainstorming, classification, closed QA, generation, information extraction, open QA and summarization. dolly-v2-12b is not a state-of-the-art model, but does exhibit surprisingly high quality instruction following behavior not characteristic of the foundation model on which it is based.

Databricks is committed to ensuring that every organization and individual benefits from the transformative power of artificial intelligence. The Dolly model family represents our first steps along this journey, and we’re excited to share this technology with the world.

The model is available on Hugging Face as databricks/dolly-v2-12b.

Model Overview

dolly-v2-12b is a 12 billion parameter causal language model created by Databricks that is derived from EleutherAI’s Pythia-12b and fine-tuned on a ~15K record instruction corpus generated by Databricks employees and released under a permissive license (CC-BY-SA)

Known Limitations

Performance Limitations

dolly-v2-12b is not a state-of-the-art generative language model and, though quantitative benchmarking is ongoing, is not designed to perform competitively with more modern model architectures or models subject to larger pretraining corpuses.

The Dolly model family is under active development, and so any list of shortcomings is unlikely to be exhaustive, but we include known limitations and misfires here as a means to document and share our preliminary findings with the community. In particular, dolly-v2-12b struggles with: syntactically complex prompts, programming problems, mathematical operations, factual errors, dates and times, open-ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc. Moreover, we find that dolly-v2-12b does not have some capabilities, such as well-formatted letter writing, present in the original model.

Dataset Limitations

Like all language models, dolly-v2-12b reflects the content and limitations of its training corpuses.

Databricks is committed to ongoing research and development efforts to develop helpful, honest and harmless AI technologies that maximize the potential of all individuals and organizations.

Getting Started with Response Generation

If you'd like to simply test the model without training, the model is available on Hugging Face as databricks/dolly-v2-12b.

To use the model with the transformers library on a machine with A100 GPUs:

from transformers import pipeline
import torch

instruct_pipeline = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

You can then use the pipeline to answer instructions:

instruct_pipeline("Explain to me the difference between nuclear fission and fusion.")

Generating on Other Instances

A100 instance types are not available in all cloud regions, or can be hard to provision. Inference is possible on other GPU instance types.

A10 GPUs

The 6.9B and 2.8B param models should work as-is.

To generate using the 12B param model on A10s (ex: g5.4xlarge, 1 x A10 24GB), it's necessary to load and run generating using 8-bit weights, which impacts the results slightly:

V100 GPUs

When using V100s (ex: p3.2xlarge, 1 x V100 16GB, NC6s_v3), in all cases, set torch_dtype=torch.float16 in pipeline() instead.

Otherwise, follow the steps above. The 12B param model may not function well in 8-bit on V100s.

Getting Started with Training

Training on Other Instances

A100 instance types are not available in all cloud regions, or can be hard to provision. Training is possible on other GPU instance types, for smaller Dolly model sizes, and with small modifications to reduce memory usage. These modifications are not optimal, but are simple to make.

Select your GPU family type from the gpu_family widget, enter the number of GPUs available in the num_gpus widget, and then run the rest of the code. A number of different options will be set for you to train the model for one of the following GPU types:

Details of the different configurations are below.

A100 GPUs

A100 GPUs are preferred for training all model sizes, and are the only GPUs that can train the 12B param model in a reasonable amount of time. As such, this is the default configuration, as set in the a100_config.json deepspeed config file.

A10 GPUs

Training the 12B param model is not recommended on A10s.

To train the 6.9B param model on A10 instances (ex: g5.24xlarge, 4 x A10 24GB; Standard_NV72ads_A10_v5, 2 x A10), simply select a10 from the gpu_family widget and enter the number of GPUs available in the num_gpus widget, then run the rest of the code. This will use the a10_config.json deepspeed config file, which makes the following changes:

V100 GPUs

To run on V100 instances with 32GB of GPU memory (ex: p3dn.24xlarge or Standard_ND40rs_v2), simply select v100 from the gpu_family widget and enter the number of GPUs available in the num_gpus widget, and then run the rest of the code. This will use the v100_config.json deepspeed config file, which makes the following changes:

You may be able to slightly increase the batch size with 32GB instances, compared to what works above for 24GB A10s.

Running Unit Tests Locally

pyenv local 3.8.13
python -m venv .venv
. .venv/bin/activate
pip install -r requirements_dev.txt
./run_pytest.sh

Citation

@online{DatabricksBlog2023DollyV2,
    author    = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin},
    title     = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM},
    year      = {2023},
    url       = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm},
    urldate   = {2023-06-30}
}

dolly (databrickslabs/dolly)

Searching Database…

10.8K ·GitHub
Dolly
What is this?Dolly is Databricks' instruction-following large language model, trained on the Databricks Machine Learning Platform and licensed for commercial use; the flagship dolly-v2-12b derives from EleutherAI's Pythia-12b.
Is this a finished model I can download, or code for training one?Both: ready-to-use models come in 12B, 6.9B and 2.8B parameter sizes on Hugging Face, while this repo contains the training notebook and the databricks-dolly-15k dataset.
How do I try it out without training anything?Pull the model from Hugging Face and run it through a transformers pipeline on a GPU machine — the README's quickstart does exactly this, no training needed.
So where does it fall short?Quite a lot: the README lists struggles with syntactically complex prompts, programming problems, math, dates and times, open-ended QA, hallucination and even a sense of humor.
How clean is that training data?Not spotless: the base GPT-J corpus (The Pile) contains much objectionable web content the model can reflect, and the 15k set may hold typos, factual errors and Wikipedia's biases.
Can I use it commercially?Yes — the README states Dolly is licensed for commercial use (Apache-2.0 on GitHub), and the fine-tuning corpus itself is released under the permissive CC-BY-SA.
What GPU do I need to run it?The 12B model runs as-is on A100 GPUs. On a 24GB A10 you must load 8-bit weights (install bitsandbytes, load_in_8bit=True); on V100s use torch_dtype=torch.float16 instead.
Why did Databricks build and open-source Dolly?The company frames it as a step toward ensuring every organization and individual can benefit from AI; the Dolly family is called its first steps on that journey, shared with the world.
Is dolly-v2-12b as good as newer chat models?No. The README says outright it is not a state-of-the-art generative model and is not designed to perform competitively with more modern architectures or models with larger pretraining corpora.
Did fine-tuning cost the base model any abilities?Yes — the README notes dolly-v2-12b lacks some capabilities present in the original model, such as well-formatted letter writing.
What are similar projects?train-llm-from-scratch, petals, nanoGPT, Intern-Decision, stanford_alpaca, pointdit, ChatGLM2-6B, airllm
Which topics is it in?databricks, instruction-tuning, text-generation, spark, model-training, transformers, dataset, gpt, fine-tuning, chatbot, inference, nvidia
Creator
databrickslabs
Size
0.1 MB
Stars
10.8K
Updated
CommitJun 30
Age
3y
Commits · 90 Days