ollama-python README

The project's README from github.com/ollama/ollama-python. As one markdown file: README.md.

Ollama Python Library

The Ollama Python library provides the easiest way to integrate Python 3.8+ projects with Ollama.

Prerequisites

Install

pip install ollama

Usage

from ollama import chat
from ollama import ChatResponse

response: ChatResponse = chat(model='gemma3', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])
# or access fields directly from the response object
print(response.message.content)

See _types.py for more information on the response types.

Streaming responses

Response streaming can be enabled by setting stream=True.

from ollama import chat

stream = chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': 'Why is the sky blue?'}],
    stream=True,
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

Cloud Models

Run larger models by offloading to Ollama’s cloud while keeping your local workflow.

Run via local Ollama

  1. Sign in (one-time):
ollama signin
  1. Pull a cloud model:
ollama pull gpt-oss:120b-cloud
  1. Make a request:
from ollama import Client

client = Client()

messages = [
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
]

for part in client.chat('gpt-oss:120b-cloud', messages=messages, stream=True):
  print(part.message.content, end='', flush=True)

Cloud API (ollama.com)

Access cloud models directly by pointing the client at https://ollama.com.

  1. Create an API key from ollama.com , then set:
export OLLAMA_API_KEY=your_api_key
  1. (Optional) List models available via the API:
curl https://ollama.com/api/tags
  1. Generate a response via the cloud API:
import os
from ollama import Client

client = Client(
    host='https://ollama.com',
    headers={'Authorization': 'Bearer ' + os.environ.get('OLLAMA_API_KEY')}
)

messages = [
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
]

for part in client.chat('gpt-oss:120b', messages=messages, stream=True):
  print(part.message.content, end='', flush=True)

Custom client

A custom client can be created by instantiating Client or AsyncClient from ollama.

All extra keyword arguments are passed into the httpx.Client.

from ollama import Client
client = Client(
  host='http://localhost:11434',
  headers={'x-some-header': 'some-value'}
)
response = client.chat(model='gemma3', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])

Async client

The AsyncClient class is used to make asynchronous requests. It can be configured with the same fields as the Client class.

import asyncio
from ollama import AsyncClient

async def chat():
  message = {'role': 'user', 'content': 'Why is the sky blue?'}
  response = await AsyncClient().chat(model='gemma3', messages=[message])

asyncio.run(chat())

Setting stream=True modifies functions to return a Python asynchronous generator:

import asyncio
from ollama import AsyncClient

async def chat():
  message = {'role': 'user', 'content': 'Why is the sky blue?'}
  async for part in await AsyncClient().chat(model='gemma3', messages=[message], stream=True):
    print(part['message']['content'], end='', flush=True)

asyncio.run(chat())

API

The Ollama Python library's API is designed around the Ollama REST API

Chat

ollama.chat(model='gemma3', messages=[{'role': 'user', 'content': 'Why is the sky blue?'}])

Generate

ollama.generate(model='gemma3', prompt='Why is the sky blue?')

List

ollama.list()

Show

ollama.show('gemma3')

Create

ollama.create(model='example', from_='gemma3', system="You are Mario from Super Mario Bros.")

Copy

ollama.copy('gemma3', 'user/gemma3')

Delete

ollama.delete('gemma3')

Pull

ollama.pull('gemma3')

Push

ollama.push('user/gemma3')

Embed

ollama.embed(model='gemma3', input='The sky is blue because of rayleigh scattering')

Embed (batch)

ollama.embed(model='gemma3', input=['The sky is blue because of rayleigh scattering', 'Grass is green because of chlorophyll'])

Ps

ollama.ps()

Errors

Errors are raised if requests return an error status or if an error is detected while streaming.

model = 'does-not-yet-exist'

try:
  ollama.chat(model)
except ollama.ResponseError as e:
  print('Error:', e.error)
  if e.status_code == 404:
    ollama.pull(model)

ollama-python (ollama/ollama-python)

Searching Database…

10.6K ·GitHub
Ollama-python
What is this?This is the official Python library for Ollama, giving Python 3.8+ projects an easy way to call a local or cloud Ollama server to run large language models.
Is this a library or a standalone app?It is a Python client library (SDK) that wraps the Ollama REST API, with a sync Client and an async Client class.
How do I add this to my project?Install the package with pip, then import ollama in your Python code to call methods like chat, generate, or embed, which map to the Ollama REST API.
What do I need before using this?You need Ollama installed and running locally, plus a model pulled with a command like 'ollama pull gemma3' before making calls from the library.
Can I use this without a local Ollama install?Yes — besides the local server, you can use Ollama's cloud models either by signing in through local Ollama and pulling a cloud model, or by pointing the client directly at ollama.com with an API key.
Does it support streaming responses?Yes, setting stream=True enables streaming; with AsyncClient this returns a Python asynchronous generator.
What operations does the API cover?It covers Chat, Generate, List, Show, Create, Copy, Delete, Pull, Push, Embed (including batch embed), and Ps for listing running models.
How are errors handled?Errors are raised as Python exceptions if a request returns an error status or if an error is detected while streaming a response.
What cloud models are supported?Supported cloud models include deepseek-v3.1:671b-cloud, gpt-oss:20b-cloud, gpt-oss:120b-cloud, kimi-k2:1t-cloud, qwen3-coder:480b-cloud, and kimi-k2-thinking.
What are similar projects?openai-python, production-agentic-rag-course, ollama, localGPT, go-openai, guzzle, Rapid-MLX, OpenLLM
Which topics is it in?gpt-oss, http-client, async, llm-inference, embeddings, local-llm, ollama, chatbot, streaming, sdk, rest-api, cloud
Creator
ollama
Size
0.6 MB
Stars
10.6K
Updated
CommitOct 16Releasev0.6.3Sep 29
Age
2y
Commits · 90 Days