txtai README

The project's README from github.com/neuml/txtai. As one markdown file: README.md.

All-in-one AI framework

Version GitHub last commit GitHub issues Join Slack Build Status Coverage Status

txtai is an all-in-one AI framework for semantic search, LLM orchestration and language model workflows.

architecture architecture

The key component of txtai is an embeddings database, which is a union of vector indexes (sparse and dense), graph networks and relational databases.

This foundation enables vector search and/or serves as a powerful knowledge source for large language model (LLM) applications.

Build autonomous agents, retrieval augmented generation (RAG) processes, multi-model workflows and more.

Summary of txtai features:

txtai is built with Python 3.10+, Hugging Face Transformers, Sentence Transformers and FastAPI. txtai is open-source under an Apache 2.0 license.

[!NOTE]

NeuML is the company behind txtai and we provide AI consulting services around our stack. Schedule a meeting or send a message to learn more.

We're also building an easy and secure way to run hosted txtai applications with txtai.cloud.

Why txtai?

why why

New vector databases, LLM frameworks and everything in between are sprouting up daily. Why build with txtai?

# Get started in a couple lines
import txtai

embeddings = txtai.Embeddings()
embeddings.index(["Correct", "Not what we hoped"])
embeddings.search("positive", 1)
#[(0, 0.29862046241760254)]

# app.yml
embeddings:
    path: sentence-transformers/all-MiniLM-L6-v2

CONFIG=app.yml uvicorn "txtai.api:app"
curl -X GET "http://localhost:8000/search?query=positive"

Use Cases

The following sections introduce common txtai use cases. A comprehensive set of over 70 example notebooks and applications are also available.

Build semantic/similarity/vector/neural search applications.

demo

Traditional search systems use keywords to find data. Semantic search has an understanding of natural language and identifies results that have the same meaning, not necessarily the same keywords.

search search

Get started with the following examples.

Notebook Description
Introducing txtai ▶️ Overview of the functionality provided by txtai Open In Colab
Similarity search with images Embed images and text into the same space for search Open In Colab
Build a QA database Question matching with semantic search Open In Colab
Semantic Graphs Explore topics, data connectivity and run network analysis Open In Colab

LLM Orchestration

Autonomous agents, retrieval augmented generation (RAG), chat with your data, pipelines and workflows that interface with large language models (LLMs).

llm

See below to learn more.

Notebook Description
Prompt templates and task chains Build model prompts and connect tasks together with workflows Open In Colab
Integrate LLM frameworks Integrate llama.cpp, LiteLLM and custom generation frameworks Open In Colab
Build knowledge graphs with LLMs Build knowledge graphs with LLM-driven entity extraction Open In Colab
Parsing the stars with txtai Explore an astronomical knowledge graph of known stars, planets, galaxies Open In Colab

Agents

Agents connect embeddings, pipelines, workflows and other agents together to autonomously solve complex problems.

agent

txtai agents are built on top of the smolagents framework. This supports all LLMs txtai supports (Hugging Face, llama.cpp, OpenAI / Claude / AWS Bedrock via LiteLLM). Agent prompting with agents.md and skill.md are also supported.

Check out this Agent Quickstart Example. Additional examples are listed below.

Notebook Description
Granting autonomy to agents Agents that iteratively solve problems as they see fit Open In Colab
TxtAI got skills Integrate skill.md files with your agent Open In Colab
Agent Tools ▶️ Learn about the txtai agent toolkit Open In Colab
Analyzing LinkedIn Company Posts with Graphs and Agents Exploring how to improve social media engagement with AI Open In Colab

Retrieval augmented generation

Retrieval augmented generation (RAG) reduces the risk of LLM hallucinations by constraining the output with a knowledge base as context. RAG is commonly used to "chat with your data".

rag rag

Check out this RAG Quickstart Example. Additional examples are listed below.

Notebook Description
Build RAG pipelines with txtai ▶️ Guide on retrieval augmented generation including how to create citations Open In Colab
RAG is more than Vector Search Context retrieval via Web, SQL and other sources Open In Colab
GraphRAG with Wikipedia and GPT OSS Deep graph search powered RAG Open In Colab
Speech to Speech RAG ▶️ Full cycle speech to speech workflow with RAG Open In Colab

Language Model Workflows

Language model workflows, also known as semantic workflows, connect language models together to build intelligent applications.

flows flows

While LLMs are powerful, there are plenty of smaller, more specialized models that work better and faster for specific tasks. This includes models for extractive question-answering, automatic summarization, text-to-speech, transcription and translation.

Check out this Workflow Quickstart Example. Additional examples are listed below.

Notebook Description
Run pipeline workflows ▶️ Simple yet powerful constructs to efficiently process data Open In Colab
Building abstractive text summaries Run abstractive text summarization Open In Colab
Transcribe audio to text Convert audio files to text Open In Colab
Translate text between languages Streamline machine translation and language detection Open In Colab

Installation

install install

The easiest way to install is via pip and PyPI

pip install txtai

Python 3.10+ is supported. Using a Python virtual environment is recommended.

See the detailed install instructions for more information covering optional dependencies, environment specific prerequisites, installing from source, conda support, lightweight minimal installation and how to run with containers.

Model guide

models

See the table below for the current recommended models. These models all allow commercial use and offer a blend of speed and performance.

Component Model(s)
Embeddings all-MiniLM-L6-v2
Image Captions BLIP
Labels - Zero Shot DeBERTa v3 Zeroshot
Labels - Fixed Fine-tune with training pipeline
Large Language Model (LLM) Gemma 4 31B
Summarization DistilBART
Text-to-Speech ESPnet JETS
Transcription Whisper
Translation OPUS Model Series

Models can be loaded as either a path from the Hugging Face Hub or a local directory. Model paths are optional, defaults are loaded when not specified. For tasks with no recommended model, txtai uses the default models as shown in the Hugging Face Tasks guide.

See the following links to learn more.

Powered by txtai

The following applications are powered by txtai.

apps

Application Description
[rag](https://github.com/n

txtai (neuml/txtai)

Searching Database…

13K ·GitHub
Txtai
What is this?txtai is an all-in-one AI framework for semantic search, LLM orchestration and language model workflows. Its core is an embeddings database that combines vector indexes, graph networks and relational databases to power search or serve as a knowledge source for LLM apps.
Is this a vector database, an LLM framework, or both?Both: it's a Python framework built around an embeddings database (vector + sparse indexes, graph, and relational storage) plus pipelines, workflows and agents for LLM orchestration.
How do I install and start using it?Install via pip from PyPI (Python 3.10+ required), ideally inside a virtual environment; Docker containers are also supported for running it.
What libraries does it build on, and do I need an API key for any LLM?It's built on Python 3.10+, Hugging Face Transformers, Sentence Transformers and FastAPI; LLMs can be run locally (e.g. llama.cpp, Hugging Face models) or via external APIs (OpenAI, Claude, AWS Bedrock through LiteLLM), so an API key is only needed if you choose a hosted model.
Can I keep everything local instead of sending data to external services?Yes, txtai is designed to run fully local without shipping data to remote services, though it can also connect to cloud LLM APIs if you prefer.
What can txtai's agents actually do?Agents connect embeddings, pipelines, workflows and other agents to autonomously solve complex problems; they're built on the smolagents framework and support agents.md/skill.md prompting plus any LLM txtai supports.
How does txtai help with RAG beyond plain vector search?Its retrieval augmented generation pipelines constrain LLM output with a knowledge base as context (reducing hallucinations) and can pull context from web, SQL and other sources, not just vector search, including citation generation.
What model does txtai recommend for the LLM component?The current model guide recommends Gemma 4 31B for the large language model component, alongside models like all-MiniLM-L6-v2 for embeddings and Whisper for transcription.
What other languages can use txtai besides Python?txtai exposes Web and Model Context Protocol (MCP) APIs with bindings available for JavaScript, Java, Rust and Go, so it isn't limited to Python.
What are similar projects?weaviate, haystack, m_flow, chroma, LLMSearchRecommender, FlagEmbedding, LightRAG, ai-system-design
Which topics is it in?vector-search-engine, information-retrieval, language-model, search-engine, retrieval-augmented-generation, vector-search, knowledge-graph, transformers, vector-database, semantic-search, embeddings, agent-framework
Creator
neuml
Size
55.5 MB
Stars
13K
Updated
CommitOct 16Releasev9.14.0Oct 06
Age
6y
Commits · 90 Days