What is this?
This repo provides ready-to-run cloud templates for RAG pipelines, AI pipelines, and enterprise search, built on the Pathway Live Data Framework and kept in sync with live data sources.
This repo provides ready-to-run cloud templates for RAG pipelines, AI pipelines, and enterprise search, built on the Pathway Live Data Framework and kept in sync with live data sources.Is this one app or a collection of templates?It is a collection of separate application templates (Question-Answering RAG, Live Document Indexing, Multimodal RAG, Unstructured-to-SQL, and more), each runnable on its own and built on top of the Pathway Python/Rust engine.
It is a collection of separate application templates (Question-Answering RAG, Live Document Indexing, Multimodal RAG, Unstructured-to-SQL, and more), each runnable on its own and built on top of the Pathway Python/Rust engine.How do I get one of these templates running?Each template folder has its own README.md with run instructions; the apps run as Docker containers and expose an HTTP API that a frontend or Streamlit UI can call.
Each template folder has its own README.md with run instructions; the apps run as Docker containers and expose an HTTP API that a frontend or Streamlit UI can call.Do I need an OpenAI API key for these templates?Most templates call GPT models and need an API key, but the 'Private RAG App with Mistral and Ollama' template runs fully locally without any external LLM key.
Most templates call GPT models and need an API key, but the 'Private RAG App with Mistral and Ollama' template runs fully locally without any external LLM key.Do I still need to set up my own vector database and cache?No, Pathway bundles vector search, hybrid search, and full-text indexing in-memory, so you don't need to separately wire up a vector DB like Pinecone, a cache like Redis, or an API framework.
No, Pathway bundles vector search, hybrid search, and full-text indexing in-memory, so you don't need to separately wire up a vector DB like Pinecone, a cache like Redis, or an API framework.Can I run any of these templates fully offline with my own data staying local?Yes, the Private RAG App template is designed to keep everything on your machine using Mistral and Ollama instead of a cloud LLM provider.
Yes, the Private RAG App template is designed to keep everything on your machine using Mistral and Ollama instead of a cloud LLM provider.What is Pathway, and how does it relate to llm-app?Pathway is the underlying Python library with a Rust engine that powers live data sync, built-in vector/hybrid indexing, and API serving for every template in this repo.
Pathway is the underlying Python library with a Rust engine that powers live data sync, built-in vector/hybrid indexing, and API serving for every template in this repo.Which data sources can these templates stay synced with?Local files, Google Drive, Sharepoint, S3, Kafka, PostgreSQL, and real-time data APIs, with automatic detection of additions, deletions, and updates.
Local files, Google Drive, Sharepoint, S3, Kafka, PostgreSQL, and real-time data APIs, with automatic detection of additions, deletions, and updates.What makes the Multimodal RAG with GPT4o template different?It uses GPT-4o during parsing to extract tables and charts from PDFs and other unstructured documents, updating results live as source files change.
It uses GPT-4o during parsing to extract tables and charts from PDFs and other unstructured documents, updating results live as source files change.Can these templates be deployed to the cloud?Yes, you can test locally and deploy to GCP, AWS, Azure, Render, or run them on-premises.
Yes, you can test locally and deploy to GCP, AWS, Azure, Render, or run them on-premises.What are similar projects?Langchain-Chatchat, langchain4j, m_flow, production-agentic-rag-course, weaviate, agentic-rag-for-dummies, vault-ai, pathway
Langchain-Chatchat, langchain4j, m_flow, production-agentic-rag-course, weaviate, agentic-rag-for-dummies, vault-ai, pathwayWhich topics is it in?enterprise-search, pinecone, milvus, chromadb, google-drive, pgvector, mistral, streamlit, duckdb, llmops, ai-security, kafka
enterprise-search, pinecone, milvus, chromadb, google-drive, pgvector, mistral, streamlit, duckdb, llmops, ai-security, kafka