airbyte README

The project's README from github.com/airbytehq/airbyte. As one markdown file: README.md.

Airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications

Test Release Slack YouTube Channel Views Build License License

We believe that only an open-source solution to data movement can cover the long tail of data sources while empowering data engineers to customize existing connectors. Our ultimate vision is to help you move data from any source to any destination — whether that destination is a data warehouse, a data lake, or an AI agent. Airbyte provides a catalog of 600+ connectors for APIs, databases, data warehouses, data lakes, and AI applications.

Airbyte Connections UI Screenshot taken from Airbyte Cloud.

Pick the right Airbyte Platform for the job

Getting Started — Data Movement (ELT)

For moving data into warehouses, lakes, and databases:

Try it out yourself with our demo app, visit our full documentation, and learn more about recent announcements. See our registry for a full list of connectors already available in Airbyte or Airbyte Cloud.

Getting Started — AI Agents

For building AI agents that need real-time business data:

Join the Airbyte Community

The Airbyte community can be found in the Airbyte Community Slack, where you can ask questions and voice ideas. You can also ask for help in our Airbyte Forum. Airbyte's roadmap is publicly viewable on GitHub.

For videos and blogs on data engineering and building your data stack, check out Airbyte's Content Hub, YouTube, and sign up for our newsletter.

Contributing

If you've found a problem with Airbyte, please open a GitHub issue. To contribute to Airbyte and see our Code of Conduct, please see the contributing guide. We have a list of good first issues that contain bugs that have a relatively limited scope. This is a great place to get started, gain experience, and get familiar with our contribution process.

PR Permission Requirements

When submitting a pull request, please ensure that Airbyte maintainers have write access to your branch. This allows us to apply formatting fixes and dependency updates directly, significantly speeding up the review and approval process.

To enable write access on your PR from Airbyte maintainers, please check the "Allow edits from maintainers" box when submitting from your PR. You must also create your PR from a fork in your personal GitHub account rather than an organization account, or else you will not see this option. The requirement to create from your personal fork is based on GitHub's additional security restrictions for PRs created from organization forks. For more information about the GitHub security model, please see the GitHub documentation page regarding PRs from forks.

For more details on contribution requirements, please see our contribution workflow documentation.

Security

Airbyte takes security issues very seriously. Please do not file GitHub issues or post on our public forum for security vulnerabilities. Email [email protected] if you believe you have uncovered a vulnerability. In the message, try to provide a description of the issue and ideally a way of reproducing it. The security team will get back to you as soon as possible.

Airbyte Enterprise also offers additional security features (among others) on top of Airbyte open-source.

License

See the LICENSE file for licensing information, and our FAQ for any questions you may have on that topic.

Thank You

Airbyte would not be possible without the support and assistance of other open-source tools and companies! Visit our thank you page to learn more about how we build Airbyte.

airbyte (airbytehq/airbyte)

Searching Database…

22.2K ·GitHub
Airbyte
这是做什么的？Airbyte 是一个开源的数据迁移平台，用于 ELT 数据管道和 AI 智能体：它把数据从 API、数据库和文件搬运到数据仓库、数据湖和 AI 应用中，背后有 600 多个连接器的目录支撑。你可以在自己的环境里自托管运行，也可以使用 Airbyte Cloud。
这是需要我自己部署的东西，还是一项托管服务？两者都有。这个仓库是 Airbyte 开源版，由你自己部署和运行；Airbyte Cloud 则是托管的替代方案，另外还有托管的 Airbyte Agents 产品，以及开源的 Agent SDK。
我该怎么开始迁移数据？部署 Airbyte 开源版或设置 Airbyte Cloud，然后通过目录里的连接器来集中管理你的数据。你可以用演示应用先试一试，数据同步可以通过 Airflow、Dagster、Kestra 或 Airbyte API 来编排。
如果目录里没有我想要的数据源，我需要写代码吗？通常不需要：无代码的 Connector Builder 和低代码的 CDK 可以让你在几分钟内创建自定义连接器。如果要在智能体代码里使用连接器，则需要用 uv pip install airbyte-agent-sdk 单独安装 Agent SDK。
开源版包含这家公司提供的所有功能吗？不是：Airbyte Enterprise 在开源平台的基础上额外增加了安全等方面的功能。项目的发展方向可以在 GitHub 上的公开路线图里看到。
开箱即用的连接器有多少个？超过 600 个连接器，覆盖 API、数据库、数据仓库、数据湖和 AI 应用；项目的注册表列出了 Airbyte 和 Airbyte Cloud 中所有可用的连接器。
Airbyte 针对 AI 智能体具体提供了什么？Airbyte Agents 是一个托管的数据与上下文层，让智能体、LLM 和 MCP 客户端能实时访问业务数据。另外，开源的 Agent SDK 可以把连接器变成类型安全的 LLM 工具，兼容 pydantic-ai、LangChain、OpenAI Agents 和 FastMCP，并内置了重试、异常转换和输出大小限制等保护机制。
它为什么要开源？团队的官方立场是：只有开源的方式做数据迁移，才能覆盖长尾的数据源，同时让数据工程师对现有连接器进行定制。
What are similar projects?kestra, redash, walshadow, debezium, airflow, WrenAI, doris, anythingmcp
Which topics is it in?redshift, data-integration, change-data-capture, data-collection, snowflake, bigquery, data-pipeline, mssql, etl, pipeline, data, data-engineering
Creator
airbytehq
Size
922.6 MB
Stars
22.2K
Updated
CommitOct 17Releasev2.0.0Oct 15
Age
6y
Commits · 90 Days