gh-ost README

The project's README from github.com/github/gh-ost. As one markdown file: README.md.

gh-ost

ci replica-tests downloads release

GitHub's online schema migration for MySQL

gh-ost is a triggerless online schema migration solution for MySQL. It is testable and provides pausability, dynamic control/reconfiguration, auditing, and many operational perks.

gh-ost produces a light workload on the master throughout the migration, decoupled from the existing workload on the migrated table.

It has been designed based on years of experience with existing solutions, and changes the paradigm of table migrations.

How?

All existing online-schema-change tools operate in similar manner: they create a ghost table in the likeness of your original table, migrate that table while empty, slowly and incrementally copy data from your original table to the ghost table, meanwhile propagating ongoing changes (any INSERT, DELETE, UPDATE applied to your table) to the ghost table. Finally, at the right time, they replace your original table with the ghost table.

gh-ost uses the same pattern. However it differs from all existing tools by not using triggers. We have recognized the triggers to be the source of many limitations and risks.

Instead, gh-ost uses the binary log stream to capture table changes, and asynchronously applies them onto the ghost table. gh-ost takes upon itself some tasks that other tools leave for the database to perform. As result, gh-ost has greater control over the migration process; can truly suspend it; can truly decouple the migration's write load from the master's workload.

In addition, it offers many operational perks that make it safer, trustworthy and fun to use.

gh-ost general flow

Highlights

Please refer to the docs for more information. No, really, read the docs.

Usage

The cheatsheet has it all. You may be interested in invoking gh-ost in various modes:

Our tips:

More tips:

Also see:

What's in a name?

Originally this was named gh-osc: GitHub Online Schema Change, in the likes of Facebook online schema change and pt-online-schema-change.

But then a rare genetic mutation happened, and the c transformed into t. And that sent us down the path of trying to figure out a new acronym. gh-ost (pronounce: Ghost), stands for GitHub's Online Schema Transmogrifier/Translator/Transformer/Transfigurator

License

gh-ost is licensed under the MIT license

gh-ost uses 3rd party libraries, each with their own license. These are found here.

Community

gh-ost is released at a stable state, but with mileage to go. We are open to pull requests. Please first discuss your intentions via Issues.

We develop gh-ost at GitHub and for the community. We may have different priorities than others. From time to time we may suggest a contribution that is not on our immediate roadmap but which may appeal to others.

Please see Coding gh-ost for a guide to getting started developing with gh-ost.

Download/binaries/source

gh-ost is now GA and stable.

gh-ost is available in binary format for Linux and Mac OS/X

Download latest release here

gh-ost is a Go project; it is built with Go 1.15 and above. To build on your own, use either:

Generally speaking, master branch is stable, but only releases are to be used in production.

Authors

gh-ost is designed, authored, reviewed and tested by the database infrastructure team at GitHub:

gh-ost (github/gh-ost)

Searching Database…

13.6K ·GitHub
Gh-ost
What is this?gh-ost is GitHub's open-source online schema migration tool for MySQL: it alters live tables by copying data into a ghost table, keeping the workload on the master light and decoupled from normal traffic.
How is it different from other online schema-change tools?It copies your table into a ghost table and swaps them at the end, but is triggerless: it reads the binary log stream and applies changes asynchronously, which lets it truly pause and decouple its load from the master.
How do I get it and run a migration?Download a prebuilt binary for Linux or macOS, or build from source with Go 1.15+ via script/build. The cheatsheet covers all modes; the recommended flow is a noop test first, then the real migration with --execute.
What does my database setup need for this to work?A MySQL setup whose changes it can follow through the binary log. It prefers running migrations via a replica (it figures out the servers involved), and there are official guides for AWS RDS and Azure Database for MySQL.
Any gotchas before I point it at my production master?If your master uses Statement Based Replication, the replica-based mode is required. Also, only tagged releases are meant for production — the master branch is generally stable but not for prod use.
Is it proven on large production databases?Yes — it is GA and stable, built by GitHub's database infrastructure team, which continuously migrates its whole production table fleet on replicas and checksums the results to verify correctness.
What can I control while a migration is running?A lot: throttling truly ceases all writes on the master, you can reconfigure interactively mid-run, query status over a unix socket or TCP, postpone the final table swap until you are available, and couple it to your environment via external hooks.
Can I test it without touching production data?Yes: --test-on-replica runs the exact same migration flow on a replica without replacing the original table, leaving two tables you can compare and checksum. For master migrations, issue a noop first, then the real run with --execute.
Why is it called gh-ost?It is pronounced Ghost and stands for GitHub's Online Schema Transmogrifier/Translator/Transformer/Transfigurator. It began as gh-osc (Online Schema Change), in the likes of Facebook's OSC and pt-online-schema-change, until a 'rare genetic mutation' turned the c into a t.
What are similar projects?flyway, goose, usql, migrate, bytebase, chartdb, miller, dolt
Which topics is it in?devops-tools, mysql, sql, devops, database, go, command-line-tool, free, tool, cli, developer-tools, mit
Creator
github
Size
18.5 MB
Stars
13.6K
Updated
CommitOct 06Releasev1.1.11Aug 17
Age
10y
Commits · 90 Days