dtm README

The project's README from github.com/dtm-labs/dtm. As one markdown file: README.md.

license Build Status codecov Go Report Card Go Reference Mentioned in Awesome Go

English | 简体中文

Distributed Transactions Manager

What is DTM

DTM is a distributed transaction framework which provides cross-service eventual data consistency. It provides saga, tcc, xa, 2-phase message, outbox, workflow patterns for a variety of application scenarios. It also supports multiple languages and multiple store engine to form up a transaction as following:

function-picture

Who's using DTM (partial)

Tencent

Bytedance

Ivydad

More

Features

Application scenarios.

DTM can be applied to data consistency issues in a large number of scenarios, here are a few common ones

Cook Book

Quick start

run dtm

git clone https://github.com/dtm-labs/dtm && cd dtm
go run main.go

Start an example

Suppose we want to perform an inter-bank transfer. The operations of transfer out (TransOut) and transfer in (TransIn) are coded in separate micro-services.

Here is an example to illustrate a solution of dtm to this problem:

git clone https://github.com/dtm-labs/quick-start-sample.git && cd quick-start-sample/workflow-grpc
go run main.go

Code

Usage

wfName := "workflow-grpc"
err = workflow.Register(wfName, func(wf *workflow.Workflow, data []byte) error {
  // ...
  // Define a transaction branch for TransOut
  wf.NewBranch().OnRollback(func(bb *dtmcli.BranchBarrier) error {
    // compensation for TransOut
    _, err := busiCli.TransOutRevert(wf.Context, &req)
    return err
  })
  _, err = busiCli.TransOut(wf.Context, &req)
  // check error

  // Define another transaction branch for TransIn
  wf.NewBranch().OnRollback(func(bb *dtmcli.BranchBarrier) error {
    _, err := busiCli.TransInRevert(wf.Context, &req)
    return err
  })
  _, err = busiCli.TransIn(wf.Context, &req)
  return err
}

// ...
req := busi.BusiReq{Amount: 30, TransInResult: ""}
data, err := proto.Marshal(&req)

// Execute workflow
_, err = workflow.ExecuteCtx(wfName, shortuuid.New(), data)
logger.Infof("result of workflow.Execute is: %v", err)

When the above code runs, we can see in the console that services TransOut, TransIn has been called.

Rollback upon failure

If any forward operation fails, DTM invokes the corresponding compensating operation of each sub-transaction to roll back, after which the transaction is successfully rolled back.

Let's purposely trigger the failure of the second sub-transaction and watch what happens

// req := busi.BusiReq{Amount: 30, TransInResult: ""}
req := busi.BusiReq{Amount: 30, TransInResult: "FAILURE"}
})

we can see in the console that services TransOut, TransIn, TransOutRevert has been called

More examples

If you want more quick start examples, please refer to dtm-labs/quick-start-sample

The above example mainly demonstrates the flow of a distributed transaction. More on this, including practical examples of how to interact with an actual database, how to do compensation, how to do rollback, etc. please refer to dtm-examples for more examples.

Give a star! ⭐

If you think this project is interesting, or helpful to you, please give a star!

dtm (dtm-labs/dtm)

Searching Database…

10.9K ·GitHub
Dtm
What is this?DTM is a distributed transaction framework that provides cross-service eventual data consistency. It implements the saga, TCC, XA, 2-phase message, outbox and workflow patterns for different application scenarios.
Is it a library I import into my code, or a separate service I run?Both halves: a standalone dtm server processes the transactions, while your services talk to it through SDKs for Go, Java, PHP, C#, Python and Node.js.
How do I get a first transaction running?The README Quick start has you run the dtm server and then an inter-bank transfer example with TransOut and TransIn living in separate microservices; more samples live in the dtm-labs/quick-start-sample repo.
Which databases can it coordinate transactions across?MySQL/MariaDB, Redis, MongoDB, Postgres and TDSQL are supported for the transaction branches. For dtm's own storage, MySQL/MariaDB is the common choice, Redis the high-performance option, BoltDB for dev and test.
What happens if one step of a transaction fails midway?DTM invokes the corresponding compensating operation of each sub-transaction to roll everything back. The README demo purposely fails the second sub-transaction, and you see TransOutRevert being called.
Is it solid enough to bet a production system on?It is built for high availability and easy horizontal scaling, and the README lists Tencent, Bytedance and Ivydad among its users.
What is the 'better outbox' the README mentions?The 2-phase message pattern: the author positions it as a more elegant solution than the classic outbox, and it works across multiple databases.
What scenarios is it actually meant for?Cache consistency management, flash-sale inventory deduction in Redis that must exactly match the orders created, non-monolithic order systems, and event publishing/subscription.
Can I already use MongoDB as dtm's storage?Not yet: the README lists MongoDB as a storage engine under planning, so stick to MySQL/MariaDB or Redis for now.
Does it plug into particular Go microservice frameworks?Yes, it supports the go-zero, go-kratos/kratos and polarismesh/polaris microservice architectures out of the box.
What are similar projects?incubator-seata, conductor, sealos, pinpoint, emqx, tork, mcp-toolbox, chat
Which topics is it in?transactions, microservice, workflow-engine, distributed, high-availability, event-driven, mongodb, microservices, php, csharp, mysql, redis
Creator
dtm-labs
Size
3.4 MB
Stars
10.9K
Updated
CommitJan 03Releasev1.19.0Feb 04
Age
5y
Commits · 90 Days