foundationdb README

The project's README from github.com/apple/foundationdb. As one markdown file: README.md.

FoundationDB logo

Build Status

FoundationDB is a distributed database designed to handle large volumes of structured data across clusters of commodity servers. It organizes data as an ordered key-value store and employs ACID transactions for all operations. It is especially well-suited for read/write workloads, but also has excellent performance for write-intensive workloads. Users interact with the database using API language bindings.

To learn more about FoundationDB, visit foundationdb.org

Documentation

Documentation can be found online at https://apple.github.io/foundationdb/. The documentation covers details of API usage, background information on design philosophy, and extensive usage examples. Docs are built from the source in this repo.

Forums

The FoundationDB Forums are the home for most of the discussion and communication about the FoundationDB project. We welcome your participation! We want FoundationDB to be a great project to be a part of, and as part of that, we have established a Code of Conduct to define what constitutes permissible modes of interaction.

Contributing

Contributions to FoundationDB can include contributions to the codebase, sharing your experience and insights with the community on the Forums, or contributing to projects that make use of FoundationDB. Please see the contributing guide for more specifics.

Getting Started

Latest Stable Releases

The latest stable releases are versions that are recommended for production use, which have been extensively validated via simulation and real cluster tests and used in our production environment.

Branch Latest Production Release Notes
7.3 7.3.77 Supported
7.2 Experimental
7.1 7.1.57 Bug fixes
7.0 Experimental
6.3 6.3.25 Unsupported

If you are running on old production releases, we recommend always upgrading to the next major release's latest version, and then continuing to the next major version, e.g., 6.2.X -> 6.3.25 -> 7.1.57 -> 7.3.69. These upgrade paths have been well tested in production (skipping a major release, not marked as Experimental, for an upgrade is only tested in simulation).

Binary Downloads

Developers interested in using FoundationDB can get started by downloading and installing a binary package. Please see the downloads page for a list of available packages.

Compiling from source

Developers on an OS for which there is no binary package, or who would like to start hacking on the code, can get started by compiling from source.

NOTE: FoundationDB has a lot of dependencies. The Docker container listed below tracks them and is what we use internally and is the recommended method of building FDB.

Build Using the Official Docker Image

The official Docker image for building is foundationdb/build, which includes all necessary dependencies. The Docker image definitions used by FoundationDB team members can be found in the dedicated repository.

To build FoundationDB with the clang toolchain,

mkdir /some/build_output_dir
cd /some/build_output_dir
CC=clang CXX=clang++ LD=lld cmake -D USE_LD=LLD -D USE_LIBCXX=1 -G Ninja /some/fdb/source_dir
ninja

To use GCC, a non-default version is necessary. The following modifies environment variables ($PATH, $LD_LIBRARY_PATH, etc) to pick up the right GCC version:

source /opt/rh/gcc-toolset-13/enable
gcc --version  # should say 13
mkdir /some/build_output_dir
cd /some/build_output_dir
cmake -G Ninja /some/fdb/source_dir
ninja

Slightly more elaborate compile commands can be found in the shell aliases defined in /root/.bashrc in the container image.

Build Locally

To build outside of the official Docker image, you'll need at least these dependencies:

  1. CMake version 3.24.2 or higher
  2. Mono
  3. ninja

This list is likely to be incomplete. Refer to the rockylinux9 Dockerfile in the fdb-build-support repo linked above for reference material on specific packages and versions that are likely to be required.

If compiling for local development, please set -DUSE_WERROR=ON in CMake. Our CI compiles with -Werror on, so this way you'll find out about compiler warnings that break the build earlier.

Once you have your dependencies, you can run cmake and then build:

  1. Check out this repository.
  2. Create a build directory (you can place it anywhere you like).
  3. cd <FDB_BUILD_DIR>
  4. cmake -G Ninja <FDB_SOURCE_DIR>
  5. ninja

Building FoundationDB requires at least 8GB of memory. More memory is needed when building in parallel. If the computer freezes or crashes, consider disabling parallelized build using ninja -j1.

FreeBSD

  1. Check out this repo on your server.

  2. Install compile-time dependencies from ports.

  3. (Optional) Use tmpfs & ccache for significantly faster repeat builds

  4. (Optional) Install a JDK for Java Bindings. FoundationDB currently builds with Java 8.

  5. Navigate to the directory where you checked out the FoundationDB repository.

  6. Build from source.

    sudo pkg install -r FreeBSD \
        shells/bash devel/cmake devel/ninja devel/ccache  \
        lang/mono lang/python3 \
        devel/boost-libs devel/libeio \
        security/openssl
    mkdir .build && cd .build
    cmake -G Ninja \
        -DUSE_CCACHE=on \
        -DUSE_DTRACE=off \
        ..
    ninja -j 10
    # run fast tests
    ctest -L fast
    # run all tests
    ctest --output-on-failure -v
    

macOS

The build under macOS will work the same way as on Linux. Homebrew can be used to install the boost library and the ninja build tool. Be careful: the current main branch uses Boost 1.86; install this version or let CMake download it. Also, if the Swift binding is not of interest, use -DBUILD_SWIFT_BINDING=OFF.

cmake -G Ninja <FDB_SOURCE_DIR> -B <BUILD_DIR>
cd <BUILD_DIR>
ninja

To generate an installable package,

<FDB_SOURCE_DIR>/packaging/osx/buildpkg.sh <BUILD_DIR> <FDB_SOURCE_DIR>

Windows

Under Windows, only Visual Studio with ClangCl is supported.

  1. Install Visual Studio 2019 (IDE or Build Tools), and enable LLVM support
  2. Install CMake 3.24.2 or higher
  3. Download Boost 1.86.0
  4. Unpack boost to C:\boost, or use -DBOOST_ROOT=<PATH_TO_BOOST> with cmake if unpacked elsewhere
  5. Install Python if it is not already installed by Visual Studio
  6. (Optional) Install OpenJDK 11 to build Java bindings
  7. (Optional) Install OpenSSL 3.x to build with TLS support
  8. (Optional) Install WIX Toolset to build the Windows installer
  9. mkdir build && cd build
  10. cmake -G "Visual Studio 16 2019" -A x64 -T ClangCl <FDB_SOURCE_DIR>
  11. msbuild /p:Configuration=Release foundationdb.sln
  12. To increase build performance, use /p:UseMultiToolTask=true and /p:CL_MPCount=<NUMBER_OF_PARALLEL_JOBS>

Language Bindings

The language bindings that CMake supports will have a corresponding README.md file in the bindings/lang directory corresponding to each language.

Generally, CMake will build all language bindings for which it can find all necessary dependencies. After each successful CMake run, CMake will tell you which language bindings it is going to build.

Generating compile_commands.json

CMake can generate a compilation database for code completion, navigation, and static analysis of the C++20 coroutine sources. Pass -DCMAKE_EXPORT_COMPILE_COMMANDS=ON when configuring a Ninja or Makefile build, then point your tooling at compile_commands.json in the build directory.

When Python support is enabled, this option also enables the processed_compile_commands target, which writes a database to the source directory. With Ninja, it includes Swift compilation commands as well.

If the build runs inside a container, the database contains container paths. Run the tooling in the same environment or map those paths to the host checkout.

Code Formatting and Static Analysis

clang-format and clang-tidy run as part of CI on every pull request. See the clang-format and clang-tidy guides for how to run them locally before pushing.

Using IDEs

CMake provides built-in support for several popular IDEs. FoundationDB's asynchronous code uses standard C++20 coroutines with the Flow runtime, so use an IDE or language server with C++20 support. See the coroutine guide for the programming model.

The CMake option OPEN_FOR_IDE creates an editing-only project for a supported IDE. This project cannot be built, but supports editing and navigation.

For example, if you want to use Xcode to make changes to FoundationDB, you can create an Xcode project with the following command:

cmake -G Xcode -DOPEN_FOR_IDE=ON <FDB_SOURCE_DIRECTORY>

A second build directory with the OPEN_FOR_IDE flag off can be created for building and debugging purposes.

foundationdb (apple/foundationdb)

Searching Database…

这是什么?FoundationDB 是一个分布式、支持事务的键值存储。它把数据组织成有序的键值映射,并能在由普通服务器组成的集群上提供 ACID 事务。
这是一个独立运行的数据库服务器,还是我需要嵌入使用的库?它是一个独立运行的分布式数据库服务器。应用通过各语言的 API 绑定(如 Python、Java、Go、Ruby、C 等)与它通信,而不是把它嵌入到应用中。
怎样才能让它跑起来?从你所用操作系统对应的下载页面下载预编译的二进制安装包,或使用 CMake 和 Ninja 从源码编译(官方 Docker 构建镜像是推荐的构建方式)。
构建或升级时有哪些需要注意的坑?构建至少需要 8GB 内存(并行构建时需要更多,在内存不足的机器上构建可能会卡死或崩溃)。另外,本地构建的依赖列表“很可能不完整”,因此推荐使用 Docker 镜像。
它是为哪类工作负载设计的?它面向大规模结构化数据的工作负载,在读写密集型以及写入密集型场景下表现尤其出色。
它支持哪些操作系统?它可以在 Linux、macOS、FreeBSD 和 Windows 上构建并运行(Windows 构建需要搭配 ClangCl 的 Visual Studio)。
异步 C++ 代码使用什么编程模型?FoundationDB 的异步代码使用标准的 C++20 协程,并基于项目自带的 Flow 运行时,因此编辑器需要支持 C++20。
我能生成 IDE 项目文件来修改源码吗?可以。CMake 的 OPENFORIDE 选项会生成一个仅用于编辑的项目(例如供 Xcode 使用),它支持代码导航和编辑,但无法编译;如果要实际构建,还需要单独的构建目录。
版本之间的升级是怎么进行的?像 6.3.25 升级到 7.1.57 再升级到 7.3.69 这样的特定升级路径已在生产环境中充分测试;而升级时跨越一个主版本的做法只在模拟环境中测试过。
在 macOS 上构建时,能否跳过 Swift 语言绑定?可以。如果不需要 Swift 绑定,在 macOS 上构建时传入 -DBUILDSWIFTBINDING=OFF 即可。
What are similar projects?cassandra, mysql-server, tidb, nebula, scylladb, oceanbase, citus, vitess
Creator
apple
Size
212.3 MB
Stars
16.8K
Age
8y
Commits · 90 Days