Benchmarks
SlateDB currently has two benchmarking tools: bencher and microbenchmarks.
-
bencher is a tool to benchmark put/get operations against an object store. You can configure the tool with a variety of options. See bencher/README.md for details. benchmark-db.sh also serves as an example.
-
Microbenchmarks run with Criterion. They are located in benches and run for specific internal SlateDB functions. A comment is left on all PRs when a > 200% slowdown is detected.
Nightly Benchmarks
Section titled “Nightly Benchmarks”We run both bencher and microbenchmarks nightly. The results are published in the Github action summary.
-
Bencher benchmarks run on WarpBuild’s warp-ubuntu-latest-x64-16x runners, which use Hetzner machines in Frankfurt. We use Tigris for object storage with the
autoregion setting, which resolves to Frankfurt as well. Bandwidth between WarpBuild (Hetzner) and Tigris seems to be about 500MiB/s down and 130MiB/s up. We routinely max out the bandwidth in the nightly tests. -
Microbenchmarks run on standard Linux Github action runners with the pprof-rs profiler. The resulting profiler protobuf files are published to pprof.me and links to each microbenchmark are provided in the Github action summary.