Performance

We currently have two benchmarking tools: bencher and microbenchmarks.

Benchmarks

SlateDB comes with a bencher tool to benchmark put/get operations against an object store. You can configure the tool with a variety of options. See bencher/README.md for details. benchmark-db.sh also serves as an example.

Nightly

We run bencher nightly and publish the results here. The benchmark runs on WarpBuild's warp-ubuntu-latest-x64-16x runners, which use Hetzner machines in Frankfurt. We use Tigris for object storage with the auto region setting, which resolves to Frankfurt as well. Bandwidth between WarpBuild (Hetzner) and Tigris seems to be about 500MiB/s down and 130MiB/s up. We routinely max out the bandwidth in the nightly tests.

Microbenchmarks

We use Criterion to run microbenchmarks (located in benches) for specific internal SlateDB functions. A comment is left on all PRs when a > 200% slowdown is detected.

Nightly

Microbenchmarks also run nightly with the pprof-rs profiler. The resulting profiler protobuf files are published here. The microbenchmarks run on standard Linux Github action runners.

We highly recommend using pprof.me to view the <microbenchmar>.pb files, though any pprof compatible tool may be used.