Run a Standalone Compactor

By default, SlateDB starts a background compactor inside each Db instance. If you want to run compaction as a separate service (for example, on dedicated compute), you can:

Open the Db with compaction disabled.
Start a standalone compactor in a separate process.

Both processes must use the same ObjectStore and database path (the path is the prefix inside your object store).

Create a Db with no compactor

To prevent Db from starting the embedded compactor, set Settings::compactor_options to None and use Db::builder:

use slatedb::config::Settings;
use slatedb::object_store::local::LocalFileSystem;
use slatedb::Db;
use std::sync::Arc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // This example uses a local object store so it can be shared with a separate
    // compactor process. In production, this could also be an object store like S3.
    let local_root = std::env::temp_dir().join("slatedb-standalone-compactor-tutorial");
    std::fs::create_dir_all(&local_root)?;
    let object_store = Arc::new(LocalFileSystem::new_with_prefix(local_root)?);

    // Disable the embedded compactor by clearing compactor options.
    let settings = Settings {
        compactor_options: None,
        ..Default::default()
    };

    let db = Db::builder("db", object_store)
        .with_settings(settings)
        .build()
        .await?;

    db.put(b"hello", b"world").await?;
    db.flush().await?;
    db.close().await?;

    Ok(())
}

Run a compactor without a Db

You can run the standalone compactor using the SlateDB CLI:

slatedb --env-file .env --path <db-path> run-compactor

The compactor runs until interrupted (Ctrl-C), then shuts down gracefully.

Alternatively, you can embed the compactor in your own process using CompactorBuilder and call run:

use slatedb::object_store::local::LocalFileSystem;
use slatedb::CompactorBuilder;
use std::sync::Arc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let local_root = std::env::temp_dir().join("slatedb-standalone-compactor-tutorial");
    std::fs::create_dir_all(&local_root)?;
    let object_store = Arc::new(LocalFileSystem::new_with_prefix(local_root)?);

    // Build a compactor without opening a Db. The database must already exist (i.e.
    // the manifest has been created).
    let compactor = Arc::new(CompactorBuilder::new("db", object_store).build());

    let compactor_task = {
        let compactor = Arc::clone(&compactor);
        tokio::spawn(async move { compactor.run().await })
    };

    println!("Compactor running. Press Ctrl-C to stop.");
    tokio::signal::ctrl_c().await?;

    compactor.stop().await?;
    compactor_task.await??;

    Ok(())
}