Trait CompactionService 

pub trait CompactionService: Send + Sync {
    // Required methods
    fn name(&self) -> &CStr;
    fn schedule(
        &self,
        info: &CompactionServiceJobInfo<'_>,
        input: &[u8],
    ) -> ScheduleResponse;
    fn wait(
        &self,
        scheduled_job_id: &CStr,
        result: &mut Vec<u8>,
    ) -> CompactionServiceJobStatus;

    // Provided methods
    fn cancel_awaiting_jobs(&self) { ... }
    fn on_installation(
        &self,
        _scheduled_job_id: &CStr,
        _status: Option<CompactionServiceJobStatus>,
    ) { ... }
}

Somewhere for the primary DB to send its compactions.

Install one with Options::set_compaction_service. RocksDB then calls schedule instead of compacting, waits in wait for the result, and installs the output files the worker wrote.

Threading

The methods take &self and the trait requires Send + Sync, because RocksDB calls them from wherever it happens to be. Each subcompaction calls schedule and wait on its own background compaction thread (compaction_service_job.cc:86 and compaction_service_job.cc:133), so several can be in flight at once, and CancelAllBackgroundWork calls cancel_awaiting_jobs from the caller’s thread while those are still blocked (db_impl.cc:584). One service can also back several DBs, since Options can be cloned. Reach for a Mutex or an atomic for anything the service needs to accumulate.

Panics

These methods are called from C++ across an extern "C" boundary, where an unwind is undefined behaviour. A panic that escapes any of them aborts the process instead. Report a problem with CompactionServiceJobStatus::Failure, or with UseLocal to have the primary DB do the work itself.

Required Methods

fn name(&self) -> &CStr

Identifies this service in the LOG file.

Read once, when the service is installed, and copied into a std::string on the C++ side (c.cc:1271), so the pointer behind the returned CStr does not have to outlive that call.

fn schedule( &self, info: &CompactionServiceJobInfo<'_>, input: &[u8], ) -> ScheduleResponse

Sends a compaction to the worker fleet.

input is the serialized job. It is opaque, it is binary rather than text, and it is the only thing the worker needs in order to run the compaction: hand exactly these bytes to open_and_compact on the other side. It is borrowed from a std::string for the length of this call (c.cc:1295), so copy it before returning.

Return ScheduleResponse::scheduled with Success and a job id to have RocksDB go on and call wait with that id. Anything else ends the attempt, and only UseLocal makes the primary run the compaction itself.

fn wait( &self, scheduled_job_id: &CStr, result: &mut Vec<u8>, ) -> CompactionServiceJobStatus

Blocks until the job scheduled under scheduled_job_id finishes.

Write the bytes open_and_compact returned into result, which starts empty. RocksDB copies them out and frees the copy this crate makes (c.cc:1321 and c.cc:1323).

A result is read for Success and for Failure, where RocksDB pulls the remote Status out of it to explain what went wrong. It is ignored for the other two.

This blocks a background compaction thread for as long as it runs.

Provided Methods

fn cancel_awaiting_jobs(&self)

Drops every job this service is still waiting on.

Called from CancelAllBackgroundWork, which runs on DB shutdown, while wait calls are still blocked on other threads. Upstream notes in compaction_service_job.cc:118 that there is currently no way to signal an abort to a job that is already running remotely, so this is about not waiting for them rather than stopping them.

fn on_installation( &self, _scheduled_job_id: &CStr, _status: Option<CompactionServiceJobStatus>, )

Reports what the primary DB did with a finished job’s output.

Success means the output files were renamed into the DB and installed. Failure means the install failed part way through. UseLocal means the primary could not read the result and is redoing the compaction itself, leaving the worker’s output untouched in the staging directory. status is None for a value this crate does not name.

This is where a worker learns it can delete a job’s staged output.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors