Struct SyncBatcher 

pub struct SyncBatcher<Req, Res, E> { /* private fields */ }

A batcher for synchronous workloads with thread-local resources.

Unlike crate::Batcher, which runs an async process closure on the tokio runtime, SyncBatcher spawns a dedicated OS thread that:

1. Initializes resources once via the init closure (e.g., TextEmbedding)
2. Processes batches synchronously using those resources

This avoids spawn_blocking overhead for each request and keeps thread-local state alive on the worker thread.

Example

let config = BatcherConfig::default();

let batcher: SyncBatcher<String, Embedding, String> = SyncBatcher::new(config, || {
    let embedding = TextEmbedding::try_new(()).unwrap();
     
    move |items: Vec<String>| {
        embedding.embed(items)
    }
});

let result = batcher.run("hello".to_string()).await?;

Implementations

impl<Req, Res, E> SyncBatcher<Req, Res, E>

where Req: Send + 'static, Res: Send + 'static, E: Clone + Send + 'static,

pub fn new<F, G>(config: BatcherConfig, init: F) -> Self

where F: FnOnce() -> G + Send + 'static, G: FnMut(Vec<Req>) -> Result<Vec<Res>, E> + Send + 'static,

Spawn the sync batch worker on a dedicated thread.

init runs once on the worker thread and returns the processing closure. This is where you initialize thread-local resources (e.g., TextEmbedding).

The processing closure receives a Vec<Req> and must return Result<Vec<Res>, E> synchronously.

pub async fn run(&self, item: Req) -> Result<Res, E>

Submit one item for processing.

Waits until the worker has processed the batch containing this item, then returns its result.

Returns Err(E) if the worker died before processing the request, or if the processing closure returned an error.

Trait Implementations

impl<Req: Clone, Res: Clone, E: Clone> Clone for SyncBatcher<Req, Res, E>

fn clone(&self) -> SyncBatcher<Req, Res, E>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.