Struct RateLimit 

pub struct RateLimit { /* private fields */ }

Implementations

impl RateLimit

pub fn new(config: &Value, storage: Arc<dyn Storage>) -> Result<Self, String>

Trait Implementations

impl Extension for RateLimit

fn name(&self) -> &str

Human-readable name for this extension, used in logs and diagnostics.

fn prefix(&self) -> Prefix

Fixed 4-byte prefix that namespaces this extension’s storage keys.

fn on_request( &self, ctx: &RequestContext, ) -> BoxFuture<'_, Result<(), ExtensionError>>

Called post-auth, pre-dispatch. Return Err to short-circuit the pipeline (no provider call, no further extensions run).

fn on_response( &self, ctx: &RequestContext, _request: &ChatCompletionRequest, response: &ChatCompletionResponse, ) -> BoxFuture<'_, ()>

Called after a non-streaming chat completion response arrives from the provider.

fn on_chunk( &self, ctx: &RequestContext, chunk: &ChatCompletionChunk, ) -> BoxFuture<'_, ()>

Called once per SSE chunk during a streaming response, before serialization.

fn storage_key(&self, suffix: &[u8]) -> Vec<u8>

Build a full storage key by prepending this extension’s prefix to suffix.

fn on_cache_lookup( &self, _request: &ChatCompletionRequest, ) -> Pin<Box<dyn Future<Output = Option<ChatCompletionResponse>> + Send + '_>>

Check for a cached response before provider dispatch. Return Some to skip the provider call entirely. Called for non-streaming requests only.

fn on_error( &self, _ctx: &RequestContext, _error: &Error, ) -> Pin<Box<dyn Future<Output = ()> + Send + '_>>

Called when the provider returns an error.