pub struct ChunkedPrefillConfig {
pub chunk_size: usize,
pub max_concurrent_prefills: usize,
}Expand description
Configuration for chunked prefill processing.
Fields§
§chunk_size: usizeMaximum number of prompt tokens processed in a single prefill step (default: 128).
max_concurrent_prefills: usizeMaximum number of concurrent prompts in prefill stage (default: 4).
Trait Implementations§
Source§impl Clone for ChunkedPrefillConfig
impl Clone for ChunkedPrefillConfig
Source§fn clone(&self) -> ChunkedPrefillConfig
fn clone(&self) -> ChunkedPrefillConfig
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for ChunkedPrefillConfig
impl Debug for ChunkedPrefillConfig
Source§impl Default for ChunkedPrefillConfig
impl Default for ChunkedPrefillConfig
Source§fn default() -> ChunkedPrefillConfig
fn default() -> ChunkedPrefillConfig
Returns the “default value” for a type. Read more
Auto Trait Implementations§
impl Forget for ChunkedPrefillConfig
impl Freeze for ChunkedPrefillConfig
impl Move for ChunkedPrefillConfig
impl RefUnwindSafe for ChunkedPrefillConfig
impl Send for ChunkedPrefillConfig
impl Sync for ChunkedPrefillConfig
impl Unpin for ChunkedPrefillConfig
impl UnsafeUnpin for ChunkedPrefillConfig
impl UnwindSafe for ChunkedPrefillConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more