pub struct ChunkedPrefillConfig {
pub chunk_size: usize,
pub max_concurrent_prefills: usize,
}Expand description
Configuration for chunked prefill processing.
Fields§
§chunk_size: usizeMaximum number of prompt tokens processed in a single prefill step (default: 128).
max_concurrent_prefills: usizeMaximum number of concurrent prompts in prefill stage (default: 4).
Trait Implementations§
Source§impl Clone for ChunkedPrefillConfig
impl Clone for ChunkedPrefillConfig
Source§impl Debug for ChunkedPrefillConfig
impl Debug for ChunkedPrefillConfig
Auto Trait Implementations§
impl Forget for ChunkedPrefillConfig
impl Freeze for ChunkedPrefillConfig
impl Move for ChunkedPrefillConfig
impl RefUnwindSafe for ChunkedPrefillConfig
impl Send for ChunkedPrefillConfig
impl Sync for ChunkedPrefillConfig
impl Unpin for ChunkedPrefillConfig
impl UnsafeUnpin for ChunkedPrefillConfig
impl UnwindSafe for ChunkedPrefillConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more