pub struct Placement<'f> {
pub row: &'f dyn Fn(usize, usize) -> usize,
pub rope_theta: Option<f32>,
pub decode_only: bool,
}Expand description
Where a stream’s output goes: element index of (head, token) row start and the token stride, plus optional RoPE (K given pre-RoPE).
Fields§
§row: &'f dyn Fn(usize, usize) -> usize(head, token) -> element offset of that row’s channel 0.
rope_theta: Option<f32>Apply half-split RoPE with this theta to each decoded token row.
decode_only: boolDecode without writing the output (benchmark: the cost a fused decode + attention kernel cannot avoid).
Auto Trait Implementations§
impl<'f> !RefUnwindSafe for Placement<'f>
impl<'f> !Send for Placement<'f>
impl<'f> !Sync for Placement<'f>
impl<'f> !UnwindSafe for Placement<'f>
impl<'f> Freeze for Placement<'f>
impl<'f> Unpin for Placement<'f>
impl<'f> UnsafeUnpin for Placement<'f>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more