pub struct Stage {Show 14 fields
pub dev: usize,
pub gpu: Gpu,
pub layers: Vec<LayerDev>,
pub embed: Option<CudaSlice<u8>>,
pub head: Option<CudaSlice<u8>>,
pub trunk_norm: Option<CudaSlice<f32>>,
pub hc_head_fn: Option<CudaSlice<f32>>,
pub fc_yarn: CudaSlice<f32>,
pub fc_plain: CudaSlice<f32>,
pub ws: CudaSlice<u8>,
pub deq: [CudaSlice<u8>; 3],
pub loaded_bytes: u64,
pub hc_head_base_dev: Option<CudaSlice<f32>>,
pub hc_head_scale_dev: Option<CudaSlice<f32>>,
}Expand description
One stage = one GPU: its runtime handle plus the resident weights of its layer range.
Fields§
§dev: usize§gpu: Gpu§layers: Vec<LayerDev>§embed: Option<CudaSlice<u8>>§head: Option<CudaSlice<u8>>§trunk_norm: Option<CudaSlice<f32>>§hc_head_fn: Option<CudaSlice<f32>>§fc_yarn: CudaSlice<f32>§fc_plain: CudaSlice<f32>§ws: CudaSlice<u8>§deq: [CudaSlice<u8>; 3]§loaded_bytes: u64§hc_head_base_dev: Option<CudaSlice<f32>>§hc_head_scale_dev: Option<CudaSlice<f32>>Auto Trait Implementations§
impl Freeze for Stage
impl RefUnwindSafe for Stage
impl Send for Stage
impl Sync for Stage
impl Unpin for Stage
impl UnsafeUnpin for Stage
impl UnwindSafe for Stage
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more