pub struct TextBatch {
pub input_ids: Tensor,
pub attention_mask: Tensor,
pub token_type_ids: Tensor,
pub special_tokens_mask: Tensor,
pub lengths: Vec<usize>,
}Expand description
Model-ready integer tensors and original unpadded lengths.
Fields§
§input_ids: TensorPadded [batch,length] Int64 vocabulary IDs.
attention_mask: TensorInt64 mask: one for valid tokens, zero for padding.
token_type_ids: TensorPadded Int64 segment IDs.
special_tokens_mask: TensorInt64 mask for original special tokens; padded positions are zero.
lengths: Vec<usize>Original token lengths.
Trait Implementations§
Source§impl MemoryFootprint for TextBatch
impl MemoryFootprint for TextBatch
Source§fn resident_bytes(&self) -> usize
fn resident_bytes(&self) -> usize
Returns the conservative logical payload size in bytes.
Auto Trait Implementations§
impl !Sync for TextBatch
impl Forget for TextBatch
impl Freeze for TextBatch
impl RefUnwindSafe for TextBatch
impl Send for TextBatch
impl Unpin for TextBatch
impl UnsafeUnpin for TextBatch
impl UnwindSafe for TextBatch
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more