Struct Batch 

pub struct Batch<'a> {
    pub ids: &'a [u32],
    pub cu: &'a [u32],
    pub markers: &'a [u32],
    pub mcu: &'a [u32],
    pub qtype: &'a [u8],
}

A batch, flattened. Sequence s is tokens cu[s]..cu[s + 1] and markers mcu[s]..mcu[s + 1], and each marker is a position within its own sequence.

Fields

ids: &'a [u32]

Token ids of every sequence, end to end.

cu: &'a [u32]

Sequence starts in ids, with the total at the end.

markers: &'a [u32]

Marker positions of every sequence, end to end.

mcu: &'a [u32]

Sequence starts in markers, with the total at the end.

qtype: &'a [u8]

The question type of each sequence.

Implementations

impl Batch<'_>

pub fn seqs(&self) -> usize

Sequences.

pub fn check(&self, vocab: usize, types: usize) -> Result<()>

Checks that the pieces agree with each other, so a backend can index without checks.

Errors

Error::Batch naming the first problem.

Trait Implementations

impl<'a> Clone for Batch<'a>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Copy for Batch<'a>

impl<'a> Debug for Batch<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a> Default for Batch<'a>

fn default() -> Self

Returns the “default value” for a type.