Struct PackingConfig 

pub struct PackingConfig {
    pub max_packed_length: usize,
    pub pad_token_id: u32,
    pub sep_token_id: Option<u32>,
    pub add_separators: bool,
    pub min_sequence_length: usize,
    pub max_sequences_per_pack: usize,
    pub strategy: PackingStrategy,
    pub preserve_boundaries: bool,
}

Configuration for sequence packing

Fields

max_packed_length: usize

Maximum sequence length after packing

pad_token_id: u32

Padding token ID

sep_token_id: Option<u32>

Separator token ID (used between packed sequences)

add_separators: bool

Whether to add separator tokens between sequences

min_sequence_length: usize

Minimum sequence length to consider for packing

max_sequences_per_pack: usize

Maximum number of sequences to pack together

strategy: PackingStrategy

Packing strategy to use

preserve_boundaries: bool

Whether to preserve sequence boundaries in attention masks

Trait Implementations

impl Clone for PackingConfig

fn clone(&self) -> PackingConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PackingConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for PackingConfig

fn default() -> Self

Returns the “default value” for a type.