Struct Fim 

pub struct Fim { /* private fields */ }

Fill-in-the-Middle transform for code training data.

Applies FIM transformation to a text column in a RecordBatch. Each row is randomly split into (prefix, middle, suffix) and reassembled in PSM or SPM format with sentinel tokens.

Rows shorter than min_chars are left unchanged.

Implementations

impl Fim

pub fn new(column: impl Into<String>) -> Self

Create a new FIM transform for the given column.

pub fn with_rate(self, rate: f64) -> Self

Set the FIM application rate (0.0-1.0).

pub fn with_format(self, format: FimFormat) -> Self

Set the FIM format variant.

pub fn with_tokens(self, tokens: FimTokens) -> Self

Set custom sentinel tokens.

pub fn with_min_chars(self, min_chars: usize) -> Self

Set minimum character count for FIM to apply.

pub fn with_seed(self, seed: u64) -> Self

Set random seed for reproducibility.

Trait Implementations

impl Clone for Fim

fn clone(&self) -> Fim

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Fim

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Transform for Fim

fn apply(&self, batch: RecordBatch) -> Result<RecordBatch>

Applies the transform to a RecordBatch.