sqry-nl 9.0.6

Natural language to sqry query translation layer
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

//! Tokenizer wrapper for thread-safe access.
//!
//! The `HuggingFace` tokenizer is not inherently thread-safe.
//! This wrapper provides thread-local or Arc-based access patterns.

// This module is a placeholder for future thread-safety enhancements.
// Currently, the classifier owns the tokenizer directly since we
// enforce single-threaded inference (batch_size=1, intra_threads=1).

// If multi-threaded access is needed in the future:
// 1. Use thread_local! for per-thread tokenizer instances
// 2. Or use Arc<parking_lot::Mutex<Tokenizer>> for shared access

#[cfg(test)]
mod tests {
    #[test]
    fn test_placeholder() {
        // Placeholder test
    }
}