Expand description
Smart Atomizer - Language-aware code parsing
This module uses Tree-sitter for AST-based code analysis to extract properly bounded code atoms with correct dependency tracking.
§Key Features
- Language-Aware Parsing: Uses Tree-sitter grammars for accurate parsing
- Symbol Resolution: Resolves imports to content-addressed chunk IDs
- Boundary Detection: Never cuts functions in half
- Dependency Tracking: Tracks what each atom imports/exports
§Supported Languages (Phase 1)
- Rust
- TypeScript/JavaScript
- Python
§Example
ⓘ
use cadi_core::atomizer::{Atomizer, AtomizerConfig};
let atomizer = Atomizer::new(AtomizerConfig::default());
// Parse a file into atoms
let atoms = atomizer.atomize_file(path, content)?;
for atom in atoms {
println!("Atom: {} ({:?})", atom.name, atom.kind);
println!(" Defines: {:?}", atom.defines);
println!(" References: {:?}", atom.references);
}Re-exports§
pub use config::AtomizerConfig;pub use config::LanguageConfig;pub use parser::AstParser;pub use extractor::AtomExtractor;pub use extractor::ExtractedAtom;pub use extractor::AtomKind;pub use resolver::SymbolResolver;pub use resolver::ResolvedImport;pub use resolver::ImportedSymbol;