textfsm-core 0.3.1

# textfsm-rust

A Rust implementation of Google's [TextFSM](https://github.com/google/textfsm) template-based state machine for parsing semi-formatted text.

## Overview

TextFSM is a template-based state machine originally developed by Google for parsing CLI output from network devices. This crate provides a native Rust implementation with the same template syntax and behavior.

## Features

- **Battle-tested compatibility** - 97.51% strict pass rate against [ntc-templates](https://github.com/networktocode/ntc-templates) (1762/1807 tests), matching Python's TextFSM exactly
- **Compile-time template validation** - Catch template errors at compile time with proc macros, not at runtime
- **Serde integration** - Deserialize parsed results directly into typed Rust structs
- **Thread-safe design** - `Template` and `CliTable` are `Send + Sync`; share across threads with zero overhead
- **Detailed error messages** - Template errors include line numbers, context, and suggestions
- **Full TextFSM syntax** - All value options (`Required`, `Filldown`, `Fillup`, `List`, `Key`) and actions (`Next`, `Continue`, `Record`, `Clear`, `Error`)
- **Efficient regex handling** - Uses [fancy-regex](https://docs.rs/fancy-regex) which delegates to the fast [regex](https://docs.rs/regex) crate when advanced features aren't needed
- **Python regex compatibility** - Handles Python regex quirks like `\<` and `\>` automatically

## Installation

Add to your `Cargo.toml`:

```toml
[dependencies]
textfsm-rust = "0.3.1"
```

## Quick Start

```rust
use textfsm_rust::Template;

let template = Template::parse_str(r#"
Value Name (\S+)
Value Age (\d+)

Start
  ^Name: ${Name}, Age: ${Age} -> Record
"#)?;

let mut parser = template.parser();
let input = "Name: Alice, Age: 30\nName: Bob, Age: 25\n";
let results = parser.parse_text(input)?;
```

## Compile-Time Template Validation

Catch template errors before your program runs:

```rust
use textfsm_rust::{validate_template, validate_templates, Template};

// Validate a single template at compile time
validate_template!("templates/cisco_show_version.textfsm");

// Validate all .textfsm files in a directory
validate_templates!("templates/");

// Parse at runtime when you need it
let template = Template::parse_str(
    include_str!("templates/cisco_show_version.textfsm")
).expect("validated at compile time");
```

If a template is invalid, you get a compile error with the template line number:

```
error: Template validation failed for 'templates/bad.textfsm':
       invalid variable substitution at line 8: unknown variable 'Interfce'
 --> src/main.rs:4:21
  |
4 | validate_template!("templates/bad.textfsm");
  |                    ^^^^^^^^^^^^^^^^^^^^^^^^
```

## Template Syntax

TextFSM templates consist of two sections:

### Value Definitions

```
Value [Options] Name (Regex)
```

Options:
- `Required` - Record only saved if this value is matched
- `Filldown` - Value retained across records
- `Fillup` - Value fills upward into previous records
- `List` - Value is a list of matches
- `Key` - Marks value as a key field

### State Rules

```
StateName
  ^Regex -> [Actions]
```

Actions:
- `Next` - Continue to next line (default)
- `Continue` - Continue processing current line
- `Record` - Save current record
- `Clear` - Clear non-filldown values
- `Error` - Stop processing with error

## Serde Integration

Enable the `serde` feature to deserialize parsed results directly into typed Rust structs:

```toml
[dependencies]
textfsm-rust = { version = "0.3", features = ["serde"] }
```

```rust
use textfsm_rust::{Deserialize, Template};

#[derive(Deserialize, Debug)]
struct Interface {
    interface: String,
    status: String,
    ip_address: Option<String>,
}

let template = Template::parse_str(r#"
Value Interface (\S+)
Value Status (up|down)
Value IP_Address (\d+\.\d+\.\d+\.\d+)

Start
  ^Interface: ${Interface} is ${Status}
  ^  IP: ${IP_Address} -> Record
"#)?;

let mut parser = template.parser();
let input = "Interface: eth0 is up\n  IP: 192.168.1.1\nInterface: eth1 is down\n  IP: 10.0.0.1\n";

// Deserialize directly into typed structs
let interfaces: Vec<Interface> = parser.parse_text_into(input)?;

assert_eq!(interfaces[0].interface, "eth0");
assert_eq!(interfaces[0].status, "up");
assert_eq!(interfaces[0].ip_address, Some("192.168.1.1".into()));
```

Field names are matched case-insensitively against template value names (underscores are preserved).

## Thread Safety

`Template` is `Send + Sync` and can be safely shared across threads. The compiled template (including all regexes) is immutable after creation, so you can wrap it in an `Arc` and share it freely.

`Parser` is a lightweight, stateful wrapper that borrows a `Template`. Create one per thread - they're cheap since they don't copy the compiled regexes.

```rust
use std::sync::Arc;
use std::thread;

let template = Arc::new(Template::parse_str(template_str)?);

let handles: Vec<_> = inputs.into_iter().map(|input| {
    let template = Arc::clone(&template);
    thread::spawn(move || {
        let mut parser = template.parser();
        parser.parse_text(&input)
    })
}).collect();
```

This design is efficient for concurrent workloads like parsing output from multiple network devices in parallel.

## CliTable - Automatic Template Selection

CliTable automatically selects the right template based on command and platform attributes, using the same index format as [ntc-templates](https://github.com/networktocode/ntc-templates):

```rust
use textfsm_rust::CliTable;
use std::collections::HashMap;

// Load index and templates directory
let cli_table = CliTable::new(
    "ntc-templates/ntc_templates/templates/index",
    "ntc-templates/ntc_templates/templates"
)?;

// Parse CLI output with automatic template selection
let mut attrs = HashMap::new();
attrs.insert("Command".to_string(), "show version".to_string());
attrs.insert("Platform".to_string(), "cisco_ios".to_string());

let table = cli_table.parse_cmd(&cli_output, &attrs)?;

// Access results
for row in table.iter() {
    println!("Version: {}", row.get("VERSION").unwrap());
}
```

With serde, deserialize directly into typed structs:

```rust
#[derive(Deserialize)]
struct Version {
    version: String,
    hostname: String,
}

let versions: Vec<Version> = cli_table.parse_cmd_into(&cli_output, &attrs)?;
```

## Error Messages

Template errors include line numbers to help you fix issues quickly:

```
invalid variable substitution at line 8: unknown variable 'Interfce'
```

```
invalid Value definition at line 3: regex must be wrapped in parentheses
```

```
invalid rule at line 12: unclosed variable substitution
```

```
missing required 'Start' state
```



## NTC Templates Compatibility

This implementation is tested against the full [ntc-templates](https://github.com/networktocode/ntc-templates) test suite using a strict, field-by-field comparison harness. The Rust implementation matches Google's Python TextFSM exactly:

| | Python TextFSM | Rust textfsm-rs |
|---|---|---|
| Test Cases | 1807 | 1807 |
| Passed | 1762 | 1762 |
| Failed | 45 | 45 |
| Errors | 0 | 0 |
| **Pass Rate** | **97.51%** | **97.51%** |

The same 45 tests fail in both implementations -- all are test data issues in the ntc-templates repository (YAML expects `"None"` strings in lists, or references columns not defined in templates), not parser bugs.

These results were produced by two equivalent strict test harnesses included in this repository:

- **Python baseline**: [`tests/python_ntc_harness.py`](tests/python_ntc_harness.py) -- runs all ntc-templates test cases through Google's [Python TextFSM](https://github.com/google/textfsm) and performs strict field-by-field comparison against the expected YAML output.
- **Rust harness**: [`textfsm-rust/examples/strict_ntc_test.rs`](textfsm-rust/examples/strict_ntc_test.rs) -- runs the same test cases through textfsm-rs with identical discovery and comparison logic.

Both harnesses discover test cases the same way (walking the `tests/{platform}/{command}/` directory structure), derive the template from the directory path, and compare parsed results against the YAML expected output with the same normalization rules. You can run them yourself:

```bash
# Python (requires PyYAML)
python3 tests/python_ntc_harness.py /path/to/ntc-templates --textfsm-path /path/to/textfsm

# Rust
cargo run --example strict_ntc_test --features serde -- /path/to/ntc-templates
```

### Known Differences

While the pass/fail classification is identical, there is one minor behavioral difference in how empty list values are represented:

| Scenario | Python TextFSM | Rust textfsm-rs |
|----------|---------------|-----------------|
| List value with no matches | `[""]` (list with empty string) | `[]` (empty list) |

This does not affect the pass rate since both representations fail against YAML test data that expects `["None"]`.

## License

Licensed under either of:

- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT License ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

at your option.

## Acknowledgments

This project is a Rust port of [Google's TextFSM](https://github.com/google/textfsm), originally developed by Google Inc. and licensed under Apache 2.0.

Thanks to [Network to Code](https://github.com/networktocode/ntc-templates) for their extensive collection of TextFSM templates and test data, which made it possible to validate this implementation against real-world use cases.