textfsm-core 0.3.0

# textfsm-rust

A Rust implementation of Google's [TextFSM](https://github.com/google/textfsm) template-based state machine for parsing semi-formatted text.

## Overview

TextFSM is a template-based state machine originally developed by Google for parsing CLI output from network devices. This crate provides a native Rust implementation with the same template syntax and behavior.

## Features

- **Battle-tested compatibility** - 98.5% pass rate against the [ntc-templates](https://github.com/networktocode/ntc-templates) test suite (1612/1637 tests)
- **Compile-time template validation** - Catch template errors at compile time with proc macros, not at runtime
- **Serde integration** - Deserialize parsed results directly into typed Rust structs
- **Thread-safe design** - `Template` and `CliTable` are `Send + Sync`; share across threads with zero overhead
- **Detailed error messages** - Template errors include line numbers, context, and suggestions
- **Full TextFSM syntax** - All value options (`Required`, `Filldown`, `Fillup`, `List`, `Key`) and actions (`Next`, `Continue`, `Record`, `Clear`, `Error`)
- **Efficient regex handling** - Uses [fancy-regex](https://docs.rs/fancy-regex) which delegates to the fast [regex](https://docs.rs/regex) crate when advanced features aren't needed
- **Python regex compatibility** - Handles Python regex quirks like `\<` and `\>` automatically

## Installation

Add to your `Cargo.toml`:

```toml
[dependencies]
textfsm-rust = "0.3"
```

## Quick Start

```rust
use textfsm_rust::Template;

let template = Template::parse_str(r#"
Value Name (\S+)
Value Age (\d+)

Start
  ^Name: ${Name}, Age: ${Age} -> Record
"#)?;

let mut parser = template.parser();
let input = "Name: Alice, Age: 30\nName: Bob, Age: 25\n";
let results = parser.parse_text(input)?;
```

## Compile-Time Template Validation

Catch template errors before your program runs:

```rust
use textfsm_rust::{validate_template, validate_templates, Template};

// Validate a single template at compile time
validate_template!("templates/cisco_show_version.textfsm");

// Validate all .textfsm files in a directory
validate_templates!("templates/");

// Parse at runtime when you need it
let template = Template::parse_str(
    include_str!("templates/cisco_show_version.textfsm")
).expect("validated at compile time");
```

If a template is invalid, you get a compile error with the template line number:

```
error: Template validation failed for 'templates/bad.textfsm':
       invalid variable substitution at line 8: unknown variable 'Interfce'
 --> src/main.rs:4:21
  |
4 | validate_template!("templates/bad.textfsm");
  |                    ^^^^^^^^^^^^^^^^^^^^^^^^
```

## Template Syntax

TextFSM templates consist of two sections:

### Value Definitions

```
Value [Options] Name (Regex)
```

Options:
- `Required` - Record only saved if this value is matched
- `Filldown` - Value retained across records
- `Fillup` - Value fills upward into previous records
- `List` - Value is a list of matches
- `Key` - Marks value as a key field

### State Rules

```
StateName
  ^Regex -> [Actions]
```

Actions:
- `Next` - Continue to next line (default)
- `Continue` - Continue processing current line
- `Record` - Save current record
- `Clear` - Clear non-filldown values
- `Error` - Stop processing with error

## Serde Integration

Enable the `serde` feature to deserialize parsed results directly into typed Rust structs:

```toml
[dependencies]
textfsm-rust = { version = "0.3", features = ["serde"] }
```

```rust
use textfsm_rust::{Deserialize, Template};

#[derive(Deserialize, Debug)]
struct Interface {
    interface: String,
    status: String,
    ip_address: Option<String>,
}

let template = Template::parse_str(r#"
Value Interface (\S+)
Value Status (up|down)
Value IP_Address (\d+\.\d+\.\d+\.\d+)

Start
  ^Interface: ${Interface} is ${Status}
  ^  IP: ${IP_Address} -> Record
"#)?;

let mut parser = template.parser();
let input = "Interface: eth0 is up\n  IP: 192.168.1.1\nInterface: eth1 is down\n  IP: 10.0.0.1\n";

// Deserialize directly into typed structs
let interfaces: Vec<Interface> = parser.parse_text_into(input)?;

assert_eq!(interfaces[0].interface, "eth0");
assert_eq!(interfaces[0].status, "up");
assert_eq!(interfaces[0].ip_address, Some("192.168.1.1".into()));
```

Field names are matched case-insensitively against template value names (underscores are preserved).

## Thread Safety

`Template` is `Send + Sync` and can be safely shared across threads. The compiled template (including all regexes) is immutable after creation, so you can wrap it in an `Arc` and share it freely.

`Parser` is a lightweight, stateful wrapper that borrows a `Template`. Create one per thread - they're cheap since they don't copy the compiled regexes.

```rust
use std::sync::Arc;
use std::thread;

let template = Arc::new(Template::parse_str(template_str)?);

let handles: Vec<_> = inputs.into_iter().map(|input| {
    let template = Arc::clone(&template);
    thread::spawn(move || {
        let mut parser = template.parser();
        parser.parse_text(&input)
    })
}).collect();
```

This design is efficient for concurrent workloads like parsing output from multiple network devices in parallel.

## CliTable - Automatic Template Selection

CliTable automatically selects the right template based on command and platform attributes, using the same index format as [ntc-templates](https://github.com/networktocode/ntc-templates):

```rust
use textfsm_rust::CliTable;
use std::collections::HashMap;

// Load index and templates directory
let cli_table = CliTable::new(
    "ntc-templates/ntc_templates/templates/index",
    "ntc-templates/ntc_templates/templates"
)?;

// Parse CLI output with automatic template selection
let mut attrs = HashMap::new();
attrs.insert("Command".to_string(), "show version".to_string());
attrs.insert("Platform".to_string(), "cisco_ios".to_string());

let table = cli_table.parse_cmd(&cli_output, &attrs)?;

// Access results
for row in table.iter() {
    println!("Version: {}", row.get("VERSION").unwrap());
}
```

With serde, deserialize directly into typed structs:

```rust
#[derive(Deserialize)]
struct Version {
    version: String,
    hostname: String,
}

let versions: Vec<Version> = cli_table.parse_cmd_into(&cli_output, &attrs)?;
```

## Error Messages

Template errors include line numbers to help you fix issues quickly:

```
invalid variable substitution at line 8: unknown variable 'Interfce'
```

```
invalid Value definition at line 3: regex must be wrapped in parentheses
```

```
invalid rule at line 12: unclosed variable substitution
```

```
missing required 'Start' state
```



## Known Limitations

### Compatibility with Python TextFSM

This implementation achieves **98.5% compatibility** with ntc-templates (1612/1637 tests passing for direct template parsing). The remaining differences are documented below.

#### Empty List Handling

When a `Value List` field is defined but never matches any input:

| Implementation | Output | Example |
|----------------|--------|---------|
| Python TextFSM | `["None"]` | List with literal string "None" |
| Rust TextFSM | `[]` | Empty list |

This affects ~14 ntc-templates tests.

**Workaround:** Check for empty lists in your code rather than checking for `["None"]`.

#### Regex: Quantified Lookbehinds

The `fancy-regex` crate doesn't support quantifiers directly after lookbehinds:

```
# This pattern in cisco_ios_show_access-list.textfsm fails:
(?<=[^()\s])+

# Error: "Target of repeat operator is invalid"
```

This is a limitation of the regex engine, not our implementation. The pattern itself is arguably malformed (lookbehinds shouldn't be quantified). This affects 3 ntc-templates tests.

### Test Results

**Direct Template Parsing: 98.5% (1612/1637 tests)**

Most failures are in edge cases for specific vendors (huawei_vrp, fortinet). Core functionality for Cisco IOS/NXOS/ASA, Arista EOS, and Juniper works reliably.

## License

Licensed under either of:

- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
- MIT License ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)

at your option.

## Acknowledgments

This project is a Rust port of [Google's TextFSM](https://github.com/google/textfsm), originally developed by Google Inc. and licensed under Apache 2.0.

Thanks to [Network to Code](https://github.com/networktocode/ntc-templates) for their extensive collection of TextFSM templates and test data, which made it possible to validate this implementation against real-world use cases.