String Pipeline

A flexible, composable string transformation CLI tool and Rust library. string_pipeline lets you chain operations like split, join, slice, replace, case conversion, trim, and more, using a concise template syntax. It is ideal for quick text manipulation, scripting, and data extraction.

Table of Contents

• Features
• Crate
• Installation
  • CLI
  • Library
• Usage
  • As a CLI
  • As a Library
• Template Syntax
  • Syntax Reference
  • Supported Operations
  • Range Specifications
  • Escaping
  • Debug Mode
• More Examples
• Testing
• Contributing
• License

Features

• Composable operations: Chain multiple string operations in a single template.
• Split and join: Extract and reassemble parts of strings.
• Slice and range: Select sub-strings or sub-lists by index or range.
• Regex replace: sed-like regex search and replace.
• Filtering: Filter lists with regex patterns using filter and filter_not.
• Case conversion: Uppercase and lowercase.
• Trim and strip: Remove whitespace, custom characters, ansi sequences.
• Append and prepend: Add text before or after.
• Map: Apply a sub-pipeline to each list item.
• Sort, reverse, unique: List operations for sorting, reversing, deduplication.
• Pad: Pad strings or list items to a given width.
• Regex extract: Extract regex matches or groups.
• Smart escaping: Contextual pipe handling - no escaping needed in most cases.
• Flexible indices: Python-like negative indices in ranges and slices with Rust-like syntax.
• Stdin support: Read input from stdin when no input argument is provided.
• Debug mode: Add ! after { to print debug info for each operation.
• Robust error handling: Helpful error messages for invalid templates or arguments.
• Tested: Comprehensive test suite.

Crate

Find the crate on crates.io:

[dependencies]
string_pipeline = "0.8.0"

Installation

CLI

Clone and build:

git clone https://github.com/lalvarezt/string_pipeline.git
cd string_pipeline
cargo build --release

Or run directly with Cargo:

cargo run -- "{template}" "input string"

Library

Add to your Cargo.toml as shown above.

Usage

As a CLI

# With input argument
string-pipeline "{template}" "input string"

# With stdin input
echo "input string" | string-pipeline "{template}"

Examples:

# Get the second item in a comma-separated list
string-pipeline "{split:,:1}" "a,b,c"
# Output: b

# Replace all spaces with underscores and uppercase
string-pipeline "{replace:s/ /_/g|upper}" "foo bar baz"
# Output: FOO_BAR_BAZ

# Trim, split, and append
string-pipeline "{split:,:..|trim|append:!}" " a, b,c , d , e "
# Output: a!,b!,c!,d!,e!

# Using map to uppercase each item
string-pipeline "{split:,:..|map:{upper}}" "a,b,c"
# Output: A,B,C

# Sort and join
string-pipeline "{split:,:..|sort:desc|join:-}" "b,a,c"
# Output: c-b-a

# Pad each item to width 3 with '*'
string-pipeline "{split:,:..|map:{pad:3:*:both}}" "a,bb,c"
# Output: *a*,bb*,*c*

# Extract numbers from each item
string-pipeline "{split:,:..|map:{regex_extract:\d+}}" "a1,b22,c333"
# Output: 1,22,333

As a Library

use string_pipeline::Template;

fn main() {
    let template = Template::parse("{split:,:..|join:\\n}").unwrap();
    let result = template.format("a,b,c").unwrap();
    assert_eq!(result, "a\nb\nc");
}

Template Syntax

Templates are enclosed in {} and consist of a chain of operations separated by |. Arguments to operations are separated by :.

Syntax Reference

• Template: { [!] operation_list? }
  • Add ! after { to enable debug mode.
• Operation List: operation ('|' operation)*
• Operation:
  • split:<sep>:<range>
    • Shorthand for split:
      • {index} (e.g. {1}, equivalent to {split: :1})
      • {range} (e.g. {1..3}, equivalent to {split: :1..3})
  • join:<sep>
  • substring:<range>
  • replace:s/<pattern>/<replacement>/<flags>
  • upper
  • lower
  • trim[:left|right|both]
  • strip:<chars>
  • append:<suffix>
  • prepend:<prefix>
  • strip_ansi
  • filter:<regex_pattern>
  • filter_not:<regex_pattern>
  • slice:<range>
  • map:{<operation_list>}
  • sort[:asc|desc]
  • reverse
  • unique
  • pad:<width>[:<char>][:left|right|both]
  • regex_extract:<pattern>[:<group>]

Supported Operations

Range Specifications

Ranges use Rust-like syntax and support negative indices like Python:

Negative indices count from the end:

• -1 = last element
• -2 = second to last element
• -3.. = last 3 elements

Escaping

The parser intelligently handles pipe characters (|) based on context:

Pipes are automatically allowed in:

• Split separators: {split:|:..} (splits on pipe)
• Regex patterns: {filter:\.(txt|md|log)} (alternation)
• Sed replacements: {replace:s/test/a|b/} (pipe in replacement)

Manual escaping needed for:

• Other arguments: Use \| for literal pipes in join, append, prepend, etc.
• Special characters: Use \: for literal colons, \\ for backslashes
• Escape sequences: Use \n, \t, \r for newline, tab, carriage return

Enable Debug Mode

• Add ! after { to enable debug output for each operation:
  • Example: {!split:,:..|upper|join:-}

More examples

# Get the last item
string-pipeline "{split:,:-1}" "a,b,c"
# Output: c

# Get the second word using the short form (only works with space separator)
string-pipeline "{1}" "foo bar baz"
# Output: bar

# Get a range of items
string-pipeline "{split:,:1..=3}" "a,b,c,d,e"
# Output: b,c,d

# Get a range of words using the short form (only works with space separator)
string-pipeline "{1..3}" "foo bar baz qux"
# Output: bar baz

# Get a range of items, another way
string-pipeline "{split:,:..|slice:1..=3}" "a,b,c,d,e"
# Output: b,c,d

# Substring string characters
string-pipeline "{substring:1..=3}" "hello"
# Output: ell

# Replace 'foo' with 'bar' globally
string-pipeline "{replace:s/foo/bar/g}" "foo foo"
# Output: bar bar

# Chain operations: uppercase, then append
string-pipeline "{upper|append:!}" "hello"
# Output: HELLO!

# Prepend with a colon (escaped)
string-pipeline "{prepend:\:foo}" "bar"
# Output: :foobar

# Map: uppercase each item
string-pipeline "{split:,:..|map:{upper}}" "a,b,c"
# Output: A,B,C

# Sort, reverse, unique, pad, regex_extract
string-pipeline "{split:,:..|sort:desc|join:-}" "b,a,c"
# Output: c-b-a

string-pipeline "{split:,:..|reverse}" "a,b,c"
# Output: c,b,a

string-pipeline "{split:,:..|unique}" "a,b,a,c"
# Output: a,b,c

string-pipeline "{split:,:..|map:{pad:3:*:both}}" "a,bb,c"
# Output: *a*,bb*,*c*

string-pipeline "{split:,:..|map:{regex_extract:\\d+}}" "a1,b22,c333"
# Output: 1,22,333

Debug Mode

# Print debug info for each operation
string-pipeline "{!split:,:..|map:{trim|upper}}" "user123,   admin456   ,guest789"
# DEBUG: Initial value: Str("user123,   admin456   ,guest789")
# DEBUG: Applying operation 1: Split { sep: ",", range: Range(None, None, false) }
# DEBUG: Result: List with 3 items:
# DEBUG:   [0]: "user123"
# DEBUG:   [1]: "   admin456   "
# DEBUG:   [2]: "guest789"
# DEBUG: ---
# DEBUG: Applying operation 2: Map { operations: [Trim { direction: Both }, Upper] }
# DEBUG: Map operation starting with 3 items
# DEBUG: Map operations to apply: 2 steps
# DEBUG:   Step 1: Trim { direction: Both }
# DEBUG:   Step 2: Upper
# DEBUG: Processing item 1 of 3: "user123"
# DEBUG:   Item 1/3 initial value: Str("user123")
# DEBUG:   Item 1/3 applying step 1: Trim { direction: Both }
# DEBUG:   Item 1/3 step 1 result: String("user123")
# DEBUG:   Item 1/3 applying step 2: Upper
# DEBUG:   Item 1/3 step 2 result: String("USER123")
# DEBUG: Processing item 2 of 3: "   admin456   "
# DEBUG:   Item 2/3 initial value: Str("   admin456   ")
# DEBUG:   Item 2/3 applying step 1: Trim { direction: Both }
# DEBUG:   Item 2/3 step 1 result: String("admin456")
# DEBUG:   Item 2/3 applying step 2: Upper
# DEBUG:   Item 2/3 step 2 result: String("ADMIN456")
# DEBUG: Processing item 3 of 3: "guest789"
# DEBUG:   Item 3/3 initial value: Str("guest789")
# DEBUG:   Item 3/3 applying step 1: Trim { direction: Both }
# DEBUG:   Item 3/3 step 1 result: String("guest789")
# DEBUG:   Item 3/3 applying step 2: Upper
# DEBUG:   Item 3/3 step 2 result: String("GUEST789")
# DEBUG: Map operation completed. Results:
# DEBUG:   Item 1: "USER123"
# DEBUG:   Item 2: "ADMIN456"
# DEBUG:   Item 3: "GUEST789"
# DEBUG: Result: List with 3 items:
# DEBUG:   [0]: "USER123"
# DEBUG:   [1]: "ADMIN456"
# DEBUG:   [2]: "GUEST789"
# DEBUG: ---
# USER123,ADMIN456,GUEST789

Testing

Run the test suite:

cargo test

Benchmarks

Run the bench suite:

cargo bench

Contributing

Contributions and suggestions are welcome! Please open issues or pull requests on GitHub.

License

This project is licensed under the MIT License. See LICENSE for details.

Enjoy fast, composable string transformations!