String Pipeline
A flexible, composable string transformation CLI tool and Rust library. string_pipeline
lets you chain operations like split, join, slice, replace, case conversion, trim, and more, using a concise template syntax. It is ideal for quick text manipulation, scripting, and data extraction.
Table of Contents
Features
- Composable operations: Chain multiple string operations in a single template.
- Split and join: Extract and reassemble parts of strings.
- Slice and range: Select sub-strings or sub-lists by index or range.
- Regex replace: sed-like regex search and replace.
- Filtering: Filter lists with regex patterns using
filter
andfilter_not
. - Case conversion: Uppercase and lowercase.
- Trim and strip: Remove whitespace, custom characters, ansi sequences.
- Append and prepend: Add text before or after.
- Map: Apply a sub-pipeline to each list item.
- Sort, reverse, unique: List operations for sorting, reversing, deduplication.
- Pad: Pad strings or list items to a given width.
- Regex extract: Extract regex matches or groups.
- Smart escaping: Contextual pipe handling - no escaping needed in most cases.
- Flexible indices: Python-like negative indices in ranges and slices with Rust-like syntax.
- Stdin support: Read input from stdin when no input argument is provided.
- Debug mode: Add
!
after{
to print debug info for each operation. - Robust error handling: Helpful error messages for invalid templates or arguments.
- Tested: Comprehensive test suite.
Crate
Find the crate on crates.io:
[]
= "0.8.0"
Installation
CLI
Clone and build:
Or run directly with Cargo:
Library
Add to your Cargo.toml
as shown above.
Usage
As a CLI
# With input argument
# With stdin input
|
Examples:
# Get the second item in a comma-separated list
# Output: b
# Replace all spaces with underscores and uppercase
# Output: FOO_BAR_BAZ
# Trim, split, and append
# Output: a!,b!,c!,d!,e!
# Using map to uppercase each item
# Output: A,B,C
# Sort and join
# Output: c-b-a
# Pad each item to width 3 with '*'
# Output: *a*,bb*,*c*
# Extract numbers from each item
# Output: 1,22,333
As a Library
use Template;
Template Syntax
Templates are enclosed in {}
and consist of a chain of operations separated by |
.
Arguments to operations are separated by :
.
Syntax Reference
- Template:
{ [!] operation_list? }
- Add
!
after{
to enable debug mode.
- Add
- Operation List:
operation ('|' operation)*
- Operation:
split:<sep>:<range>
- Shorthand for split:
{index}
(e.g.{1}
, equivalent to{split: :1}
){range}
(e.g.{1..3}
, equivalent to{split: :1..3}
)
- Shorthand for split:
join:<sep>
substring:<range>
replace:s/<pattern>/<replacement>/<flags>
upper
lower
trim[:left|right|both]
strip:<chars>
append:<suffix>
prepend:<prefix>
strip_ansi
filter:<regex_pattern>
filter_not:<regex_pattern>
slice:<range>
map:{<operation_list>}
sort[:asc|desc]
reverse
unique
pad:<width>[:<char>][:left|right|both]
regex_extract:<pattern>[:<group>]
Supported Operations
Operation | Syntax | Description |
---|---|---|
Split | split:<sep>:<range> |
Split by separator, select by index/range |
Join | join:<sep> |
Join a list with separator |
Substring | substring:<range> |
Extract substring(s) by character index/range |
Replace | replace:s/<pattern>/<replacement>/<flags> |
Regex replace (sed-like, supports flags) |
Uppercase | upper |
Convert to uppercase |
Lowercase | lower |
Convert to lowercase |
Trim | trim[:left|right|both] |
Trim whitespace (or side-specific) |
Strip | strip:<chars> |
Strip custom characters from both ends |
Append | append:<suffix> |
Append text |
Prepend | prepend:<prefix> |
Prepend text |
StripAnsi | strip_ansi |
Remove ANSI escape sequences |
Filter | filter:<regex_pattern> |
Keep only items matching regex pattern |
FilterNot | filter_not:<regex_pattern> |
Remove items matching regex pattern |
Slice | slice:<range> |
Select elements from a list by index/range |
Map | map:{<operation_list>} |
Apply a sub-pipeline to each list item |
Sort | sort[:asc|desc] |
Sort list ascending/descending |
Reverse | reverse |
Reverse string or list |
Unique | unique |
Remove duplicate items from a list |
Pad | pad:<width>[:<char>][:left|right|both] |
Pad string/list items to width with char/side |
RegexExtract | regex_extract:<pattern>[:<group>] |
Extract first match or group from string/list items |
Range Specifications
Ranges use Rust-like syntax and support negative indices like Python:
Range | Description | Example |
---|---|---|
N |
Single index | {split:,:1} → second element |
N..M |
Exclusive range | {split:,:1..3} → elements 1,2 |
N..=M |
Inclusive range | {split:,:1..=3} → elements 1,2,3 |
N.. |
From N to end | {split:,:2..} → from 2nd to end |
..N |
From start to N | {split:,:..3} → first 3 elements |
..=N |
From start to N inclusive | {split:,:..=2} → first 3 elements |
.. |
All elements | {split:,:..} → all elements |
Negative indices count from the end:
-1
= last element-2
= second to last element-3..
= last 3 elements
Escaping
The parser intelligently handles pipe characters (|
) based on context:
Pipes are automatically allowed in:
- Split separators:
{split:|:..}
(splits on pipe) - Regex patterns:
{filter:\.(txt|md|log)}
(alternation) - Sed replacements:
{replace:s/test/a|b/}
(pipe in replacement)
Manual escaping needed for:
- Other arguments: Use
\|
for literal pipes in join, append, prepend, etc. - Special characters: Use
\:
for literal colons,\\
for backslashes - Escape sequences: Use
\n
,\t
,\r
for newline, tab, carriage return
Enable Debug Mode
- Add
!
after{
to enable debug output for each operation:- Example:
{!split:,:..|upper|join:-}
- Example:
More examples
# Get the last item
# Output: c
# Get the second word using the short form (only works with space separator)
# Output: bar
# Get a range of items
# Output: b,c,d
# Get a range of words using the short form (only works with space separator)
# Output: bar baz
# Get a range of items, another way
# Output: b,c,d
# Substring string characters
# Output: ell
# Replace 'foo' with 'bar' globally
# Output: bar bar
# Chain operations: uppercase, then append
# Output: HELLO!
# Prepend with a colon (escaped)
# Output: :foobar
# Map: uppercase each item
# Output: A,B,C
# Sort, reverse, unique, pad, regex_extract
# Output: c-b-a
# Output: c,b,a
# Output: a,b,c
# Output: *a*,bb*,*c*
# Output: 1,22,333
Debug Mode
# Print debug info for each operation
# DEBUG: Initial value: Str("user123, admin456 ,guest789")
# DEBUG: Applying operation 1: Split { sep: ",", range: Range(None, None, false) }
# DEBUG: Result: List with 3 items:
# DEBUG: [0]: "user123"
# DEBUG: [1]: " admin456 "
# DEBUG: [2]: "guest789"
# DEBUG: ---
# DEBUG: Applying operation 2: Map { operations: [Trim { direction: Both }, Upper] }
# DEBUG: Map operation starting with 3 items
# DEBUG: Map operations to apply: 2 steps
# DEBUG: Step 1: Trim { direction: Both }
# DEBUG: Step 2: Upper
# DEBUG: Processing item 1 of 3: "user123"
# DEBUG: Item 1/3 initial value: Str("user123")
# DEBUG: Item 1/3 applying step 1: Trim { direction: Both }
# DEBUG: Item 1/3 step 1 result: String("user123")
# DEBUG: Item 1/3 applying step 2: Upper
# DEBUG: Item 1/3 step 2 result: String("USER123")
# DEBUG: Processing item 2 of 3: " admin456 "
# DEBUG: Item 2/3 initial value: Str(" admin456 ")
# DEBUG: Item 2/3 applying step 1: Trim { direction: Both }
# DEBUG: Item 2/3 step 1 result: String("admin456")
# DEBUG: Item 2/3 applying step 2: Upper
# DEBUG: Item 2/3 step 2 result: String("ADMIN456")
# DEBUG: Processing item 3 of 3: "guest789"
# DEBUG: Item 3/3 initial value: Str("guest789")
# DEBUG: Item 3/3 applying step 1: Trim { direction: Both }
# DEBUG: Item 3/3 step 1 result: String("guest789")
# DEBUG: Item 3/3 applying step 2: Upper
# DEBUG: Item 3/3 step 2 result: String("GUEST789")
# DEBUG: Map operation completed. Results:
# DEBUG: Item 1: "USER123"
# DEBUG: Item 2: "ADMIN456"
# DEBUG: Item 3: "GUEST789"
# DEBUG: Result: List with 3 items:
# DEBUG: [0]: "USER123"
# DEBUG: [1]: "ADMIN456"
# DEBUG: [2]: "GUEST789"
# DEBUG: ---
# USER123,ADMIN456,GUEST789
Testing
Run the test suite:
Benchmarks
Run the bench suite:
Contributing
Contributions and suggestions are welcome! Please open issues or pull requests on GitHub.
License
This project is licensed under the MIT License. See LICENSE for details.
Enjoy fast, composable string transformations!