string_pipeline 0.14.0

# Template System

This document describes the template syntax used by `string_pipeline`.

## Contents

- [Quick Start](#quick-start)
- [Template Syntax](#template-syntax)
- [Evaluation Rules](#evaluation-rules)
- [Templates With Literal Text](#templates-with-literal-text)
- [Rich Rendering](#rich-rendering)
- [Operation Reference](#operation-reference)
- [Range Specifications](#range-specifications)
- [Escaping Rules](#escaping-rules)
- [Map Semantics](#map-semantics)
- [Debug Mode](#debug-mode)
- [Troubleshooting](#troubleshooting)

## Quick Start

Templates are enclosed in braces and operations are separated by `|`.

```text
{operation1|operation2|operation3}
```

Example:

```bash
string-pipeline "{split:,:..|map:{upper}|join:-}" "hello,world,test"
# HELLO-WORLD-TEST
```

## Template Syntax

```text
{[!][operation[|operation...]]}
```

| Component      | Required | Description                                |
|----------------|----------|--------------------------------------------|
| `{` `}`        | yes      | Template delimiters                        |
| `!`            | no       | Debug flag, immediately after `{`          |
| operation list | no       | One or more operations separated by a pipe |

Notes:

- `{}` is valid and returns the input unchanged.
- Operations are evaluated from left to right.
- A template can contain either only a template block (`{...}`) or literal text with one or more blocks.

### Shorthand syntax

Shorthand index/range syntax is equivalent to splitting on a space separator.

```text
{1}      == {split: :1}
{1..3}   == {split: :1..3}
{..}     == {split: :..}
```

## Evaluation Rules

The pipeline works with two runtime value types:

- `String`
- `List<String>`

Operations are type-checked during execution.

### Type categories

| Category         | Operations                                                                                                                       |
|------------------|----------------------------------------------------------------------------------------------------------------------------------|
| string -> string | `replace`, `upper`, `lower`, `trim`, `substring`, `append`, `prepend`, `surround`, `quote`, `strip_ansi`, `pad`, `regex_extract` |
| list -> list     | `slice`, `sort`, `unique`, `map`                                                                                                 |
| type-preserving  | `filter`, `filter_not`, `reverse`                                                                                                |
| type-converting  | `split`, `join`                                                                                                                  |

### Final list rendering

If a pipeline ends with a list and no explicit `join`, the list is rendered as a string using the separator from the
most recent `split` or `join` operation in that pipeline.

```text
{split:,:..}                    # "a,b,c" -> "a,b,c"
{split:,:..|sort}               # "c,a,b" -> "a,b,c"
{split:,:..|join:-}             # "a,b,c" -> "a-b-c"
{split:\|:..|split:a:..}        # "apple|banana|cherry" -> "appleabananaacherry"
```

Use an explicit `join` as the final step when output format must be fixed.

## Templates With Literal Text

A template with literal text combines static content and one or more template
sections.

```text
User: {split:,:0} Score: {split:,:1}
```

Examples:

```bash
string-pipeline "Hello {upper}, welcome" "world"
# Hello WORLD, welcome

string-pipeline "Name: {split: :0} Age: {split: :1}" "John 25"
# Name: John Age: 25

string-pipeline "Host: {split: :0|split:=:1} Port: {split: :1|split:=:1} SSL: {split: :-1|split:=:1|upper}" "host=localhost port=8080 ssl=true"
# Host: localhost Port: 8080 SSL: TRUE
```

### Caching behavior

Within one `format()` call, repeated template sections with the same operation sequence and input are cached.

```bash
string-pipeline "First: {split:,:0} Again: {split:,:0}" "apple,banana,cherry"
# First: apple Again: apple
```

## Rich Rendering

The standard rendering path returns only the final string. The rich rendering
path keeps that final string and also exposes the output produced by each
template section.

This is useful when the caller needs to treat template substitutions
differently from literal text, for example to post-process only the dynamic
parts before passing the rendered result to another system.

### Single-input rendering

Use `format_rich()` to format one input while preserving per-section outputs.

```rust
use string_pipeline::Template;

let template = Template::parse("Preview {upper} ({lower})").unwrap();
let result = template.format_rich("MiXeD").unwrap();

assert_eq!(result.rendered(), "Preview MIXED (mixed)");
assert_eq!(result.template_output(0), Some("MIXED"));
assert_eq!(result.template_output(1), Some("mixed"));
```

Each `TemplateOutput` also exposes metadata:

- `template_position()`: position among template sections only
- `overall_position()`: position among all sections, including literals
- `rendered_range()`: byte range inside the final rendered string
- `as_str(result.rendered())`: borrowed view of that section output

The per-section outputs are stored as ranges into the final rendered string, so
the rich path does not retain a second owned string per template section.

### Structured-input rendering

Use `format_with_inputs_rich()` when each template section receives its own
input slice and separator.

```rust
use string_pipeline::Template;

let template = Template::parse("User: {upper} | File: {lower}").unwrap();
let result = template
    .format_with_inputs_rich(
        &[&["john doe", "jane smith"], &["README.MD"]],
        &[" / ", " "],
    )
    .unwrap();

assert_eq!(result.rendered(), "User: JOHN DOE / JANE SMITH | File: readme.md");
assert_eq!(result.template_output(0), Some("JOHN DOE / JANE SMITH"));
assert_eq!(result.template_output(1), Some("readme.md"));
```

In this mode, each rich template output is the fully joined output inserted for
that section after applying the same rules as `format_with_inputs()`.

## Deprecations

Use `Template` as the public type name in new code.

`MultiTemplate` is retained only as a compatibility name in the current
release line and is planned for removal in the next major release.

## Operation Reference

### split

- Syntax: `split:SEPARATOR:RANGE`
- Input: string or list
- Output: string (index range) or list (range)

Notes:

- `RANGE` is required; use `..` for all parts.
- For list input, each item is split and the results are flattened.

```text
{split:,:..}            # split all items by comma
{split: :0..2}          # keep first two parts
{split:\n:-1}          # keep last line

{split: :..|map:{append:,x}|split:,:..|join:-}
# "a b" -> "a-x-b-x"
```

### slice

- Syntax: `slice:RANGE`
- Input: list
- Output: list

```text
{split:,:..|slice:1..3}   # "a,b,c,d" -> "b,c"
```

### join

- Syntax: `join:SEPARATOR`
- Input: list or string
- Output: string

Behavior:

- On lists, joins items using `SEPARATOR`.
- On strings, returns the input unchanged.

```text
{split:,:..|join:-}       # "a,b,c" -> "a-b-c"
{split:,:..|join:}        # "a,b,c" -> "abc"
{join:-}                  # "hello" -> "hello"
```

### substring

- Syntax: `substring:RANGE`
- Input: string
- Output: string

```text
{substring:1..4}          # "hello" -> "ell"
{substring:-3..}          # "hello" -> "llo"
```

### trim

- Syntax: `trim[:CHARS][:DIRECTION]`
- Input: string
- Output: string
- `DIRECTION`: `both` (default), `left`, `right`

```text
{trim}                    # trim whitespace
{trim:left}               # trim from start only
{trim:xy}                 # trim x/y from both ends
{trim:*-+:right}          # trim from right only
```

### pad

- Syntax: `pad:WIDTH[:CHAR[:DIRECTION]]`
- Input: string
- Output: string
- `DIRECTION`: `left`, `right` (default), `both`

```text
{pad:5}                   # "hi" -> "hi   "
{pad:5:0:left}            # "42" -> "00042"
```

### upper

- Syntax: `upper`
- Input: string
- Output: string

```text
{upper}                   # "hello" -> "HELLO"
```

### lower

- Syntax: `lower`
- Input: string
- Output: string

```text
{lower}                   # "HELLO" -> "hello"
```

### append

- Syntax: `append:TEXT`
- Input: string
- Output: string

```text
{append:.txt}             # "file" -> "file.txt"
```

### prepend

- Syntax: `prepend:TEXT`
- Input: string
- Output: string

```text
{prepend:/tmp/}           # "file.txt" -> "/tmp/file.txt"
```

### surround

- Syntax: `surround:TEXT`
- Input: string
- Output: string

```text
{surround:"}             # "hello" -> "\"hello\""
{surround:**}             # "text" -> "**text**"
```

### quote

- Syntax: `quote:TEXT`
- Input: string
- Output: string

`quote` is an alias of `surround`.

```text
{quote:'}                 # "hello" -> "'hello'"
```

### replace

- Syntax: `replace:s/PATTERN/REPLACEMENT/FLAGS`
- Input: string
- Output: string
- Supported flags: `g`, `i`, `m`, `s`

```text
{replace:s/hello/hi/}     # first match
{replace:s/\d+/NUM/g}     # global replacement
{replace:s/(.+)/[$1]/}    # capture groups
```

### regex_extract

- Syntax: `regex_extract:PATTERN[:GROUP]`
- Input: string
- Output: string

If there is no match, returns an empty string.

```text
{regex_extract:\d+}        # first number
{regex_extract:@(.+):1}    # group extraction
```

### sort

- Syntax: `sort[:DIRECTION]`
- Input: list
- Output: list
- `DIRECTION`: `asc` (default), `desc`

```text
{split:,:..|sort}          # "c,a,b" -> "a,b,c"
{split:,:..|sort:desc}     # "a,b,c" -> "c,b,a"
```

### reverse

- Syntax: `reverse`
- Input: string or list
- Output: same type as input

```text
{reverse}                  # "hello" -> "olleh"
{split:,:..|reverse}       # "a,b,c" -> "c,b,a"
```

### unique

- Syntax: `unique`
- Input: list
- Output: list

Keeps first occurrence order.

```text
{split:,:..|unique}        # "a,b,a,c,b" -> "a,b,c"
```

### filter

- Syntax: `filter:PATTERN`
- Input: string or list
- Output: same type as input

```text
{split:,:..|filter:^test}  # keep list items starting with "test"
```

### filter_not

- Syntax: `filter_not:PATTERN`
- Input: string or list
- Output: same type as input

```text
{split:,:..|filter_not:^#} # remove items starting with "#"
```

### strip_ansi

- Syntax: `strip_ansi`
- Input: string
- Output: string

```text
{strip_ansi}               # remove ANSI escape sequences
```

### map

- Syntax: `map:{operation1|operation2|...}`
- Input: list
- Output: list

Notes:

- Nested `map` is not allowed.
- String operations and list operations are both available inside `map`.

```text
{split:,:..|map:{trim|upper}}                    # " a , b " -> "A,B"
{split:,:..|map:{split: :..|join:-}}             # "hello world,foo bar" -> "hello-world,foo-bar"
{split:,:..|map:{split: :..|filter:o}}           # "hello world,foo bar,test orange" -> "hello world,foo,orange"
```

### shorthand index and ranges

Shorthand forms operate as `split` with a space separator.

```text
{1}                      # "a b c d" -> "b"
{-1}                     # "a b c d" -> "d"
{1..3}                   # "a b c d" -> "b c"
{1..=3}                  # "a b c d" -> "b c d"
{..3}                    # "a b c d" -> "a b c"
{..}                     # "a b c d" -> "a b c d"
```

## Range Specifications

Ranges are used by `split`, `slice`, `substring`, and shorthand syntax.

| Syntax  | Description         |
|---------|---------------------|
| `N`     | single index        |
| `N..M`  | exclusive range     |
| `N..=M` | inclusive range     |
| `N..`   | from index to end   |
| `..M`   | from start to `M-1` |
| `..=M`  | from start to `M`   |
| `..`    | full range          |

Negative indexes count from the end (`-1` is last item).

Edge behavior:

- Single indexes are clamped to valid bounds (out-of-range resolves to nearest valid index).
- Ranges are clamped to valid bounds.
- If computed start is greater than or equal to end, the result is empty.
- Empty input always returns empty output.

## Escaping Rules

### Simple arguments

For operations such as `append`, `prepend`, `join`, `surround`, `quote`, and `trim` arguments, escape these characters when needed:

| Character | Escape |
|-----------|--------|
| `:`       | `\:`   |
| pipe      | `\|`   |
| `{`       | `\{`   |
| `}`       | `\}`   |
| `\`       | `\\`   |

### Special escape sequences

`process_arg` supports these sequences in simple/split arguments:

- `\n` newline
- `\t` tab
- `\r` carriage return
- `\:` literal colon
- `\|` literal pipe
- `\\` literal backslash
- `\/` literal slash
- `\{` literal `{`
- `\}` literal `}`

Any other `\X` sequence is treated as literal `X`.

### Regex arguments

For `filter`, `filter_not`, and `regex_extract`, the pattern is read as raw template content and passed to the regex engine.

Examples:

```text
{append:\:value}         # append ":value"
{split:\:\::..|join:-}   # split on "::"
{filter:\.txt$}          # regex pattern for .txt suffix
```

When calling from a shell, prefer single quotes around templates to reduce extra shell escaping.

## Map Semantics

`map` runs a sub-pipeline for each list item.

Important details:

- Each item is processed independently.
- If a map item ends as a list, it is auto-rendered to a string using that sub-pipeline's current separator.
- After mapping, the outer pipeline continues with a list of mapped strings.

```text
Input: "hello world,foo bar,test orange"
Template: {split:,:..|map:{split: :..|filter:o}}
Output: "hello world,foo,orange"
```

## Debug Mode

Enable debug mode with either:

- Inline template flag: `{!...}`
- CLI flag: `--debug` or `-d`

Debug output is written to `stderr`; final result is written to `stdout`.

`--quiet` suppresses debug output even when debug is enabled inline or via CLI.

```bash
string-pipeline "{!split:,:..|map:{upper}|join:-}" "hello,world"
```

For full debug output structure and examples, see `docs/debug-system.md`.

## Troubleshooting

Common issues:

- Parse errors: check missing braces, missing separators, or invalid operation names.
- Type errors: apply string-only operations through `map` when working with lists.
- Empty output: verify regex and range expressions; filter/range steps may remove all items.

Quick checks:

```bash
string-pipeline --validate '{split:,:..|map:{upper}|join:-}'
string-pipeline -d '{split:,:..|map:{upper}|join:-}' 'a,b,c'
```

Related documentation:

- `docs/command-line-options.md`
- `docs/debug-system.md`
- `docs/benchmarking.md`