code-product-lib 0.0.1

**Table of Contents**
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->


- [Code Product Library](#code-product-library)
- [The Product Syntax](#the-product-syntax)
  - [Scopes](#scopes)
  - [Product Definitions](#product-definitions)
  - [References](#references)
  - [Escaping the @ character](#escaping-the--character)
- [Examples](#examples)
  - [Preprocessor like Macro Replacement](#preprocessor-like-macro-replacement)
- [EBNF Syntax](#ebnf-syntax)

<!-- markdown-toc end -->

# Code Product Library

This library implements a engine for expanding one `TokenTree` into another `TokenTree` by
expanding code productions.


# The Product Syntax

The product syntax contains three elements described in detail below:
 1. Scopes for product definitions
    ```text
    @{ ... }
    ```
 2. The product definitions
    ```text
    @( ... )
    ```
 3. References to products either indexed or named
    ```text
    @0 @foo
    ```

Since we use rust `TokenTrees` the only requirement is that everything tokenizes. That means
that literals must be properly formatted and that all opening brackets must be closed.

All syntactic elements relevant to the product syntax start with the pound sign `@`.

## Scopes

Allow local product definitions in @ followed by braces.  These scopes are evaluated
inside-out, named references that are not local will be forwarded to the parent scope.


## Product Definitions

A list of nested parenthesis. The inner parenthesis contain the produced rust tokens. A
product definition can be named or unnamed, hidden or visible. This leads to the three forms
(unnamed-hidden is not supported).

Names can be used to reference the product at multiple places. This works for names from outer
scopes as well. Unnamed definitions can only be referenced by their numeric position in the product
list, these will be local and not reference outer scopes.

Hidden definitions will not be expanded at the place of their definition, but can be
referenced later. This is useful for defining a production first and using it multiple times.
Visible definitions will be expanded at the place of their definition and can  be
referenced later as well.

 * `@((a)(b))` - unnamed, visible  
   Defining a unnamed product will implicitly leave a numeric reference to it in place. Thus
   it will be expanded at the place of its definition. This is especially useful for defining
   a production that is used only once.
 * `@(@name:(a)(b))` - named, visible  
    Defining a named production with a `@` in front of its name will leave a reference to it
    in place. Thus it will be expanded at the place of its definition.
 * `@(name:(a)(b))` - named, hidden  
   Defining a named production without the `@` in front of its name will leave no reference
   to it. Thus it will not be expanded at the place of its definition. This can be used when
   one want to define a set of productions at the start of a scope which later become
   referenced.

The number of the product definitions starts with one, named definitions are included in the
numbering.

**Note 1:**  
Productions can only contain rust tokens, not product definitions or
references. This is intentional chosen for simplicity for now.

**Note 2:**  
Code expansion is the product of all defined products. Hidden products that are not referenced
this will it will still create the same code multiple times. This also applies to empty
definitions:

```text
@(unused (1)()(3))
@(used (a)(b))
foo(@used)
```

will expand to:

```text
foo(a)
foo(a)
foo(a)
foo(b)
foo(b)
foo(b)
```


## References

References are used to reference an earlier defined production. They are written as `@`
followed by a number or the name of a named product definition. When named this can refer to a
name from a outer scope. Numbered references are only valid in the scope of the product
definition they are defined, indexing starts at zero.

Unresolved references will lead to a compile error. Everything has to be defined before being
used.


## Escaping the @ character

The `@` character is used to start product syntax elements. To use a literal `@` character
just double it. 

# Examples

The following three examples will all expand to:

```text
impl Trait<Foo> for MyType<Foo> {}
impl Trait<Bar> for MyType<Bar> {}
impl Trait<Baz> for MyType<Baz> {}
```

 1. Using unnamed product definitions:
```text
impl Trait<@((Foo)(Bar)(Baz))> for MyType<@1> {}
```

 2. Using a named, hidden product definition: 
```text
@(T: (Foo)(Bar)(Baz))
impl Trait<@T> for MyType<@T> {}
```

 3. Using a named, visible product definition:
```text
impl Trait<@(@T: (Foo)(Bar)(Baz))> for MyType<@T> {}
```


## Preprocessor like Macro Replacement

When product definitions have only one element to expand then code product expansion acts like
a simple preprocessor substituting defined products. For example:

```text
@(T: (Foo))
@(U: (Bar))
impl Trait<@T> for MyType<@U> {}
```

will expand to:

```text
impl Trait<Foo> for MyType<Bar> {}
```


# EBNF Syntax

Here is a informal code product syntax as EBNF for reference. The actual implementations
differs slightly depending on what `ScopeMode` is chosen.

```ebnf
start = { product_entity | rust } ;
rust = ?any_valid_rust_token? ;
product_entity = "@" ( product_scope | product_definition | product_reference ) ;
product_scope = "{" { product_entity | rust } "}" ;
product_definition = "(" [ "@" product_name ":" ] { product } ")" ;
product = "(" { rust } ")" ;
product_name = ?identifier? ;
product_reference = "#" ( ?number? | ?identifier? ) ;
```