# Architecture Internals
This document describes the internal architecture of the Hypen Engine, including the reactive system, reconciliation algorithm, IR expansion, and patch generation.
## Table of Contents
- [System Overview](#system-overview)
- [IR System](#ir-system)
- [Reactive System](#reactive-system)
- [Reconciliation Algorithm](#reconciliation-algorithm)
- [Patch Generation](#patch-generation)
- [Instance Tree](#instance-tree)
- [Component Resolution](#component-resolution)
- [State Management](#state-management)
---
## System Overview
The engine follows a unidirectional data flow:
```
Hypen DSL Source
│
▼
┌─────────────┐
│ Parser │ Chumsky combinator parser
│ (AST) │ ComponentSpecification → arguments, applicators, children
└──────┬──────┘
│ ast_to_ir_node()
▼
┌─────────────┐
│ IR │ Element / IRNode (ForEach, Conditional)
│ Expansion │ Component registry, slot resolution, Tailwind expansion
└──────┬──────┘
│
▼
┌─────────────┐
│ Reactive │ DependencyGraph: path → NodeId mapping
│ System │ Prefix index for O(log n) affected-node lookup
└──────┬──────┘
│
▼
┌─────────────┐
│ Reconciler │ Keyed diffing with LIS algorithm
│ (diff.rs) │ InstanceTree management
└──────┬──────┘
│
▼
┌─────────────┐
│ Patches │ Create, SetProp, SetText, Insert, Move, Remove
│ (output) │ Platform-agnostic mutation instructions
└──────┬──────┘
│
▼
┌─────────────┐
│ Renderer │ DOM, Canvas, iOS, Android, Remote UI
│ (platform) │ Applies patches to platform objects
└─────────────┘
```
## IR System
### Source Files
- `src/ir/node.rs` — Core types: `IRNode`, `Element`, `Value`, `Props`
- `src/ir/expand.rs` — AST-to-IR lowering, Tailwind expansion, binding extraction
- `src/ir/component.rs` — Component registry, resolution, passthrough/lazy components
### IRNode Enum
Control flow is represented as first-class IR nodes:
```rust
pub enum IRNode {
Element(Element), // Regular UI element
ForEach { ... }, // Iteration construct
Conditional { ... }, // When/If construct
}
```
This design enables exhaustive pattern matching in the reconciler, so the compiler catches any missing control flow handling.
### Value Types
Props can hold four types of values:
```rust
pub enum Value {
Static(serde_json::Value), // Literal: "hello", 42, true
Binding(Binding), // ${state.user.name}
TemplateString { // "Count: ${state.count}"
template: String,
bindings: Vec<Binding>,
},
Action(String), // @actions.signIn
}
```
### Props (Arc-Wrapped)
Props use `Arc<IndexMap<String, Value>>` for O(1) cloning during reconciliation:
```rust
pub struct Props(Arc<PropsMap>);
```
Copy-on-write semantics: calling `make_mut()` clones the inner map only if there are multiple references. This means cloning an element during reconciliation is nearly free, while mutations only pay the cost when necessary.
### AST-to-IR Conversion
The `ast_to_ir_node()` function detects control flow components by name:
```
"ForEach" → IRNode::ForEach { source, item_name, key_path, template, props }
"When" → IRNode::Conditional { value, branches, fallback }
"If" → IRNode::Conditional { value, [true-branch], fallback }
other → IRNode::Element(element)
```
During conversion:
1. Arguments are mapped to props (named or positional)
2. Applicators become props with dotted keys (e.g., `.fontSize(18)` → `fontSize.0: 18`)
3. `.tw()` applicators are expanded to individual CSS properties via `hypen-tailwind-parse`
4. `${state.xxx}` strings are parsed into `Value::Binding`
5. `@actions.xxx` strings become `Value::Action`
6. Template strings with mixed bindings become `Value::TemplateString`
---
## Reactive System
### Source Files
- `src/reactive/binding.rs` — Binding parsing and representation
- `src/reactive/graph.rs` — Dependency graph with prefix index
### Binding Model
Bindings represent reactive references to state paths:
```rust
pub struct Binding {
pub source: BindingSource, // State or Item
pub path: Vec<String>, // ["user", "name"]
}
```
Two sources:
- `BindingSource::State` — `${state.user.name}` references module state
- `BindingSource::Item` — `${item.name}` references the current iteration item in a ForEach
### Dependency Graph
The `DependencyGraph` maps state paths to the nodes that depend on them:
```rust
pub struct DependencyGraph {
dependencies: IndexMap<String, IndexSet<NodeId>>, // path → nodes
node_bindings: IndexMap<NodeId, IndexSet<String>>, // node → paths
prefix_index: BTreeMap<String, IndexSet<String>>, // prefix → full paths
}
```
#### Registration
When a node with bindings is created, each binding's path is registered:
```
Text("${state.user.name}")
→ registers dependency: "user.name" → node_42
→ prefix index entries: "user" → {"user.name"}, "user.name" → {"user.name"}
```
#### Affected Node Lookup
When a state path changes, `get_affected_nodes(changed_path)` finds all nodes that need re-rendering using three strategies:
1. **Exact match**: Nodes depending on exactly `changed_path`
2. **Parent paths**: Nodes depending on any prefix of `changed_path`
- If `"user.name"` changes, nodes depending on `"user"` are affected
3. **Child paths**: Nodes depending on any path starting with `changed_path`
- If `"user"` changes, nodes depending on `"user.name"` and `"user.email"` are affected
The prefix index (`BTreeMap`) makes child-path lookups O(log n) instead of O(n).
#### Example
```
Dependencies registered:
node_1 → "user"
node_2 → "user.name"
node_3 → "user.email"
node_4 → "posts"
State change: "user" changed
→ Exact match: node_1
→ Child paths: node_2, node_3 (via prefix index)
→ Result: {node_1, node_2, node_3}
State change: "user.name" changed
→ Exact match: node_2
→ Parent paths: node_1 (depends on "user", a prefix of "user.name")
→ Result: {node_1, node_2}
State change: "user.email" changed
→ Exact match: node_3
→ Parent paths: node_1
→ Result: {node_1, node_3}
```
### Path-Based (Not Value-Based)
The engine tracks **which paths changed**, not **what values changed**. It does not store or compare old/new state values. The host (JavaScript/Kotlin/Swift) is responsible for signaling which paths have changed.
This means:
- Setting `state.count = 5` when it's already 5 still triggers re-rendering
- The host should track dirty paths to avoid unnecessary updates
---
## Reconciliation Algorithm
### Source Files
- `src/reconcile/diff.rs` — Core diffing algorithm (~1900 lines)
- `src/reconcile/tree.rs` — Instance tree data structure
- `src/reconcile/patch.rs` — Patch type definitions
### Overview
Reconciliation compares the new IR tree against the existing instance tree and generates the minimal set of patches to bring the instance tree up to date.
Two entry points:
- `reconcile()` — For `Element`-based trees (legacy)
- `reconcile_ir()` — For `IRNode`-based trees (first-class control flow)
### Keyed Children Diffing
The algorithm for reconciling children with keys:
**Phase 1: Match children**
```
Old children: [A, B, C, D, E] (by key)
New children: [B, D, A, F] (by key)
Build maps:
old_keyed: { A→node1, B→node2, C→node3, D→node4, E→node5 }
For each new child:
B → found in old_keyed → reconcile node2, remove from map
D → found in old_keyed → reconcile node4, remove from map
A → found in old_keyed → reconcile node1, remove from map
F → not found → create new node
```
**Phase 2: Minimize moves with LIS**
The Longest Increasing Subsequence algorithm determines which nodes are already in correct relative order and don't need Move patches.
```
New children: [B, D, A, F]
Old positions: [1, 3, 0, None] (None = newly created)
LIS of old positions: [1, 3] → indices 0, 1 (B, D are in order)
Nodes NOT in LIS need Move patches:
A (index 2) → Move
F (index 3) → Insert (new)
```
**Phase 3: Cleanup**
Unused old children (C, E) are removed with `Remove` patches and their dependencies are cleaned from the graph.
### Control Flow Reconciliation
ForEach and Conditional nodes have special reconciliation paths:
**ForEach:**
1. Re-evaluate the source binding to get the current array
2. For each item, replace `${item.*}` bindings with actual values
3. Generate keys using `key_path` or default `{item_name}-{index}`
4. Reconcile generated children against existing children using keyed diffing
**Conditional:**
1. Re-evaluate the condition value
2. Match against branch patterns
3. If the matching branch changed, remove old branch children and create new ones
4. If the same branch matches, reconcile children in place
### Transparent Control Flow Nodes
Control flow nodes (`__ForEach`, `__Conditional`) exist in the instance tree but are **transparent** to the DOM. Their children render directly into the control flow node's parent. This means:
```hypen
Column {
ForEach(items: ${state.items}) {
Text(${item.name})
}
}
```
Results in the DOM structure:
```
Column
├── Text("Alice") ← direct child of Column, not of __ForEach
├── Text("Bob")
└── Text("Charlie")
```
---
## Patch Generation
### Patch Types
```rust
pub enum Patch {
Create { id, element_type, props }, // Create a new node
SetProp { id, name, value }, // Update a single property
SetText { id, text }, // Update text content
Insert { parent_id, id, before_id }, // Insert into parent
Move { parent_id, id, before_id }, // Move to new position
Remove { id }, // Remove from tree
}
```
All patches use string IDs (serialized `NodeId`). `before_id` is `Option<String>` — `None` means append to end.
### Serialization
Patches are serialized as JSON with `camelCase` tag names:
```json
[
{ "type": "create", "id": "n1", "elementType": "Text", "props": { "text": "Hello" } },
{ "type": "insert", "parentId": "root", "id": "n1", "beforeId": null },
{ "type": "setProp", "id": "n1", "name": "text", "value": "World" }
]
```
### Event Handling
Event handling (click, input, etc.) is **not** part of the patch system. Events are managed at the renderer level. The engine passes event-related props (e.g., `onClick: @actions.increment`) as regular props, and the renderer is responsible for attaching event listeners.
---
## Instance Tree
### Source File
- `src/reconcile/tree.rs`
### Structure
The instance tree is a SlotMap-based tree of `InstanceNode` values:
```rust
pub struct InstanceTree {
nodes: SlotMap<NodeId, InstanceNode>,
root: Option<NodeId>,
}
pub struct InstanceNode {
pub id: NodeId,
pub element_type: String,
pub props: IndexMap<String, serde_json::Value>, // Resolved values
pub raw_props: Props, // Unresolved (with bindings)
pub ir_node_template: Option<Arc<IRNode>>, // For ForEach/Conditional re-rendering
pub control_flow: Option<ControlFlowKind>,
pub key: Option<String>,
pub parent: Option<NodeId>,
pub children: im::Vector<NodeId>,
}
```
Key properties:
- `props` contains **resolved** values (bindings evaluated against state)
- `raw_props` contains **unresolved** values (with `Binding` variants intact) for change detection
- `ir_node_template` stores the original IRNode for control flow nodes so they can be re-rendered on state change
- `children` uses `im::Vector` for O(1) structural sharing during clones
### NodeId
`NodeId` is a SlotMap key, providing:
- Stable identity across re-renders
- O(1) lookup
- Automatic reuse of removed slots
- Type-safe (can't mix up with other IDs)
---
## Component Resolution
### Source File
- `src/ir/component.rs`
### Component Types
| **Regular** | Template function expands props into an element tree |
| **Passthrough** | Props preserved, children expanded, no template transform |
| **Lazy** | Children not expanded until explicitly requested |
### Resolution Flow
```
Component name ("ProfileCard")
│
▼
ComponentRegistry.resolve()
│
├── Found in registry? → Use registered component
│
└── Not found? → Call ComponentResolver callback
│
├── Resolver returns source code → Parse & expand
│
└── Resolver returns None → Keep as-is (primitive)
```
The `ComponentResolver` is a callback set by the host that resolves component names to source code. This enables file-based component discovery and lazy loading:
```typescript
engine.setComponentResolver((name, context) => {
const source = readComponentFile(name);
return source ? { source, path: `/components/${name}`, passthrough: false, lazy: false } : null;
});
```
---
## State Management
### Source Files
- `src/state.rs` — StateChange type and tracking
- `src/engine.rs` — State update orchestration
### Update Flow
```
Host calls engine.updateState({ count: 42 })
│
▼
Merge state patch into current state
│
▼
Extract changed paths: ["count"]
│
▼
DependencyGraph.get_affected_nodes("count")
│
▼
Re-reconcile affected nodes
│
▼
Generate and emit patches
```
### Single Active Module
The engine maintains one active module at a time. All `${state.xxx}` bindings resolve against this single module's state:
```rust
pub struct Engine {
active_module: Option<ModuleInstance>,
// ...
}
```
Cross-module communication requires explicit prop passing or a host-level coordination layer.
### Revision Tracking
Each state update increments a revision counter. This is used by the Remote UI protocol to ensure patches are applied in order:
```rust
pub fn revision(&self) -> u64
```
The serialization layer can include revision numbers and optional integrity hashes for clients to verify patch ordering.