pixelsrc 0.2.0

Pixelsrc - GenAI-native pixel art format and compiler
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
# Phase 12: Composition Tiling (`cell_size` Extension)

**Goal:** Extend the composition system to support tiling - arranging sprites at larger cell sizes to enable chunked generation of larger images.

**Status:** Complete

**Depends on:** Phase 2 (Composition) complete

---

## Motivation

### The Context Problem

GenAI models have limited context windows. A 64x64 sprite with semantic tokens like `{skin}{hair}{outline}` requires significant token budget. Larger images (128x128, 256x256) become impractical to generate in a single pass.

### The Solution: Tiling

Instead of generating one massive sprite, generate multiple smaller sprites (tiles) and compose them into a larger image. This:

1. Keeps each generation within comfortable context limits
2. Allows parallel generation of independent regions
3. Enables progressive refinement (overview → details)
4. Reuses the existing composition mental model

### Why Extend Composition (Not New Concept)

The current composition system already places sprites on a canvas using a character map:

```jsonl
{"type": "composition", "size": [64, 64], "sprites": {"H": "hero"}, "layers": [{"map": ["..H.."]}]}
```

Each character represents a sprite placed at a position. **Tiling is the same concept at a different scale** - instead of placing sprites at pixel positions, we place them at region positions.

Adding `cell_size` to composition unifies both use cases under one mental model.

---

## Design

### Current Composition (Implicit `cell_size: [1, 1]`)

```jsonl
{"type": "sprite", "name": "hero", "size": [16, 16], "grid": [...]}
{"type": "composition", "name": "scene", "size": [64, 64],
  "sprites": {"H": "hero", "T": "tree"},
  "layers": [{"map": [
    "........",
    "..H.....",
    "........",
    "T......T"
  ]}]
}
```

Each character in the map = 1 pixel position. Sprites are placed at that coordinate.

### Extended Composition (Explicit `cell_size`)

```jsonl
{"type": "sprite", "name": "sky_left", "size": [32, 32], "grid": [...]}
{"type": "sprite", "name": "sky_right", "size": [32, 32], "grid": [...]}
{"type": "sprite", "name": "ground_left", "size": [32, 32], "grid": [...]}
{"type": "sprite", "name": "ground_right", "size": [32, 32], "grid": [...]}

{"type": "composition", "name": "scene", "size": [64, 64],
  "cell_size": [32, 32],
  "sprites": {"A": "sky_left", "B": "sky_right", "C": "ground_left", "D": "ground_right"},
  "layers": [{"map": [
    "AB",
    "CD"
  ]}]
}
```

Each character in the map = one 32x32 cell. The composition renders a 2x2 grid of 32x32 tiles = 64x64 total.

### Key Properties

| Property | Default | Description |
|----------|---------|-------------|
| `cell_size` | `[1, 1]` | Size of each map character in pixels |
| `size` | Required | Total canvas size (must be divisible by cell_size) |

### Validation Rules

1. `size[0]` must be divisible by `cell_size[0]`
2. `size[1]` must be divisible by `cell_size[1]`
3. Sprites placed in cells should ideally match `cell_size` (warning if not, still renders)
4. Map dimensions must match `size / cell_size`

---

## Examples

### Example 1: 2x2 Tiled Scene (64x64 from 32x32 tiles)

```jsonl
{"type": "palette", "name": "nature", "colors": {"{_}": "#00000000", "{sky}": "#87CEEB", "{grass}": "#228B22"}}

{"type": "sprite", "name": "tile_tl", "size": [32, 32], "palette": "nature", "grid": [...]}
{"type": "sprite", "name": "tile_tr", "size": [32, 32], "palette": "nature", "grid": [...]}
{"type": "sprite", "name": "tile_bl", "size": [32, 32], "palette": "nature", "grid": [...]}
{"type": "sprite", "name": "tile_br", "size": [32, 32], "palette": "nature", "grid": [...]}

{"type": "composition", "name": "landscape", "size": [64, 64], "cell_size": [32, 32],
  "sprites": {"1": "tile_tl", "2": "tile_tr", "3": "tile_bl", "4": "tile_br"},
  "layers": [{"map": ["12", "34"]}]
}
```

### Example 2: 3x3 Tiled Scene (96x96 from 32x32 tiles)

```jsonl
{"type": "composition", "name": "large_scene", "size": [96, 96], "cell_size": [32, 32],
  "sprites": {
    "A": "sky_left", "B": "sky_mid", "C": "sky_right",
    "D": "mid_left", "E": "mid_mid", "F": "mid_right",
    "G": "ground_left", "H": "ground_mid", "I": "ground_right"
  },
  "layers": [{"map": [
    "ABC",
    "DEF",
    "GHI"
  ]}]
}
```

### Example 3: Mixed - Tiled Background + Placed Sprites

```jsonl
{"type": "composition", "name": "game_scene", "size": [128, 64], "cell_size": [32, 32],
  "sprites": {
    "S": "sky_tile",
    "G": "ground_tile"
  },
  "layers": [
    {"map": ["SSSS", "GGGG"]}
  ]
}

{"type": "composition", "name": "game_scene_with_characters", "size": [128, 64],
  "sprites": {"H": "hero", "E": "enemy"},
  "layers": [
    {"base": "game_scene"},
    {"map": ["....H...", "......E."]}
  ]
}
```

This shows how tiling and traditional composition can be combined - a tiled background with characters placed on top.

---

## Implementation Notes

### Changes to `composition.rs`

1. Add `cell_size` field to `Composition` struct (default `[1, 1]`)
2. Modify map parsing to scale positions by `cell_size`
3. Add validation for size/cell_size divisibility
4. Update rendering loop to place sprites at scaled positions

### Changes to `models.rs`

```rust
#[derive(Debug, Deserialize)]
pub struct Composition {
    pub name: String,
    pub size: [u32; 2],
    #[serde(default = "default_cell_size")]
    pub cell_size: [u32; 2],
    pub sprites: HashMap<char, String>,
    pub layers: Vec<Layer>,
}

fn default_cell_size() -> [u32; 2] {
    [1, 1]
}
```

### Rendering Logic Change

Current:
```rust
// Position = map character position
let x = col as u32;
let y = row as u32;
```

New:
```rust
// Position = map character position * cell_size
let x = col as u32 * composition.cell_size[0];
let y = row as u32 * composition.cell_size[1];
```

---

## Tasks

### Task 12.1: Add `cell_size` to Composition Model

- Add `cell_size` field to `Composition` struct in `models.rs`
- Default to `[1, 1]` for backwards compatibility
- Add serde deserialization support

### Task 12.2: Update Composition Rendering

- Modify `composition.rs` to use `cell_size` when calculating sprite positions
- Ensure existing compositions (no `cell_size`) work identically

### Task 12.3: Add Validation

- Validate `size` is divisible by `cell_size`
- Validate map dimensions match expected grid (`size / cell_size`)
- Warn (don't error in lenient mode) if sprite size doesn't match `cell_size`

### Task 12.4: Add Examples and Tests

- Create `examples/tiled_scene.jsonl` demonstrating the feature
- Add unit tests for `cell_size` rendering
- Add integration tests for validation rules

### Task 12.5: Update Documentation

- Update `docs/spec/format.md` with `cell_size` specification
- Add examples to website gallery
- Update system prompts with tiling guidance

---

## AI Generation Workflow

This is how an AI would use tiling to generate a large image:

### Step 1: Plan the Layout

```
User: "Create a 128x128 forest scene"
AI thinks: "128x128 is large. I'll tile it as 4x4 grid of 32x32 tiles."
```

### Step 2: Generate Overview (Optional)

AI might first generate a low-res "guide" to plan the scene:
```jsonl
{"type": "sprite", "name": "guide", "size": [4, 4], "grid": [
  "{sky}{sky}{sky}{sky}",
  "{tree}{sky}{sky}{tree}",
  "{tree}{grass}{grass}{tree}",
  "{grass}{grass}{grass}{grass}"
]}
```

### Step 3: Generate Each Tile

AI generates each 32x32 tile, using the guide for context:
```
"Generate tile (0,0): This is top-left, should be mostly sky based on guide"
"Generate tile (1,0): This is top-middle, pure sky"
...
```

### Step 4: Compose Final Image

```jsonl
{"type": "composition", "name": "forest", "size": [128, 128], "cell_size": [32, 32],
  "sprites": {...},
  "layers": [{"map": ["ABCD", "EFGH", "IJKL", "MNOP"]}]
}
```

---

## Relationship to Guide Concept

The "guide" mentioned above is **not a format feature** - it's a workflow pattern:

1. AI generates a small sprite as a planning sketch
2. AI uses that sprite as context when generating detailed tiles
3. The renderer only sees the final composition

This keeps the format simple while enabling sophisticated generation workflows. The guide is just another sprite that the AI references mentally.

---

## Composition as Editing Jig

### The Alignment Problem

When AI generates complex sprites with multiple elements (like a `{pxl}` banner with brackets and letters), maintaining alignment is extremely difficult. Each element has boundaries that shouldn't overlap, and manual grid editing loses track of column positions.

### Solution: Compose Instead of Monolith

Instead of generating one large sprite with everything, generate each element as a separate sprite and use composition to assemble them:

```jsonl
{"type": "palette", "name": "dracula", "colors": {
  "{_}": "#00000000",
  "{c}": "#8BE9FD",
  "{k}": "#FF79C6",
  "{p}": "#BD93F9",
  "{x}": "#50FA7B",
  "{l}": "#FFB86C"
}}

{"type": "sprite", "name": "bracket_l", "size": [3, 8], "palette": "dracula", "grid": [
  "{_}{c}{c}",
  "{c}{_}{_}",
  "{c}{_}{_}",
  "{c}{_}{_}",
  "{_}{c}{_}",
  "{c}{_}{_}",
  "{c}{_}{_}",
  "{_}{c}{c}"
]}

{"type": "sprite", "name": "letter_p", "size": [3, 8], "palette": "dracula", "grid": [
  "{_}{_}{_}",
  "{_}{_}{_}",
  "{_}{_}{_}",
  "{p}{p}{p}",
  "{p}{_}{p}",
  "{p}{p}{p}",
  "{p}{_}{_}",
  "{p}{_}{_}"
]}

{"type": "sprite", "name": "letter_x", "size": [3, 8], "palette": "dracula", "grid": [
  "{_}{_}{_}",
  "{_}{_}{_}",
  "{_}{_}{_}",
  "{x}{_}{x}",
  "{_}{x}{_}",
  "{_}{x}{_}",
  "{x}{_}{x}",
  "{_}{_}{_}"
]}

{"type": "sprite", "name": "letter_l", "size": [4, 8], "palette": "dracula", "grid": [
  "{_}{_}{_}{_}",
  "{_}{_}{_}{_}",
  "{_}{_}{_}{_}",
  "{l}{_}{_}{_}",
  "{l}{_}{_}{_}",
  "{l}{_}{_}{_}",
  "{l}{l}{l}{l}",
  "{_}{_}{_}{_}"
]}

{"type": "sprite", "name": "bracket_r", "size": [3, 8], "palette": "dracula", "grid": [
  "{k}{k}{_}",
  "{_}{_}{k}",
  "{_}{_}{k}",
  "{_}{_}{k}",
  "{_}{k}{_}",
  "{_}{_}{k}",
  "{_}{_}{k}",
  "{k}{k}{_}"
]}

{"type": "composition", "name": "banner_pxl", "size": [18, 8],
  "sprites": {"{": "bracket_l", "p": "letter_p", "x": "letter_x", "l": "letter_l", "}": "bracket_r", ".": null},
  "layers": [{"map": ["{.p.x.l.}"]}]
}
```

### Benefits for AI

1. **Isolated editing** - Modify one letter without breaking others
2. **Enforced boundaries** - Each sprite has fixed dimensions, can't bleed
3. **Easier reasoning** - Work on 3x8 grid instead of 18x8
4. **Reusable elements** - Same bracket sprites for multiple banners
5. **Clear spacing** - Use `.` (null) sprites for gaps

### Workflow

1. Define slot sizes (all letters 3 wide, brackets 3 wide, gaps 1 wide)
2. Generate each element sprite independently
3. Compose with map showing layout
4. Adjust individual sprites without affecting others

This transforms the hard problem of "generate aligned 18x8 banner" into the easier problem of "generate five small sprites and compose them."

### Multi-File Support

For context efficiency, element sprites can live in separate files in the same directory:

```bash
pxl render bracket_l.jsonl letter_p.jsonl letter_x.jsonl banner.jsonl
```

CLI builds a shared namespace - compositions reference sprites by name regardless of which file defined them. See BACKLOG for future project mode ideas.

---

## Future Considerations

Not in scope for Phase 12:

| Feature | Notes |
|---------|-------|
| Edge constraints | Formal `edges` field to specify tile connectivity (see BACKLOG) |
| Overlap/blending | Tiles that overlap for seamless transitions |
| Auto-tiling | Renderer automatically splits large sprites into tiles |
| Tile libraries | Reusable tile collections for terrain, etc. |

---

## Success Criteria

1. Existing compositions (no `cell_size`) render identically (backwards compatible)
2. `cell_size` compositions render tiles at correct positions
3. Validation catches size/cell_size mismatch
4. Examples demonstrate practical tiled scene creation
5. Documentation explains tiling workflow for AI generation