Module preprocess 

.opy preprocessing: includes, #!define macros (textual and __script__ JavaScript-backed), #!postCompileHook, and expansion.

Operates at the token level, matching the reference frontend’s observable behavior: #!include "file.opy" splices the included file’s tokens at the directive site; #!define NAME value and #!define name(args) value register macros that expand at their use sites, recursively (a macro may reference earlier macros). The output is a single-file token stream whose spans point at use sites, mirroring the reference adapter’s provenance convention (the HIR file registry keeps the main file). Invalid include graphs (cycles, missing files) and recursive defines fail deterministically with structured diagnostics that name the offending file/line.

JavaScript macros and hooks

A function-like define whose replacement starts with __script__("…") (OverPy 9.7.10 ABI, src/compiler/tokenizer.ts) is a script macro: the script path resolves root-relative at the define site (missing files are a script-not-found diagnostic, mirroring the reference’s ENOENT failure), and each expansion runs the script through opy_macro_js::MacroRuntime with the call-site arguments injected as var <name>=<raw>; declarations (the reference’s resolveMacro). The string completion value is lexed back into the token stream at the call site, with the reference’s per-line indentation rule applied to the text; the frontend token model makes indentation unobservable (the parser never consumes it), so the rule is preserved in the expansion text only. Runtime failures map to the structured script-* diagnostics with the script path, line, and column.

#!postCompileHook "hook.js" registers the post-compile hook script (duplicate declarations are rejected like the reference). The frontend recognizes, parses, validates, and records the directive only — it never executes the hook: real hook execution receives the final Workshop text produced by lowering and is lowering-dependent (workshop-rs emission, issue #8); the frontend never fabricates a Workshop payload.

Boundary: __script__ macros expand at compile time through the runtime (source-supported); #!postCompileHook is recorded and executed only against the real Workshop output (lowering-dependent). The runtime’s hook ABI is tested separately on synthetic content in opy-macro-js (see its hooks test suite).

Structs

DefineRecord

A recorded preprocessing define (HIR provenance).

FileRecord

The output file registry: the main file only (reference convention).

PostCompileHook

A registered #!postCompileHook script (the declaration record).

PreprocessOutcome

The outcome of preprocessing with overlays, retaining the file registry registered so far even when a directive or expansion fails, so callers can map an error’s span file id to its actual source.

Preprocessed

The result of preprocessing.

ScriptMacro

A resolved __script__("…") macro backing.

Functions

preprocess

Preprocess the main source text with its include root.

preprocess_with_overlay

Preprocess with open-document overlays: includes resolve to overlay text (keyed by the include string or the resolved canonical path) before the filesystem. Overlays model unsaved editor buffers without changing the compiler’s source-loading contract.

preprocess_with_overlay_outcome

Preprocess with open-document overlays while retaining the file registry registered so far on failure.