tm25ray 0.2.5

Reader, writer and far-field converter for IES TM-25 ray files (.TM25RAY): near-field LED and light-source ray sets — zero-copy, streaming, WASM-clean, no dependencies
Documentation
# Prompt: Build `tm25-rs`, a Rust reader for IES TM-25 ray files

> Status (2026-09-18): milestones 1 and 2 are implemented in `crates/tm25ray`
> (reader, streaming reader, `mmap` feature, writer, subsampling, far-field converter,
> LDT/ATLA conversion lives in eulumdat-rs's `eulumdat-tm25`). All fixture assertions below are tests;
> the real-file test runs when `TM25RAY_FIXTURE` (or the vendor package in
> `~/Downloads`) is present, synthetic files from the writer cover CI. Measured: the
> 20M-ray file (560 MB) decodes at ~91 M rays/s, ~19 M rays/s through the full
> far-field + reservoir pipeline (M2 Max; `examples/tm25_bench.rs`). Remaining `VERIFY` items are marked
> in the code and listed at the end of this document.
>
> Update (2026-10-23): offsets, block sizes and the official field names were
> checked against Table 4-2 of the standard. That settled the block after the
> spectral tables (32-byte padding, fixed 512-byte column names, text size in
> bytes), which 0.1.x had modelled as an optional trailer, and the `VERIFY`
> list at the end shrank to two items. Two more vendor packages (ams OSRAM
> GW CSSRMU blue and yellow, with larger spectral tables) and two Lumileds
> files round-trip byte for byte under the corrected layout.

## Role and goal

You are building a Rust crate that reads IES TM-25 ray files (`.TM25RAY`).
It joins an existing open-source Rust photometric ecosystem (gldf-rs, eulumdat-rs, l3d-rs).
It must follow the same cross-platform pattern: pure Rust core, usable from WASM, iOS/Android via UniFFI, Python via PyO3, and servers via Axum.
The first milestone is a correct, fast, well-tested reader.
The second milestone is a far-field converter that turns rays into an intensity distribution usable by eulumdat-rs.

## Background

TM-25 is the IES vendor-neutral format for near-field light source emission.
It stores individual rays (start point, direction, flux), not an angular intensity table.
The standard is IES TM-25-13 "Ray File Format for the Description of the Emission Properties of Light Sources", revised as ANSI/IES TM-25-20 (reaffirmed 2025). The 2020 revision was designed so that 2013 files stay readable.
The IES standard text is paywalled.

Who produces and consumes it (verified September 2026):

- Producers: ams OSRAM ray file portal (TM-25 packages up to 20M rays, plus vendor formats up to 5M), TechnoTeam RiGO801/Converter801 near-field goniophotometer software (TM-25 plus ASAP, SPEOS, LightTools, LucidShape, Zemax, TracePro, SimuLux, Photopia, incl. spectral multi-channel data), Radiant Vision Systems ProSource.
- Consumers with verified native import: Ansys Zemax OpticStudio since 24 R2 (extension `.tm25ray`, adds its own phase and E-field columns on export). LightTools, SPEOS and LucidShape are listed as targets by the producers but native import was not verified. TracePro and Photopia do not list TM-25. Renderers (V-Ray) and lighting-design tools (DIALux, Relux) do not read it.
- Open-source references: `JuliusMuschaweck/TM25RaySetTools` (C++, Unlicense; reader/writer, converters for Zemax, LightTools and ASAP, ray-set interpolation) and `LarryBoxler/IESTM25RayFiles` (C#, Unlicense; library and viewer with a sample file). No Rust implementation exists.

The layout below was reverse-engineered from four real ams OSRAM files (100k, 500k, 5M, 20M rays) and then cross-checked field by field against the field names of the C++ reference implementation, which carry the spec section numbers (`version_4_7_1_2`, `phi_v_4_7_1_4`, ...). Every field that both sources agree on is marked "confirmed". Treat the rest as the working spec until verified against the official document.

## Version handling

All known samples declare version `2013` (TM-25-13).
TM-25-20 exists; its on-disk changes are not documented publicly, but it is backward compatible with 2013 files.
Read the version field, support 2013 fully, and return a clear `UnsupportedVersion(u32)` error for anything else.
Structure the code so a second version layout can be added without touching the ray reader.

## Binary layout (TM-25-13, little-endian)

All integers are little-endian, all floats are `f32` LE. Spec section numbers follow the reference implementation.

| Offset | Size | Type | Observed value | Meaning |
|---|---|---|---|---|
| 0 | 4 | ASCII | `TM25` | File type magic, 4.7.1.1 (confirmed) |
| 4 | 4 | i32 | 2013 | Format version, 4.7.1.2 (confirmed) |
| 8 | 4 | i32 | -1 | Creation method, 4.7.1.3. Reference code uses 0 = simulation; -1 is presumably "unknown / not specified". Do not infer "measured" from it (enum values VERIFY) |
| 12 | 4 | f32 | 0.0 | Total luminous flux Φv in lm, 4.7.1.4. Zero here because the sample is a UV-C emitter (confirmed) |
| 16 | 4 | f32 | 0.051 | Total radiant flux Φ in W, 4.7.1.5. Rays sum to it (confirmed) |
| 20 | 8 | u64 | 100000 | Number of rays, 4.7.1.6. The former "unknown i32 at 24" is the high dword. Decodes 100k, 500k, 5M and 20M correctly (confirmed) |
| 28 | 28 | ASCII, NUL-padded | `2026-05-12 13:57:44` | File creation date and time, 4.7.1.7. Nominally ISO 8601; the vendor writes `YYYY-MM-DD hh:mm:ss` (confirmed, 28 bytes) |
| 56 | 4 | i32 | 0 | Ray start position flag, 4.7.1.8, observed 0 (position confirmed; meaning of other values VERIFY) |
| 60 | 4 | i32 | 3 | Spectral data identifier, 4.7.1.9, range 0..=4: 0 none, 1 single wavelength, 2 per-ray wavelength, 3 one spectral table shared by all rays, 4 per-ray index into several tables (confirmed) |
| 64 | 4 | f32 | NaN (`0x7FC00001`) | Single wavelength in nm, 4.7.1.10; NaN when unused (confirmed) |
| 68 | 4 | f32 | 220.0 | Minimum wavelength in nm, 4.7.1.11, set for identifiers 2..=4 (confirmed) |
| 72 | 4 | f32 | 320.0 | Maximum wavelength in nm, 4.7.1.12 (confirmed) |
| 76 | 4 | i32 | 1 | Number of spectral tables, 4.7.1.13 (confirmed) |
| 80 | 4 | i32 | 0 | Number of additional per-ray data items, 4.7.1.14 (confirmed) |
| 84 | 4 | i32 | 0 | Size of the additional text block 4.7.6, in bytes (confirmed) |
| 88 | 168 | zeros | | Reserved for future use, header block ends at 256 |
| 256 | 32 | 8 × i32 | 1, 1, 1, 0, 0, 0, 0, 0 | Known data flags block, 4.7.2: position, direction, radiant flux, wavelength, luminous flux, Stokes, tristimulus, spectrum index. Each 0 or 1 (confirmed) |
| 288 | 9 × 4000 | UTF-32LE, NUL-padded | see below | Nine description strings, 4.7.3.1 to 4.7.3.9, 1000 code points each (confirmed) |
| 36288 | 4 | i32 | 51 | First spectral table: point count N (confirmed) |
| 36292 | N × 8 | (f32 λ, f32 value) | 220 nm ... 320 nm, step 2 | Interleaved pairs, wavelength first, values in percent of peak (peak = 100). Identical to the `spectrum.txt` shipped in the package, which is the same table peak-normalised to 1.0 (confirmed by data) |
| 36292 + 8N | 0 – 28 | zeros | 4 bytes here | Padding: the spectral block (everything from 36288) is zero padded to a multiple of 32 bytes; 412 bytes of table need 4. Then `n_addtl_items` column names, 4.7.5, each a fixed 512-byte UTF-32LE string, NUL padded, with no count and no length prefix; then the additional text block, 4.7.6, of the size given at offset 84 (confirmed against the reference implementation and both vendors) |
| 36704 | n_rays × 28 | 7 × f32 | | Ray records (confirmed) |

The offsets above are from an ams OSRAM file. The fixed part up to 36288 is the
same everywhere; what follows depends on the spectral tables.

### Vendor differences (confirmed on real files)

Two producers, two shapes. Both are version 2013 and both store ray data
little-endian.

**The ray block starts at different offsets, and the header does say where.**
ams OSRAM files start their rays at 36704, Lumileds files at 36288. Versions
0.1.x of this crate explained the difference as an optional "column-name
count" trailer that one vendor wrote and the other omitted. The real rule is
simpler and deterministic: the spectral-table block is zero padded to a
multiple of 32 bytes. The ams OSRAM UV-C table is 412 bytes and gets 4 bytes
of padding, which looked exactly like a count of zero; the Lumileds files
have no table, so nothing to pad. Two more ams OSRAM packages with larger
tables (ray blocks at 37184 and 37664) confirm the rule. Column names, when
`n_addtl_items > 0`, follow as fixed 512-byte strings, and then the
additional text block. A file with additional columns (Zemax exports carry
phase and E-field) could not be read by 0.1.x at all, and the files 0.1.x
wrote are 4 to 20 bytes off for a reader that follows the standard.

**The "unknown value" sentinel may be byte-swapped.** Fields that are not known
(a radiometric file's luminous flux, the wavelength bounds of a file without a
spectrum) hold a NaN. ams OSRAM writes it little-endian like every other field,
`01 00 C0 7F`. Lumileds writes `7F 80 00 01`, the same idea with the bytes
reversed — which, read little-endian as the rest of the file demands, is not a
NaN at all but the denormal 2.36 × 10⁻³⁸. That is the dangerous case: a reader
that only checks `is_nan()` accepts it as a real measurement, and a radiant
flux of 2.36e-38 W silently poisons every derived quantity instead of failing
loudly. Match the exact bit pattern; do not classify by "would the swapped
bytes look like a NaN", which also matches roughly 0.4 % of ordinary positive
values.

**Record layouts differ accordingly.** The ams OSRAM UV-C files carry position,
direction and radiant flux (28 bytes). The Lumileds file carries position,
direction, luminous flux and tristimulus X and Z (36 bytes); Y is omitted
because for a photometric file it is the luminous flux already present.

Consistency rules the reference implementation enforces, worth mirroring in validation:

- At least one of the radiant-flux and luminous-flux flags must be set.
- Identifiers 2 and 4 require the radiant-flux flag; identifier 4 requires the spectrum-index flag; the tristimulus flag requires the luminous-flux flag and identifier 0; Stokes requires radiant flux.
- Flux fields are zero, positive, or NaN when unknown.
- Every spectral table wavelength is > 0 and every weight is ≥ 0.

Important: the ray block start depends on the spectral tables, their padding, the column names and the additional text block.
For the sample it is `36288 + pad32(4 + 8*51) + 512 * n_addtl_items + text_size = 36288 + 416 + 0 + 0 = 36704`.
Do not hardcode 36704.
Sanity check: `ray_start + n_rays * record_size == file_len` must hold (all four sample files pass with record size 28), otherwise return a descriptive error.
For identifier 0, 1 or 2 no spectral table is present and the block is absent; for identifier 4 there are `n_spectra` tables, each a point count followed by its pairs. No per-table index is stored; the reference implementation numbers them 1..n in memory (confirmed).

### Text fields (sample content)

| Index | Offset | Sample content | Meaning (4.7.3.x) |
|---|---|---|---|
| 0 | 288 | `SU CULCP1.VC` | Name of light source |
| 1 | 4288 | `ams-OSRAM AG` | Manufacturer of light source |
| 2 | 8288 | `BZ` | Creator of optical light source model (a person's initials here, not a type code) |
| 3 | 12288 | `ams-OSRAM AG` | Creator of the ray file |
| 4 | 16288 | `Measurement Equipment / Simulation Software Not Available` | Measurement equipment / simulation software |
| 5 | 20288 | `Camera Information Not Available` | Camera information |
| 6 | 24288 | `150mA` | Light source operation information |
| 7 | 28288 | `Additional Information Not Available` | Additional information |
| 8 | 32288 | `See Information In PDF file` | Data reference to light source geometry |

The names are the official ones (confirmed). Decode each field as UTF-32LE, stop at the first NUL, and reject invalid code points with an error instead of panicking.
Expose them as named fields plus a raw `[String; 9]`.

### Ray record

A record is `x y z kx ky kz` followed by the optional columns enabled by the known-data flags, in this order:

| Column | Present when | Unit |
|---|---|---|
| x, y, z | always | mm, start point near or on the source surface |
| kx, ky, kz | always | unit vector, \|k\| = 1 |
| radiant flux | radiant-flux flag | W |
| wavelength | wavelength flag (identifier 2) | nm |
| luminous flux (tristimulus Y) | luminous-flux flag | lm |
| S1, S2, S3 | Stokes flag | relative to the flux |
| polarisation ellipse x, y, z | Stokes flag | unit vector |
| tristimulus X, Z | tristimulus flag | |
| spectrum index | spectrum-index flag (identifier 4) | 1-based table index, an i32 (confirmed). The C++ reference reads and writes it as f32; the two bit patterns cannot collide, so the reader accepts both |
| additional items | `n_addtl_items` | named by the column-name block |

All columns are 4 bytes wide (`f32` except the spectrum index), row-major, so `record_size = 4 × column_count`.
The sample enables only radiant flux, hence 28 bytes.
Files written by Zemax OpticStudio may add phase and E-field columns as additional items, so a reader must compute the record size from the flags rather than assume 28.

## Test fixture and expected values

Fixture: `rayfile_SU_CULCP1_VC_100k_20260512_IES_TM25.TM25RAY`, 2,836,704 bytes.
It is an ams OSRAM UV-C LED, so the data is radiometric (51 mW), not photometric.
The same package ships 500k (14,036,704 bytes), 5M (140,036,704) and 20M (560,036,704) files with identical headers except ray count and timestamp, plus the spectrum as `.txt`, OPTIS `.spectrum`, Zemax `.spcd` (µm) and LightTools `.sre`, an orientation `info.pdf` and STEP/IGES/SLDPRT geometry.

Tests must assert:

- magic `TM25`, version 2013, n_rays 100000 read as u64
- creation method -1, luminous flux 0.0, radiant flux 0.051 (f32 tolerance)
- date string `2026-05-12 13:57:44`
- spectral identifier 3, single wavelength NaN, min 220, max 320, 1 spectral table, 0 additional items
- flags `[1,1,1,0,0,0,0,0]`, record size 28
- text field 0 == `SU CULCP1.VC`, field 1 == `ams-OSRAM AG`, field 6 == `150mA`
- spectrum has 51 points, first λ = 220, last λ = 320, peak value 100 at 266 nm, values ≥ 50 span 262 to 272 nm, and the table equals the shipped `spectrum.txt` × 100 (percent vs peak-normalised)
- ray block starts at offset 36704 and the size check passes for all four file sizes
- every ray flux == 5.1e-7, sum of ray fluxes ≈ 0.051
- every direction has \|k\| ≈ 1 (tolerance 1e-4)
- x in [-1.75, 1.61], y in [-1.61, 1.64], z in [-0.815, 0.0] mm
- kz in [0.1605, 1.0], so all rays go into the upper hemisphere, max polar angle ≈ 81°
- first ray: x ≈ 0.39076, y ≈ 0.12505, z = -0.755, k ≈ (-0.51988, 0.14747, 0.84141), flux 5.1e-7

Also add tests for truncated files, wrong magic, unknown version, a size mismatch, a flag combination that violates the consistency rules, and a synthetic file with `n_addtl_items = 1` to prove the column-name block shifts the ray start.

## Crate design requirements

1. Zero-copy core: parse the header from `&[u8]`, then expose rays as a view over the byte slice.
   Use `bytemuck` (or manual `f32::from_le_bytes`) so the same code works in WASM and native.
2. Streaming reader: a `Tm25Reader<R: Read>` that parses the header and yields rays in chunks, for multi-GB files.
3. Native fast path: optional `mmap` feature using `memmap2`, disabled for WASM.
4. WASM: must handle the 100k and 1M files fully in memory; for larger files, read via `Blob.slice` chunks from JS and feed the streaming reader.
   Keep in mind the 4 GB WASM32 address limit.
5. Subsampling: `subsample(n, seed)` returns n randomly chosen rays and rescales each flux so the sum still equals the header total.
6. Units: an enum `FluxKind { Radiometric, Photometric, Both }` derived from the known-data flags and the two header flux fields. No caller override is needed.
7. Spectrum access: expose the shared table (identifier 3) as `(wavelength, value)` pairs so eulumdat-rs can attach it as a relative SPD, per-ray wavelengths for identifier 2, and the table list plus per-ray index for identifier 4.
8. No panics on untrusted input.
   All offsets are bounds-checked and all errors use `thiserror`.
9. Writer (later milestone): write TM-25-13 files so subsampled or converted sets can be exported and round-trip tested byte-for-byte on the header.

## Far-field converter (milestone 2)

Bin ray directions into a C/γ grid compatible with EULUMDAT conventions.
Intensity per bin is the summed flux divided by the bin's solid angle: `ΔΩ = Δφ · (cos γ1 − cos γ2)`.
Output W/sr for radiometric files and cd for photometric files.
For radiometric sources such as this UV-C LED, cd/klm is meaningless; either store relative values or clearly mark the unit when handing data to eulumdat-rs.
Ignore start positions for the far field, since that is exactly the information the far-field approximation discards.

Expected shape for the fixture (10° polar bins, relative intensity):

| γ bin centre | 5 | 15 | 25 | 35 | 45 | 55 | 65 | 75 | ≥85 |
|---|---|---|---|---|---|---|---|---|---|
| I / Imax | 0.99 | 1.00 | 0.98 | 0.89 | 0.72 | 0.54 | 0.30 | 0.09 | 0.00 |

Half intensity is near 57°, so the beam angle is roughly 115°, slightly narrower than Lambertian.
Use this as a regression test with a loose tolerance, because it comes from 100k Monte Carlo rays.

Integration points in eulumdat-rs once the reader exists: the shared spectral table maps onto `atla::spd_loader::attach_spd`, and the ray set can back a `Source::RayFile` variant in `eulumdat-goniosim` (flux-weighted sampling with replacement, per-ray or table wavelength feeding the spectral Monte Carlo).

## Size expectations

| Rays | Approx. file size |
|---|---|
| 100k | 2.8 MB |
| 1M | 28 MB |
| 10M | 280 MB |
| 100M | 2.8 GB |

The header is a fixed ~36 KB, almost entirely zero padding from the UTF-32 text fields.
Float ray data compresses poorly, so do not rely on zip or GLDF containers to shrink it.

## Open questions to verify against the official IES TM-25 document

- Creation method enumeration (the reference code expects 0 or 1 and only warns otherwise; ams OSRAM writes -1).
- Meaning of a non-zero ray start position flag (4.7.1.8).

Settled in October 2026: the date field is 28 bytes with the start-position flag at 56; the text-block size sits at 84 and counts bytes; the spectral block is padded to 32 bytes and stores no per-table index; column names are fixed 512-byte strings; the nine text fields carry the names listed above; the spectrum index is an integer. TM-25-20 added the polarisation-ellipse axis columns (4.8.1.13 to 4.8.1.15), which this layout already carries; files otherwise stay 2013-compatible.

Mark every assumption in code with a `// VERIFY:` comment so they are easy to find once the spec is available.

## Sources

- IES TM-25-13 preview, ANSI webstore: https://webstore.ansi.org/preview-pages/IESNA/preview_IES+TM-25-13.pdf
- ANSI/IES TM-25-20 (R2025): https://webstore.ansi.org/standards/iesna/ansiiestm2520r2025
- Reference implementation (field names carry spec section numbers): https://github.com/JuliusMuschaweck/TM25RaySetTools
- C# library with sample file: https://github.com/LarryBoxler/IESTM25RayFiles
- ams OSRAM AN086, Importing rayfiles and ray-measurement files of LEDs: https://look.ams-osram.com/m/f66d917c5267092/original/Importing-rayfiles-and-ray-measurement-files-of-LEDs.pdf
- Zemax OpticStudio 2024 R2 release notes: https://community.zemax.com/opticstudio-release-notes-71/ansys-zemax-opticstudio-2024-r2-release-notes-5222
- TechnoTeam light source characterization: https://www.technoteamvision.com/main/applications/light_sources__luminaires/light_source_characterization/index_eng.html