Expand description
xold: a maintainable, multi-threaded linker for ELF, Mach-O and COFF.
The crate is organised around three observations from studying lld and mold:
- Each binary format lives behind its own reader/writer module (
elf,macho,coff) and produces a common view of sections, symbols and relocations consumed by the format-neutral core. - Per-architecture relocation handling is driven by a declarative table
keyed on a shared [
RelExpr] semantic enum, so the scan and apply phases are written once instead of copy-pasted per architecture. - All on-disk structures are read zero-copy from a memory-mapped file via
bytemuck, with nounsafein the parsing layer.
§Deterministic output (hard requirement)
The linked image is a pure function of the inputs. The same object files and the same command line produce the same bytes, on every run, on any machine, at any thread count. Treat this as a correctness property with the same standing as producing a runnable binary at all, because a build system that caches or compares link outputs depends on it.
Most of the linker runs in parallel, so the rule has teeth:
- Nothing scheduling-dependent may reach the output. A concurrent or sharded structure may decide identity – which name is which symbol, whether two strings are equal – but never order. Every ordering that reaches the image (symbol table order, member placement, GOT and PLT slot assignment, merged string layout) derives from input order (file index, then index within that file) or from content.
- Iteration order of a hash container is not an ordering. It is stable for one insertion sequence and nothing more. Walk a dense index or an explicitly sorted list.
- Verify it rather than assuming it. Link the same inputs at
RAYON_NUM_THREADSof 1, 2, 4, 8 and 16 and compare hashes; a change exercised only at the default thread count has not been tested.
Two places show what this costs and why it is worth it.
symbol::SymbolTable assigns ids in first-seen input order, which is
inherently serial, so the parallel fold in symbol::bulk reproduces that
order rather than abandoning it. merge deduplicates through a hash map
but emits pieces in first-seen order, never map order.
The practical consequence for changes: a refactor or an optimisation must produce byte-identical output to the commit before it. That is a far stricter regression test than the suite, and it catches ordering mistakes the suite cannot. A change that is meant to alter the output should say so in its commit message and explain why the new bytes are correct.
§Running under Miri
cargo +nightly miri test interprets the readers, the relocation
arithmetic and the plan builders, which is where a zero-copy parser would
hide undefined behaviour. .cargo/config.toml carries the two flags the
run needs and says why each is there.
Miri neither spawns processes nor maps files, so the end-to-end tests –
which compile with the host toolchain and link through a mapping – are
marked ignore under it and the harness reports them as such. What is left
is every test that builds its input in memory.
Those in-memory fixtures must be aligned. A Vec<u8> or a [u8; N] asks
for alignment 1 and gets more only because a real allocator rounds up;
Miri grants exactly what was asked, so a fixture handed straight to a
reader is refused there and accepted everywhere else.
input::AlignedBytes is the storage that makes the request explicit, and
is what the tests use.
Re-exports§
Modules§
- archive
- Minimal
ararchive reader, enough for lazy member extraction. - buildid
--build-id: theNT_GNU_BUILD_IDnote that names one build of an image.- coff
- COFF/PE binary format support: constants, records, a reader, and a writer.
- comdat
- ELF section-group (COMDAT) deduplication.
- debug
- Aggregates non-allocated DWARF
.debug_*sections for the writer. - defsym
- Linker-defined symbols: the names a C runtime refers to that no input defines.
- detach
- Returning to the caller before teardown: the fork the CLI runs under.
- dynamic
- Dynamic linking emission: the synthetic tables a shared object or a dynamic executable needs so the runtime loader can relocate the image and resolve its imports and exports.
- ehframe
.eh_frameparsing and.eh_frame_hdrgeneration.- elf
- ELF binary format support: on-disk structures, format constants, and a zero-copy reader for relocatable object files.
- endian
- Little-endian integers for on-disk structures.
- error
- Error handling for the linker.
- gc
- Dead-section garbage collection for
--gc-sections. - icf
- Identical code folding (
--icf=all/--icf=safe). - input
- Input files and the per-symbol view consumed by resolution.
- layout
- Output image layout: the bridge between symbol resolution and the writer.
- linker
- The link driver: owns the inputs and runs the resolution and layout passes.
- lto
- Link-time optimisation through the GNU linker-plugin interface.
- macho
- Mach-O binary format support: on-disk constants, plain view records, a
zero-copy reader for 64-bit relocatable objects, and a writer + static link
driver that produces a 64-bit
MH_EXECUTEimage. - merge
- Mergeable section deduplication (
SHF_MERGE,SHF_STRINGS). - mmap_
file - Memory-mapped files: read-only input views and the writable output image.
- output
- Output sections: how allocated input sections are grouped for the image.
- plt
- The procedure linkage table: the trampolines an image calls a function through when the address to jump to is not fixed at link time.
- pool
- How many threads one link runs on.
- reloc
- Architecture-neutral relocation handling, driven by a declarative table.
- script
- GNU ld linker scripts, to the extent an input list needs one.
- search
- Library search: resolves
-l NAMEto a concrete file by scanning the directories named with-Lfollowed by the host’s default search paths. - startlib
- The
--start-lib/--end-libgroup, packed into a real archive. - startstop
__start_NAME/__stop_NAME: the bounds a linker supplies for a section whose name is a valid C identifier.- symbol
- The resolved symbol model and the global symbol table.
- tls
- Static TLS (local-exec) layout: the thread-pointer-relative offset arithmetic shared by every architecture.
- util
- Small helpers shared by the format backends and the layout passes.
- versionscript
--version-script: which of an image’s global symbols stay visible to whoever loads it.- writer
- The ELF image writer: serialises a
crate::layout::Layoutinto anET_EXECbyte image.