Skip to main content

Crate xold

Crate xold 

Source
Expand description

xold: a maintainable, multi-threaded linker for ELF, Mach-O and COFF.

The crate is organised around three observations from studying lld and mold:

  • Each binary format lives behind its own reader/writer module (elf, macho, coff) and produces a common view of sections, symbols and relocations consumed by the format-neutral core.
  • Per-architecture relocation handling is driven by a declarative table keyed on a shared [RelExpr] semantic enum, so the scan and apply phases are written once instead of copy-pasted per architecture.
  • All on-disk structures are read zero-copy from a memory-mapped file via bytemuck, with no unsafe in the parsing layer.

§Deterministic output (hard requirement)

The linked image is a pure function of the inputs. The same object files and the same command line produce the same bytes, on every run, on any machine, at any thread count. Treat this as a correctness property with the same standing as producing a runnable binary at all, because a build system that caches or compares link outputs depends on it.

Most of the linker runs in parallel, so the rule has teeth:

  • Nothing scheduling-dependent may reach the output. A concurrent or sharded structure may decide identity – which name is which symbol, whether two strings are equal – but never order. Every ordering that reaches the image (symbol table order, member placement, GOT and PLT slot assignment, merged string layout) derives from input order (file index, then index within that file) or from content.
  • Iteration order of a hash container is not an ordering. It is stable for one insertion sequence and nothing more. Walk a dense index or an explicitly sorted list.
  • Verify it rather than assuming it. Link the same inputs at RAYON_NUM_THREADS of 1, 2, 4, 8 and 16 and compare hashes; a change exercised only at the default thread count has not been tested.

Two places show what this costs and why it is worth it. symbol::SymbolTable assigns ids in first-seen input order, which is inherently serial, so the parallel fold in symbol::bulk reproduces that order rather than abandoning it. merge deduplicates through a hash map but emits pieces in first-seen order, never map order.

The practical consequence for changes: a refactor or an optimisation must produce byte-identical output to the commit before it. That is a far stricter regression test than the suite, and it catches ordering mistakes the suite cannot. A change that is meant to alter the output should say so in its commit message and explain why the new bytes are correct.

§Running under Miri

cargo +nightly miri test interprets the readers, the relocation arithmetic and the plan builders, which is where a zero-copy parser would hide undefined behaviour. .cargo/config.toml carries the two flags the run needs and says why each is there.

Miri neither spawns processes nor maps files, so the end-to-end tests – which compile with the host toolchain and link through a mapping – are marked ignore under it and the harness reports them as such. What is left is every test that builds its input in memory.

Those in-memory fixtures must be aligned. A Vec<u8> or a [u8; N] asks for alignment 1 and gets more only because a real allocator rounds up; Miri grants exactly what was asked, so a fixture handed straight to a reader is refused there and accepted everywhere else. input::AlignedBytes is the storage that makes the request explicit, and is what the tests use.

Re-exports§

pub use error::Error;
pub use error::Result;

Modules§

archive
Minimal ar archive reader, enough for lazy member extraction.
buildid
--build-id: the NT_GNU_BUILD_ID note that names one build of an image.
coff
COFF/PE binary format support: constants, records, a reader, and a writer.
comdat
ELF section-group (COMDAT) deduplication.
debug
Aggregates non-allocated DWARF .debug_* sections for the writer.
defsym
Linker-defined symbols: the names a C runtime refers to that no input defines.
detach
Returning to the caller before teardown: the fork the CLI runs under.
dynamic
Dynamic linking emission: the synthetic tables a shared object or a dynamic executable needs so the runtime loader can relocate the image and resolve its imports and exports.
ehframe
.eh_frame parsing and .eh_frame_hdr generation.
elf
ELF binary format support: on-disk structures, format constants, and a zero-copy reader for relocatable object files.
endian
Little-endian integers for on-disk structures.
error
Error handling for the linker.
gc
Dead-section garbage collection for --gc-sections.
icf
Identical code folding (--icf=all / --icf=safe).
input
Input files and the per-symbol view consumed by resolution.
layout
Output image layout: the bridge between symbol resolution and the writer.
linker
The link driver: owns the inputs and runs the resolution and layout passes.
lto
Link-time optimisation through the GNU linker-plugin interface.
macho
Mach-O binary format support: on-disk constants, plain view records, a zero-copy reader for 64-bit relocatable objects, and a writer + static link driver that produces a 64-bit MH_EXECUTE image.
merge
Mergeable section deduplication (SHF_MERGE, SHF_STRINGS).
mmap_file
Memory-mapped files: read-only input views and the writable output image.
output
Output sections: how allocated input sections are grouped for the image.
plt
The procedure linkage table: the trampolines an image calls a function through when the address to jump to is not fixed at link time.
pool
How many threads one link runs on.
reloc
Architecture-neutral relocation handling, driven by a declarative table.
script
GNU ld linker scripts, to the extent an input list needs one.
search
Library search: resolves -l NAME to a concrete file by scanning the directories named with -L followed by the host’s default search paths.
startlib
The --start-lib/--end-lib group, packed into a real archive.
startstop
__start_NAME / __stop_NAME: the bounds a linker supplies for a section whose name is a valid C identifier.
symbol
The resolved symbol model and the global symbol table.
tls
Static TLS (local-exec) layout: the thread-pointer-relative offset arithmetic shared by every architecture.
util
Small helpers shared by the format backends and the layout passes.
versionscript
--version-script: which of an image’s global symbols stay visible to whoever loads it.
writer
The ELF image writer: serialises a crate::layout::Layout into an ET_EXEC byte image.