Module window 

A file view that slides inside a fixed reservation of address space.

What this is for

A decoder in the sandbox asks for byte ranges. The host has to put those bytes at an address the decoder can read, and the obvious way to do that is to map the whole file, which is what the prior art does. Mapping the whole file is fast and it costs address space proportional to the dataset, which is fine until the dataset is larger than the address space or until there are a thousand of them open at once.

A window is the other trade. It reserves a fixed span of addresses once, maps a piece of the file into it, and moves that piece when a request falls outside it. The address space cost is constant and chosen by the host rather than by the data. The cost is that a request which does not fall in the current view pays for a remap, so the win depends entirely on requests being clustered, which for a columnar scan they are.

The property that matters

When the view moves, everything the old view pointed at has to stop being readable. Not stop being correct, stop being readable. A decoder that reads through a stale pointer and gets bytes from the part of the file that used to be there produces an answer that is wrong and looks right, and there is nothing downstream that can catch it. So the vacated range does not become free memory and does not become zeroes: it goes back to being reserved and unreadable, and a read through a stale pointer faults.

Rust’s own rules cover the safe path already, because Window::range borrows the window mutably and hands back a slice tied to that borrow, so a slice cannot outlive the view it came from. That is not the case the gate is about. The case the gate is about is a raw address that crossed into a sandbox, where the borrow checker is not present, and the test for it is in tests/window.rs.

Alignment

Offsets and lengths are rounded by two different numbers, which is a Windows distinction that Unix does not have and is the easiest thing here to get wrong on a machine where they happen to be equal. A mapping offset has to be a multiple of the allocation granularity, sixty four kibibytes on Windows and the page size everywhere else. A mapping length has to be a multiple of the page size. Rounding a length up by the allocation granularity instead looks correct on Unix and fails on Windows at the end of a file, because the pages past the end of the section are not part of it.

A view is therefore rounded out at both ends and is usually larger than what was asked for. The slice handed back is not: it is exactly the requested range, cut out of the middle.

Structs

Window

A read only view of part of a file, at a fixed address, that can be moved.

Enums

WindowError

What can go wrong when opening a window or moving it.

Constants

DEFAULT_SPAN

How much address space a window reserves when the caller does not say.