pub fn scroll_page(
db: &Database,
collection: &str,
cursor: Option<u64>,
batch: usize,
) -> Result<(Vec<RawFact>, Option<u64>), MemoryError>Expand description
One batch of collection, starting strictly after cursor, with the cursor
to resume from.
This is the enumeration the rebuild should use, and it is NOT the VelesQL
walk above. The engine already carries a cursor primitive — scroll_batch
on the collection itself, keyed on the point id, exclusive, ascending — and
it bypasses the query pipeline entirely. That matters twice over:
- The query pipeline asks the collection for
limit + offsetrows and clamps the total toMAX_LIMIT(100_000), so anOFFSETwalk goes blind past that mark. A cursor never accumulates an offset, so it has no such bound. WHERE id > nreally is unavailable — filters read the payload and the id is not in it — but that only ever ruled out expressing the cursor inVelesQL. It never ruled out the cursor.
Returns the batch and the cursor to pass next; None means the collection
is exhausted.
§Errors
Returns crate::MemoryError if the collection is absent, is of a kind
that does not scroll, or if the scroll itself fails.