Skip to main content

scroll_page

Function scroll_page 

Source
pub fn scroll_page(
    db: &Database,
    collection: &str,
    cursor: Option<u64>,
    batch: usize,
) -> Result<(Vec<RawFact>, Option<u64>), MemoryError>
Expand description

One batch of collection, starting strictly after cursor, with the cursor to resume from.

This is the enumeration the rebuild should use, and it is NOT the VelesQL walk above. The engine already carries a cursor primitive — scroll_batch on the collection itself, keyed on the point id, exclusive, ascending — and it bypasses the query pipeline entirely. That matters twice over:

  • The query pipeline asks the collection for limit + offset rows and clamps the total to MAX_LIMIT (100_000), so an OFFSET walk goes blind past that mark. A cursor never accumulates an offset, so it has no such bound.
  • WHERE id > n really is unavailable — filters read the payload and the id is not in it — but that only ever ruled out expressing the cursor in VelesQL. It never ruled out the cursor.

Returns the batch and the cursor to pass next; None means the collection is exhausted.

§Errors

Returns crate::MemoryError if the collection is absent, is of a kind that does not scroll, or if the scroll itself fails.