Crate idiolect_indexer 

Firehose consumer for dev.idiolect.* records.

idiolect-indexer sits between a tapped-backed firehose transport and an appview’s per-record handlers. The crate is organized around three narrow traits that mirror the adapter pattern already used in idiolect-lens:

• EventStream: source of raw firehose commits.
• RecordHandler: appview-side consumer of decoded records.
• CursorStore: durable seat for the firehose cursor so the indexer resumes cleanly after a restart.

The orchestrator drive_indexer pulls from the stream, filters to the dev.idiolect.* nsid prefix, decodes bodies into idiolect_records::AnyRecord, dispatches to the handler, and commits the cursor on live events.

The default feature set is transport-agnostic: every trait ships an InMemory* impl that appview tests can wire up directly. Enable firehose-tapped to plug the tapped-based adapter in.

Quickstart (in-memory)

use idiolect_indexer::{
    InMemoryCursorStore, InMemoryEventStream, IndexerAction, IndexerConfig,
    NoopRecordHandler, RawEvent, drive_idiolect_indexer,
};

let mut stream = InMemoryEventStream::new();
stream.push(RawEvent {
    seq: 1,
    live: true,
    did: "did:plc:alice".to_owned(),
    rev: "3l5".to_owned(),
    collection: idiolect_records::Nsid::parse("dev.idiolect.encounter").unwrap(),
    rkey: "3l5".to_owned(),
    action: IndexerAction::Delete,
    cid: None,
    body: None,
});

let handler = NoopRecordHandler::new();
let cursors = InMemoryCursorStore::new();
let cfg = IndexerConfig::default();

drive_idiolect_indexer(&mut stream, &handler, &cursors, &cfg)
    .await
    .unwrap();

assert_eq!(handler.observed(), 1);
assert_eq!(handler.deletes(), 1);

Re-exports

pub use cursor::CursorStore;

pub use cursor::InMemoryCursorStore;

pub use error::IndexerError;

pub use event::IndexerAction;

pub use event::IndexerEvent;

pub use handler::NoopRecordHandler;

pub use handler::RecordHandler;

pub use indexer::IndexerConfig;

pub use indexer::drive_idiolect_indexer;

pub use indexer::drive_indexer;

pub use stream::EventStream;

pub use stream::InMemoryEventStream;

pub use stream::RawEvent;

Modules

cursor

The CursorStore boundary — where the firehose sequence number lives across indexer restarts.

error

Error types for the indexer crate.

event

The shape of a firehose event once it has been decoded into a record from one (or several composed) record families.

handler

The RecordHandler boundary — what an appview does with a decoded record.

indexer

The orchestrator that glues EventStream, RecordHandler, and CursorStore into a loop.

stream

The EventStream boundary — where firehose events come from.