yah-scryer 0.8.26

Per-machine event store: ring buffer + short-disk SQLite + cross-service query surface
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101

//! @arch:layer(kg_store)
//! @arch:role(substrate)
//! @arch:see(.yah/docs/architecture/A049-yah-scryer.md)
//!
//! `scryer` — per-machine event store with tiered retention.
//!
//! Tier model (this crate — F1 / scryer-1):
//!   - Recent ring: in-memory ring of last 10k events / 64 MB.
//!   - Short disk: per-machine SQLite at `/var/lib/yah/scryer/events.db`.
//!   - (Long tier / Parquet is R093-F5, off by default, not in this crate.)
//!
//! Query surface (F1, TaskRun scope only):
//!   - `Scryer::events(scope, filter)` — query events; delegates to task-runs store for TaskRun scope.
//!   - `Scryer::tail(scope, cursor)` — cursor-based live follow.
//!   - `Scryer::subscribe(scope)` — push stream for tile viewers and gnomes.
//!
//! F2 adds:
//!   - `EventScope::Service(MeshIdent)` ingestion via [`adapters::ContainerdLogsAdapter`]
//!     (containerd gRPC log API) and [`adapters::WardenRpcAdapter`] (yubaba's
//!     own structured emissions).
//!   - [`beholders::ServiceBeholder`] trait + bundled `vanilla` (rfc5424-ish)
//!     and `unstructured` (passthrough) parsers.
//!
//! F3 adds:
//!   - [`beholders::PinoBeholder`] — Node.js pino NDJSON structured parser.
//!   - [`beholders::TracingJsonBeholder`] — Rust tracing-subscriber JSON parser.
//!   - Image-label opt-in (`yah.beholder=<name>`) forcing beholder selection.
//!   - `ServiceBeholder::unknown_format_reason()` for schema versioning + decline.
//!   - [`quota::ServiceQuotaManager`] — per-`MeshIdent` rate limiting (1000 ev/s
//!     default) with Synth drop-count events on window rollover.
//!
//! F7 adds (P2 narrow):
//!   - [`adapters::JournaldAdapter`] — host-level systemd units (sshd, kernel,
//!     yubaba's own unit). Entries scoped to `Service(MeshIdent("<unit>.host"))`.
//!     Explicit allow-list; yah-managed services stay on `containerd_logs`.
//!
//! F8 adds (P2 tier-2):
//!   - [`ingestion::IngestionServer`] — Unix socket ingestion server for
//!     `yah-log` service-scope events. Yubaba injects `YAH_SERVICE_IDENT` +
//!     `YAH_SCRYER_SOCKET` into workload env; the shim connects and writes
//!     JSON-lines with `scope_kind = "service"` scope envelope.
//!
//! F5 adds (P3 opt-in):
//!   - [`long_tier::LongTierStore`] — per-day Parquet shard rollover from
//!     short-disk to R2/MinIO, keyed by `(machine_id, day)`.
//!   - [`long_tier::ObjectStore`] trait — production wires an S3-compat impl;
//!     tests use [`long_tier::InMemoryObjectStore`].
//!   - [`service::Scryer::aggregate`] — cross-boundary rollup: routes events
//!     older than `short_disk_retention_ms` to the Parquet tier.

pub mod adapters;
pub mod beholders;
pub mod federation;
pub mod federation_http;
#[cfg(unix)]
pub mod ingestion;
pub mod long_tier;
pub mod promotion;
pub mod quota;
pub mod ring;
pub mod service;
pub mod snapshot;
pub mod store;

#[cfg(unix)]
pub use ingestion::{IngestionError, IngestionServer};
pub use long_tier::{
    InMemoryObjectStore, LongTierConfig, LongTierError, LongTierStore, MS_PER_DAY, ObjectStore,
};
pub use adapters::{
    Adapter, AdapterError, BackoffConfig, ContainerLogSource, ContainerdLogsAdapter,
    JournaldAdapter, JournaldEntry, JournaldSource, Supervisor, WardenEvent, WardenRpcAdapter,
};
pub use beholders::{
    BeholderCtx, LogLine, PinoBeholder, PinoFactory, ServiceBeholder, ServiceBeholderFactory,
    ServiceBeholderRegistry, ServiceHints, TracingJsonBeholder, TracingJsonFactory,
    UnstructuredBeholder, VanillaBeholder,
};
pub use federation::{
    DenyAllAcl, FederationAcl, FederationError, FederationPeer, FederationRule, OperatorTagAcl,
    PeerIdentity, ScopedEvent, TimeOrdered, federated_events, merge_events, merge_ordered,
};
pub use federation_http::{
    BucketDto, FederateAggregateReq, FederateAggregateResp, FederateEventsReq, FederateEventsResp,
    FederationState, HealthResp, HttpFederationPeer, OPERATOR_TAG_HEADER, ScopeInfoDto, ScopesResp,
    router as federation_router, serve as serve_federation,
};
pub use promotion::{
    DEFAULT_PROMOTE_INTERVAL, PromotionConfig, PromotionConsumer,
};
pub use snapshot::{
    AnalyticsSnapshot, DEFAULT_EVENT_LIMIT, DEFAULT_SNAPSHOT_INTERVAL, POINTER_KEY,
    SNAPSHOT_PREFIX, SnapshotBucket, SnapshotConfig, SnapshotError, SnapshotEvent,
    SnapshotLevelCount, SnapshotPointer, SnapshotProducer,
};
pub use quota::{QuotaDecision, ServiceQuotaManager, DEFAULT_QUOTA_PER_SECOND};
pub use ring::{EventRing, RingConfig};
pub use service::{
    AggregateBucket, EventFilter, QueryCursor, Scryer, ScryerConfig, ScryerError, TailResult,
};
pub use store::{EventStore, ScopeFilter, ScopeInfo, ScryerStoreError};