polyc-query 2026.9.0

//! `personas`/`participations`/`persona_wallets`/`persona_spend_policies`/
//! `persona_credentials`/`persona_usage` reference-table decoders — persona
//! directory records → Arrow (#1312, part of the #1178 epic; the four
//! wallet/spend/credential/usage tables are tracked under #1578).
//!
//! Persona and participation are GENERAL domain reference data, not journal
//! events: they live in `PersonaHost`/`PersonaStore` (`crates/persona`), a
//! commonware qmdb key-value store that is a SEPARATE substrate from the
//! event journal `crate::decode`'s other modules decode. This module is the
//! query engine's FIRST non-journal data source
//! (a "mint a session, then
//! filter server-side" design names `personas`/`participations` as
//! the eventual per-caller filter source, sourced from
//! `PersonaStore::participations`/`overview` — this module backs the same
//! two tables today, Fleet-only, with no filter yet).
//!
//! Row types wrap the store's own proto records
//! (`polyc_proto::proto::polychrome::persona::v1::{PersonaProfile,
//! Participation}`) rather than re-declaring a parallel struct: `polyc-proto`
//! is already this crate's dependency (see `crates/query/Cargo.toml`) and,
//! per `scripts/check_rings.py`'s `R1`/`R2` rules, a plain (non-foundation)
//! Component may depend on any other Component — `polyc-query` depending on
//! the persona proto types carries no layer-rule conflict. `crates/persona`
//! itself (the `PersonaHost`/`PersonaStore` runtime) is deliberately NOT a
//! dependency of this crate: decode here only ever touches the already-typed
//! [`PersonaProfile`]/[`Participation`] values a caller supplies (see
//! `crate::engine::ReferenceData`), never the store/host machinery that
//! reads them off disk — that stays the caller's job, mirroring
//! `crate::engine::PartitionEvents`'s "this crate never opens a journal
//! handle itself" rule for the event-log side.
//!
//! # File layout (#1623 review split)
//!
//! One file per table family, colocated with its own tests — a pure move
//! out of a single 1798-line `persona.rs`, no behavior or signature change:
//! [`core`] (`personas`/`participations`/`persona_identities`, #1312),
//! [`wallets`] (`persona_wallets`), [`spend_policies`]
//! (`persona_spend_policies`), [`credentials`] (`persona_credentials`),
//! [`usage`] (`persona_usage`) — the last four all #1578. This
//! module re-exports every submodule's public surface unchanged, so every
//! existing `crate::decode::persona::X` path (registration, the engine
//! module docs, test call sites) still resolves without an edit.
//!
//! # Column selection
//!
//! All three #1312 tables are flat and keyed on `persona_id` — the join key
//! back to `participations`/`persona_identities`, and (via
//! `participations.conversation_id`) to the event tables' `partition`
//! column. Deliberately reusable, general-purpose columns only, not every
//! field [`PersonaProfile`]/[`Participation`] carry:
//!
//! - **`personas`**: `persona_id`, `display_name` (presentation only, per
//!   `persona.proto`'s own field doc), `status` (`"provisional"`/`"linked"`/
//!   `"merged"`/`"removed"`), `created_at_ms`, `merged_into` (the surviving
//!   persona id when `status == "merged"`; empty string otherwise — a plain
//!   proto3 string, whose zero value is `""`, never absent, same as
//!   `display_name`). `merged_into` is the fold this table needs to let a
//!   Fleet consumer canonicalize a merged persona's data into its survivor
//!   without a second lookup, mirroring `forensics.rs`'s `resolve_persona_id`
//!   (#1312's G1 gap). `identities` (linked external identities) is a
//!   `repeated` field — not flat — so it is its own child table,
//!   `persona_identities`, below (G2). Merge/split/removal audit fields
//!   beyond `merged_into` itself, `profile` (the preferences/authorization
//!   document), and admin/operator flags were left out of this table's FIRST
//!   cut — #1578 widens `personas` to carry them after all: the
//!   lifecycle/audit columns `merged_by`/`merge_evidence`/`merged_at_ms`/
//!   `split_from`/`split_by`/`split_reason`/`split_at_ms`/`removed_by`/
//!   `removed_at_ms`/`admin_removed_by`/`admin_removed_at_ms` (verbatim off
//!   [`PersonaProfile`] — empty-string/zero proto3 sentinels for "never
//!   happened", mirroring `merged_into`'s own convention, NOT
//!   `forensics.rs`'s `PersonaLifecycleJson` nested-`Option` shape), plus the
//!   flattened `profile` document: `profile_language`, `profile_tone`,
//!   `profile_updated_at_ms`, `profile_preferences` (the `map<string,
//!   string>` field, JSON-object text — see "JSON-shaped string columns"
//!   below), and its nested `Permissions`: `permissions_tool_grants`
//!   (JSON-array text), `permissions_approval_policy`,
//!   `permissions_turn_quota`, `permissions_admin`, `permissions_auto_review`,
//!   `permissions_operator`. An absent `profile`/`permissions` (the
//!   `MessageField` is unset — `buffa`'s true proto3 message-presence, not
//!   flattened away) decodes to every one of these columns at its type's
//!   zero value, the same "absent ≡ all-default" reading
//!   `forensics::api_persona`'s `profile.profile.as_option()` already relies
//!   on for the floor a provisional persona gets regardless. This closes the
//!   ONLY reason these columns were deferred in the first cut ("no query
//!   justifies them yet") — #1578's forensics-parity bar is that query.
//! - **`participations`**: `persona_id` (supplied by the caller alongside
//!   each [`Participation`] — the proto record itself is stored keyed by
//!   `(persona_id, conversation_id)` and does not repeat `persona_id` in the
//!   value, see `persona.proto`'s own field doc), `conversation_id`, `role`
//!   (`"initiator"`/`"participant"`), `first_at_ms` (this table's "joined
//!   at"). `via_persona_id` (merge provenance: which persona originally
//!   accrued a tie before a merge copied it to the survivor) is left out —
//!   audit provenance, not the general join-shape this table's first cut
//!   needs.
//! - **`persona_identities`**: one row per (`persona_id`, identity) pair off
//!   [`PersonaProfile::identities`] — `persona_id` (supplied by the caller
//!   the same way `participations` pairs one in, since [`ExternalIdentity`]
//!   itself carries no `persona_id`), then every [`ExternalIdentity`] field
//!   verbatim: `provider`, `scope`, `external_id`, `display_name`. This is
//!   the child table the fact model's "child-table" class
//!   names for a `repeated` field —
//!   `caller`/`participant`'s nested `ExternalIdentity` is the event-log
//!   precedent this table mirrors on the reference-data side (#1312's G2
//!   gap). It backs the reverse `provider`/`scope`/`external_id` →
//!   `persona_id` lookup a caller builds as `SELECT persona_id FROM
//!   persona_identities WHERE provider = ? AND scope = ? AND external_id =
//!   ?` — this module only makes the table queryable, it does not build that
//!   query itself.
//! - **`persona_wallets`**: one row per persona with a linked [`WalletLink`]
//!   (no row when unlinked) — `persona_id`, `wallet_address`, `currency`,
//!   `expiry_unix`, `created_at_ms`, `revoked`, `revoked_at_ms`. Every
//!   [`WalletLink`] field except `key_ref` — the delegated key's opaque,
//!   non-secret reference, held back for the same reason
//!   `forensics::PersonaWalletJson` holds it back (UI/query material, not
//!   key material itself, but no read surface needs it and every surface
//!   that DOES resolve `key_ref` to a live signing capability is admin-path
//!   only).
//! - **`persona_spend_policies`**: one row per persona with a set
//!   [`SpendPolicy`] (no row when never set) — `persona_id`, `limit`,
//!   `period_secs`, `max_lifetime_secs`, `allowed_hosts`, `updated_at_ms`,
//!   `set_by`. Every [`SpendPolicy`] field verbatim — this record has no
//!   key-material field to begin with. `limit` collides with the SQL
//!   `LIMIT` clause keyword; reference it quoted (`"limit"`) in a query that
//!   also uses a row-count limit clause.
//! - **`persona_credentials`**: one row per persona with an enrolled
//!   [`PersonaCredential`] (no row when never enrolled) — `persona_id`,
//!   `rp_id`, `origin`, `created_at_ms`, `revoked`, `revoked_at_ms`. Every
//!   [`PersonaCredential`] field EXCEPT `credential_id`,
//!   `p256_public_key_sec1`, `signing_public_key` — the three key-material
//!   fields `forensics::PersonaPasskeyJson` also holds back; see the
//!   "Bulk-export invariant" section below for why this crate pins their
//!   absence as a schema-level test, not just a doc comment.
//! - **`persona_usage`**: one row per persona with a maintained
//!   [`UsageRollup`] (no row when the persona has never accrued a committed
//!   turn as caller) — `persona_id`, `committed_turns`, `input_tokens`,
//!   `output_tokens`, `last_active_ms`, `conversation_ids` (JSON-array text
//!   of the tied conversation ids), `updated_at_ms`. Every [`UsageRollup`]
//!   field verbatim — this is the STORE'S maintained aggregate, keyed by
//!   persona id; see "`usage` vs. `persona_usage`" below for why this is a
//!   different table from the event-log-derived `usage` table
//!   `crate::decode::usage` backs.
//!
//! # JSON-shaped string columns
//!
//! `allowed_hosts` (`persona_spend_policies`), `conversation_ids`
//! (`persona_usage`), `profile_preferences`, and `permissions_tool_grants`
//! (both `personas`) all back a `repeated`/`map` proto field with `Utf8`
//! JSON text rather than a native Arrow list/struct type — the same
//! convention [`crate::decode::grant_replays`]'s `covered_capabilities` and
//! [`crate::decode::handoffs`]'s `allowed` already use, chosen there (and
//! here) specifically so this table stays a flat `MemTable` schema like
//! every other typed table in this crate, at the cost of needing
//! `datafusion_functions_json`'s registered JSON SQL functions (or a
//! `LIKE`/substring match) to filter into the encoded list/map from SQL.
//!
//! # `usage` vs. `persona_usage`: a deliberate naming collision
//!
//! [`crate::decode::usage`]'s `usage` table is PER-TURN, decoded from the
//! event journal's `UsageEvent`s, committed-turn filtered
//! (`crate::views`'s "Committed-turn filter invariant"). `persona_usage`
//! (this module's own [`usage`] submodule) is PER-PERSONA, decoded from
//! `PersonaHost`'s maintained [`UsageRollup`] aggregate — a SEPARATE
//! substrate, the same journal-vs-store split every other table in this
//! module documents. The two names share the word "usage" because they
//! answer genuinely related questions ("how much did this turn cost" vs.
//! "how much has this persona cost in total"), and neither name is renamed
//! to avoid the collision: a rename here would break `persona_usage`'s
//! query-engine consumers for a purely cosmetic reason, and `usage` already
//! predates this table by several phases. The collision is documented here,
//! deliberately not fixed.
//!
//! # Bulk-export invariant (QRY-8)
//!
//! Every wallet address, spend cap, host allowlist, and passkey-enrollment
//! date `persona_wallets`/`persona_spend_policies`/`persona_credentials`
//! expose is UNREDACTED at
//! [`Fleet`](crate::session::QueryScope::Fleet) scope —
//! deliberately: this is the SAME data `forensics::api_persona` already
//! serves unredacted per-persona over HTTP
//! (`PersonaWalletJson`/`PersonaSpendPolicyJson`/`PersonaPasskeyJson`,
//! `crates/control-plane/src/forensics.rs:439-472`), so a Fleet-scoped SQL
//! session (admin-only — see `crate::engine`'s module docs' "Reference
//! data" section) is not granted anything forensics does not already grant
//! one persona at a time. What a general SQL surface changes is not
//! WHETHER this data is readable but HOW MUCH of it one query can read at
//! once: `SELECT wallet_address, "limit", allowed_hosts FROM persona_wallets
//! JOIN persona_spend_policies USING (persona_id)` is a single fleet-wide
//! export, where the forensics HTTP surface pays one request per persona.
//! QRY-8 is the invariant that this widened blast radius is a KNOWN,
//! ACCEPTED trade — not an oversight — gated on ONE line never moving: no
//! key-MATERIAL column (`key_ref`, `credential_id`, `p256_public_key_sec1`,
//! `signing_public_key`, or any other raw public/private key byte string)
//! is EVER reachable through any of these tables, at any scope, under any
//! name. The bulk-export test in `crate::engine`'s test module pins both
//! halves: a Fleet-scope join across all four new tables returns exactly
//! the parity columns enumerated above, and this module's own
//! `qry8_no_key_material_column_in_any_new_reference_table` test (below)
//! proves none of the four key-material field names appear in ANY of their
//! Arrow schemas — not merely that today's SQL happens not to select them.
//!
//! No `event_time`-style uniform key column applies here (unlike
//! `crate::decode::usage`'s uniform keys): these rows are not journal events
//! at all, so `partition`/`position`/`turn_id` have no meaning for them —
//! `conversation_id` is `participations`' own natural join key back to the
//! event tables' `partition` column instead.
//!
//! # `conversation_id` vs. `partition`: the join needs a prefix
//!
//! `participations.conversation_id` stores the BARE conversation id exactly
//! as `PersonaStore`/`PersonaHost::attribute` receive it (e.g. `"abc123"` —
//! `crates/control-plane/src/grpc/mod.rs`'s `cid`, passed to
//! `persona.attribute` unprefixed). The event tables' `partition` column
//! (`crate::decode::events_batch`, `crate::decode::usage`) stores the
//! journal partition name, which is that same id run through
//! `crate::forensics::partition_for` — `"conv-{conversation_id}"` — a
//! `"conv-"` prefix `participations` deliberately does NOT duplicate (it
//! mirrors the persona store's own stored value verbatim, not a
//! query-layer-invented re-encoding). A query joining the two must account
//! for the prefix explicitly, e.g. `events.partition = 'conv-' ||
//! participations.conversation_id` — see
//! `crate::engine::tests::fleet_scope_reference_tables_return_rows_and_participation_joins_to_its_conversation`
//! for a worked example.

// Doc-link-only imports: every type below is referenced from this module's
// own doc comment above but never named in code here (each submodule that
// actually decodes one of these types imports it separately for real use).
#[allow(unused_imports)]
use polyc_proto::proto::polychrome::persona::v1::{
    ExternalIdentity, Participation, PersonaCredential, PersonaProfile, SpendPolicy, UsageRollup,
    WalletLink,
};

mod core;
mod credentials;
mod spend_policies;
mod usage;
mod wallets;

pub(crate) use core::*;
pub(crate) use credentials::*;
pub(crate) use spend_policies::*;
pub(crate) use usage::*;
pub(crate) use wallets::*;

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn empty_rows_yield_zero_row_batches_with_the_full_schema() {
        let personas = decode_personas_batch(&[]).expect("batch build");
        assert_eq!(personas.num_rows(), 0);
        assert_eq!(personas.schema(), personas_schema());

        let participations = decode_participations_batch(&[]).expect("batch build");
        assert_eq!(participations.num_rows(), 0);
        assert_eq!(participations.schema(), participations_schema());

        let persona_identities = decode_persona_identities_batch(&[]).expect("batch build");
        assert_eq!(persona_identities.num_rows(), 0);
        assert_eq!(persona_identities.schema(), persona_identities_schema());

        let persona_wallets = decode_persona_wallets_batch(&[]).expect("batch build");
        assert_eq!(persona_wallets.num_rows(), 0);
        assert_eq!(persona_wallets.schema(), persona_wallets_schema());

        let persona_spend_policies = decode_persona_spend_policies_batch(&[]).expect("batch build");
        assert_eq!(persona_spend_policies.num_rows(), 0);
        assert_eq!(
            persona_spend_policies.schema(),
            persona_spend_policies_schema()
        );

        let persona_credentials = decode_persona_credentials_batch(&[]).expect("batch build");
        assert_eq!(persona_credentials.num_rows(), 0);
        assert_eq!(persona_credentials.schema(), persona_credentials_schema());

        let persona_usage = decode_persona_usage_batch(&[]).expect("batch build");
        assert_eq!(persona_usage.num_rows(), 0);
        assert_eq!(persona_usage.schema(), persona_usage_schema());
    }

    /// No key-material field name — `key_ref`, `credential_id`,
    /// `p256_public_key_sec1`, `signing_public_key` — appears in ANY of the
    /// four new tables' schemas, under ANY column name. A schema-level
    /// assertion (not merely "today's SQL doesn't select it"), per this
    /// module's own "Bulk-export invariant (QRY-8)" doc section.
    #[test]
    fn qry8_no_key_material_column_in_any_new_reference_table() {
        let banned = [
            "key_ref",
            "credential_id",
            "p256_public_key_sec1",
            "signing_public_key",
        ];
        let schemas = [
            ("persona_wallets", persona_wallets_schema()),
            ("persona_spend_policies", persona_spend_policies_schema()),
            ("persona_credentials", persona_credentials_schema()),
            ("persona_usage", persona_usage_schema()),
        ];
        for (table, schema) in schemas {
            let names: Vec<&str> = schema.fields().iter().map(|f| f.name().as_str()).collect();
            for field in &banned {
                assert!(
                    !names.contains(field),
                    "{table} must never expose key-material column {field}"
                );
            }
        }
    }
}