Skip to main content

ArtifactExchange

Struct ArtifactExchange 

Source
pub struct ArtifactExchange { /* private fields */ }
Expand description

Node-local state of the artifact piece protocol.

Implementations§

Source§

impl ArtifactExchange

Source

pub fn new( node_id: impl Into<String>, store: Arc<ArtifactStore>, config: ExchangeConfig, ) -> Result<Self, ExchangeError>

Exchange for node_id over store.

Source

pub fn set_chain_manager(&mut self, cm: Arc<ChainManager>)

Chain transfer outcomes to cm.

Source

pub fn node_id(&self) -> &str

This node’s id.

Source

pub fn store(&self) -> &Arc<ArtifactStore> ⓘ

Backing store.

Source

pub fn config(&self) -> &ExchangeConfig

Configuration.

Source

pub fn seed_reader( &self, reader: &mut dyn Read, expect: Option<([u8; 32], u64)>, ) -> Result<ArtifactDescriptor, ExchangeError>

Split reader into pieces, store each, and register the artifact. Streams: at most one piece is in memory. When expect is given (content hash, size), a mismatch is an error and nothing is registered.

Source

pub fn seed_bytes( &self, content: &[u8], ) -> Result<ArtifactDescriptor, ExchangeError>

Self::seed_reader over an in-memory buffer.

Source

pub fn descriptor(&self, id: &ArtifactId) -> Option<ArtifactDescriptor>

Verified descriptor by id.

Source

pub fn is_verified(&self, id: &ArtifactId) -> bool

True when id is verified on this node.

Source

pub fn resolve(&self, key: &ArtifactKey) -> Option<ArtifactDescriptor>

Verified descriptor by key.

Source

pub fn have(&self, id: &ArtifactId) -> Option<Bitfield>

Pieces held locally for a verified or pending artifact, derived from the store.

Source

pub fn read_to( &self, id: &ArtifactId, sink: &mut dyn FnMut(&[u8]) -> Result<(), ExchangeError>, ) -> Result<u64, ExchangeError>

Stream the whole artifact through sink in piece order, checking every piece and the whole-content hash. Returns the byte count.

Source

pub fn read_all(&self, id: &ArtifactId) -> Result<Vec<u8>, ExchangeError>

Whole artifact in memory (bounded by materialize_limit).

Source

pub fn is_servable(&self, d: &ArtifactDescriptor) -> bool

Governance: may d be listed in broadcast facts and seeded? It must be verified on this node (its pieces assemble to its content_hash), a signed manifest that verified here must list it, the redistribution policy must allow it for Audience::Advertise, and neither the package, its signers nor the content hash may be revoked. Serving a particular peer is Self::is_servable_to.

Source

pub fn is_servable_to( &self, d: &ArtifactDescriptor, audience: &Audience<'_>, ) -> bool

Self::is_servable for a specific audience.

Source

pub fn peer_have(&self, id: &ArtifactId, peer: &str) -> Option<Bitfield>

Last have announced by peer for id.

Source§

impl ArtifactExchange

Source

pub fn seed_package( &self, manifest: &[u8], anchors: &TrustAnchors, open: &mut dyn FnMut(&FileRef) -> Result<Box<dyn Read>>, ) -> Result<ExchangedPackage, PackageExchangeError>

Seed a signed package. open yields each listed file’s bytes; files are streamed, so large payloads never sit in memory whole.

Source

pub fn seed_package_in( &self, manifest: &[u8], anchors: &TrustAnchors, kinds: &KindRegistry, open: &mut dyn FnMut(&FileRef) -> Result<Box<dyn Read>>, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::seed_package against a caller-supplied kind registry.

Source

pub fn seed_package_dir( &self, dir: &Path, anchors: &TrustAnchors, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::seed_package from an unpacked package directory.

Source

pub fn seed_package_dir_in( &self, dir: &Path, anchors: &TrustAnchors, kinds: &KindRegistry, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::seed_package_dir against a caller-supplied kind registry.

Source

pub fn seed_package_dir_manifest( &self, dir: &Path, manifest: &[u8], anchors: &TrustAnchors, kinds: &KindRegistry, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::seed_package_dir_in for a caller that already read the manifest bytes from dir (so the manifest is read once). manifest must be the content of dir’s cogpkg.json; it is verified here.

Source

pub async fn fetch_package( &self, peers: &mut PeerSet, manifest_hash: &str, anchors: &TrustAnchors, ) -> Result<ExchangedPackage, PackageExchangeError>

Fetch a signed package by its manifest hash. Signatures are checked before any file is requested; each file must match its pinned size and hash. Once the manifest verifies, this node may serve the listed files it holds verified.

Source

pub async fn fetch_package_in( &self, peers: &mut PeerSet, manifest_hash: &str, anchors: &TrustAnchors, kinds: &KindRegistry, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::fetch_package against a caller-supplied kind registry. A node that places a registered non-cog kind from a peer must use this: with the builtin registry the kind is refused, failing closed.

Source§

impl ArtifactExchange

Source

pub async fn serve( &self, stream: &mut dyn MeshStream, peer: &str, ) -> Result<ServeStats, ExchangeError>

Serve one peer until it closes the stream. Only pieces of servable artifacts (Self::is_servable) are sent. An oversize or malformed frame ends the session with an error, and so does a peer that sends nothing for serve_idle_timeout.

peer is only a claimed id (the peer is ServePeer::unverified): with a policy that serves verified peers only, nothing is served. Use Self::serve_as when admission authenticated the peer.

Source

pub async fn serve_as( &self, stream: &mut dyn MeshStream, who: &ServePeer, ) -> Result<ServeStats, ExchangeError>

Self::serve for a peer whose identity the caller has (or has not) verified: the redistribution policy sees peer for every request.

Source

pub async fn serve_frame( &self, stream: &mut dyn MeshStream, peer: &str, raw: &[u8], stats: &mut ServeStats, ) -> Result<(), ExchangeError>

Serve one frame already read from stream, for sessions that carry the piece protocol beside other traffic on one stream (the workload.ctl fetch-before-load, mesh-placement-12). Same rules as Self::serve: only servable artifacts are sent.

Source

pub async fn serve_frame_as( &self, stream: &mut dyn MeshStream, who: &ServePeer, raw: &[u8], stats: &mut ServeStats, ) -> Result<(), ExchangeError>

Self::serve_frame with the peer’s verified-ness stated.

Source

pub async fn fetch( &self, peers: &mut PeerSet, key: ArtifactKey, ) -> Result<FetchOutcome, FetchError>

Fetch key from peers with the v1 SequentialScheduler.

Source

pub async fn fetch_with( &self, peers: &mut PeerSet, key: ArtifactKey, scheduler: &mut dyn PieceScheduler, ) -> Result<FetchOutcome, FetchError>

Fetch key, trying each live peer in turn until every piece is held and the whole content verifies. Pieces already held are never requested, so a repeated call resumes an interrupted transfer. A peer whose pieces do not assemble to its declared content is dropped and its descriptor discarded. The outcome (verified or failed) is chained as artifact.fetch.

Source§

impl ArtifactExchange

Source

pub async fn swarm_fetch( self: &Arc<Self>, dialer: Arc<dyn PeerDialer>, candidates: Vec<PeerCandidate>, key: ArtifactKey, opts: &SwarmFetchOptions, ) -> Result<FetchOutcome, FetchError>

Fetch key from several holders at once (see the module docs). Pieces already held are not requested again, so a repeated call resumes. candidates are the peers that may hold it (from crate::mesh_swarm_lookup); banned peers are skipped.

A fetch by content hash trusts nobody’s descriptor: when the pieces a peer’s descriptor describes do not assemble to the content hash, that peer and the sources that vouched for the same descriptor are banned and the fetch carries on with the remaining candidates (up to three attempts), so one liar cannot block it or get honest peers blamed.

Source§

impl ArtifactExchange

Source

pub fn set_revocations(&self, list: Arc<RevocationList>) -> bool

Use list as this node’s revocation list (first call wins).

Source

pub fn is_revoked(&self, content_hash: &[u8; 32]) -> bool

True when the content hash is revoked, or every package that allowed it (or those packages’ signers) is. A package revoked while another, unrevoked one still lists the same content leaves the content allowed.

Source

pub fn apply_revocations(&self) -> Vec<RevokedArtifact>

Apply the revocation list to everything held: drop each revoked grant; when no grant is left for a content hash, evict its bytes (and any partial copy), and chain artifact.revoke and artifact.evict. Safe to call repeatedly.

Revocation evicts even pinned cache entries (the bytes are removed and serving is blocked). This sweep does not touch a workload that is already running from them; stopping those is the host’s job (WorkloadHostService::enforce_revocations), which the daemon runs after every applied revocation.

Revocation is monotonic mesh-wide: entries are only ever added, by an operator command or a signed notice, and a notice is re-flooded to every peer. Removing one locally (unrevoke_subject) is undone by the next notice a peer re-sends; to lift a revocation, clear the list on every node or issue the package under a new key.

Source

pub fn ban_peer(&self, peer: &str, reason: &str) -> bool

Ban peer: no more fetching from it or serving it. Returns true if newly banned (chained as artifact.peer_ban).

Source

pub fn unban_peer(&self, peer: &str) -> bool

Lift a ban. Returns true if the peer was banned.

Source

pub fn is_banned(&self, peer: &str) -> bool

True if peer is banned.

Source

pub fn banned_peers(&self) -> Vec<(String, String)>

Banned peers and why.

Measured link speeds to peers.

Source

pub fn pending_bytes(&self) -> u64

Bytes of downloads in progress (what their descriptors claim).

Source

pub fn servable_artifacts(&self) -> Vec<ArtifactDescriptor>

Verified artifacts this node may serve (verified, allowed, not revoked).

Source

pub fn forget(&self, id: &ArtifactId, reason: &str) -> Option<Forgotten>

Remove id (verified or pending) and the blobs no other artifact lists. The grant is kept, so a later fetch can seed it again; revocation drops the grant itself. Chains artifact.evict.

Only blobs this exchange created and nobody else has stored since (store reference count 1) are removed: the store is shared with package verification and installed workloads, which may hold the same bytes. A pinned cache entry is not forgotten, except for revoked.

Source

pub fn held_capabilities( &self, model_shards: &HashSet<[u8; 32]>, ) -> Vec<Capability>

Capabilities for what this node can serve, to merge into its facts: store.artifact.<16 hex of content hash> per artifact (attrs content, size) and, for content hashes in model_shards, model.present with the shard list. Held artifacts are probed: this node verified them itself.

Source§

impl ArtifactExchange

Source

pub async fn who_has( self: &Arc<Self>, dialer: Arc<dyn PeerDialer>, peers: &[PeerCandidate], key: ArtifactKey, ) -> Vec<Holder>

Ask peers whether they hold key. Peers that refuse, fail or are banned are left out. Sorted by peer id.

Source

pub async fn find_holders( self: &Arc<Self>, dialer: Arc<dyn PeerDialer>, facts: &NodeFactsCache, neighbors: &[PeerCandidate], key: ArtifactKey, now: u64, ) -> Vec<PeerCandidate>

Candidates for key: peers advertising it in their facts plus neighbors that answer a who_has query now.

Source§

impl ArtifactExchange

Source

pub async fn swarm_fetch_package( self: &Arc<Self>, dialer: Arc<dyn PeerDialer>, candidates: &[PeerCandidate], manifest_hash: &str, anchors: &TrustAnchors, opts: &SwarmFetchOptions, ) -> Result<ExchangedPackage, PackageExchangeError>

Fetch a signed package by its manifest hash from candidates.

Source

pub async fn swarm_fetch_package_in( self: &Arc<Self>, dialer: Arc<dyn PeerDialer>, candidates: &[PeerCandidate], manifest_hash: &str, anchors: &TrustAnchors, opts: &SwarmFetchOptions, kinds: &KindRegistry, ) -> Result<ExchangedPackage, PackageExchangeError>

Self::swarm_fetch_package against a caller-supplied kind registry.

Source§

impl ArtifactExchange

Source

pub fn grant_checkout(&self, v: &VerifiedCheckoutGrant)

Make the bytes of a verified checkout grant shareable inside the mesh: one grant per listed BLAKE3, signed by the bound grant key, with GrantOrigin::Cognitum. A revoked key or hash is skipped by grant_with, and revocation applies through the usual sweep. This does not make the bytes runnable (see super::may_run).

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Pointee for T

Source§

type Pointer = u32

Source§

fn debug( pointer: <T as Pointee>::Pointer, f: &mut Formatter<'_>, ) -> Result<(), Error>

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more