Crate safe_vault [−] [src]
Implementation of the "Vault" node for the SAFE Network.
General
Vaults form an overlay network on top of a Kademlia-like routing network to provide decentralised, autonomous, cloud storage.
Clients can connect to this network and make requests to store, retrieve, mutate or delete data.
Currently the network handles two types of data chunks: ImmutableData
and
StructuredData
.
The underlying routing network provides the ability to send and receive messages from peers
or groups of "close" peers. These singletons or groups are referred to as authorities. See the
general routing docs and the Authority
docs for further details.
Vaults have two main sets of responsibilities, each set being associated with a "persona" of the Vault; managing Client accounts and managing data stored on the network. Among other things, it involves passing responsibility to new Vaults when there is local churn (i.e. a Vault joins or leaves nearby).
Client Manager Persona
The Client Manager persona (a.k.a. MaidManager
) holds an account for each Client which is
close to it in the network address space. Each such account is managed by a group of
GROUP_SIZE
Vaults.
A Client's account contains details of how many chunks of data have been put to the network by
that Client and how many new chunks can still be put. If a Client's account indicates that no
more chunks can be put to the network, the Client Managers for that Client disallow any further
Put
requests, responding with a LowBalance
error.
Clients can retrieve their account balances by sending a specific request to their Managers,
namely a GetAccountInfo
request.
Churn
When a Vault joins the network nearby, the Client Manager will remove all accounts for which it is no longer responsible and will send the remainder to the close group for each account. These are termed "refresh" messages; they allow all members of each group to synchronise their account information.
When a close Vault leaves the network, the Client Manager simply sends a set of refreshes. It will thus be given any accounts which it was not previously managing, but for which it is now responsible.
Data Manager Persona
The Data Manager persona holds a chunk store where data chunks are held and is responsible for chunks whose names are close to it in the network address space. Not every Data Manager in a given group close to a chunk will necessarily hold that chunk, but every one should be aware of which peers do hold it.
Chunk Cache
The Data Manager also holds a cache of chunks for which it is not directly responsible. These are chunks for which the Vault was recently responsible but has churned out of the close group, or chunks which have recently been requested by a Client.
The purpose for caching the former is that there is a reasonable chance that the Vault will soon become responsible for that chunk again due to churn. This saves the chunk from having to be retrieved via new Get requests.
The rationale for the latter is that popular chunks will receive many Get requests by many different Clients and the work of satisfying these requests needs to be spread across more Vaults than normal. By opportunistically caching chunks which have been retrieved by Clients, popular chunks will end up being cached by many more Vaults than the close group, hence the close group will be protected from excessive numbers of Get requests.
Churn
In a similar way to the Client Manager, the Data Manager will clear out records and send refresh messages. In addition, it will also try to retrieve any chunks for which it is responsible from any close peer which is not currently busy, i.e. a peer which is not currently being asked for a chunk by this Vault. Chunk replication to this Vault continues repeatedly until it holds all chunks for which it is responsible.
Message Flows
As mentioned above, two chunk types are handled by Vaults: ImmutableData
and StructuredData
.
Both types can be stored to the network by using Put
messages and retrieved from the network
by using Get
messages. StructuredData
can also be mutated by using Post
messages or
removed by using Delete
messages.
The following sections give an overview of the end-to-end process involved in each of these operations.
Put ImmutableData
- Client sends
Put
to itsMaidManager
group MaidManager
responds with failure (ending the message flow) if:- Client doesn't have an account
- Client's account has insufficient balance to be allowed to store a new chunk
MaidManager
sendsPut
toDataManager
group for the chunk- if
DataManager
already has a copy of the chunk, it responds with success toMaidManager
group. Otherwise, it:- responds with failure (
NetworkFull
error) toMaidManager
group if its chunkstore is full - tries to store the chunk and responds with appropriate success or failure to
MaidManager
group - if the store attempt was successful, sends
Refresh
to its fellowDataManager
s
- responds with failure (
MaidManager
then:- refunds the Client's account if the
DataManager
group reports failure - sends
Refresh
to its fellowMaidManager
s - responds with appropriate success or failure to Client
- refunds the Client's account if the
Put StructuredData
- Client sends
Put
to itsMaidManager
group - if the type tag of the chunk is
0
, (representing an account creation request)MaidManager
responds with failure (ending the message flow) if this account already exists - if the type tag of the chunk is not
0
,MaidManager
responds with failure (ending the message flow) if:- Client doesn't have an account
- Client's account has insufficient balance to be allowed to store a new chunk
MaidManager
sendsPut
toDataManager
group for the chunk- if
DataManager
already has a copy of the chunk, it responds with failure toMaidManager
group. Otherwise, it:- responds with failure ("network full" error) to
MaidManager
group if its chunkstore is full - tries to store the chunk and responds with appropriate success or failure to
MaidManager
group - if the store attempt was successful, sends
Refresh
to its fellowDataManager
s
- responds with failure ("network full" error) to
MaidManager
then:- refunds the Client's account if the
DataManager
group reports failure - sends
Refresh
to its fellowMaidManager
s - responds with appropriate success or failure to Client
- refunds the Client's account if the
Get ImmutableData
or Get StructuredData
- Client sends
Get
to theDataManager
group for the chunk DataManager
responds with appropriate success or failure to Client depending on whether or not it can retrieve the requested chunk from its chunkstore
Post StructuredData
- Client sends
Post
to theDataManager
group for the chunk DataManager
responds with failure (ending the message flow) to the Client if:- there isn't an existing
StructuredData
chunk with the same name - the new chunk isn't a valid successor of the existing one
- the chunkstore can't store the new chunk
- there isn't an existing
- if storing the chunk succeeds,
DataManager
:- responds with success to the Client
- sends
Refresh
to its fellowDataManager
s
Delete StructuredData
Note that deleting involves sending a new version of the chunk to be deleted in order to allow validation of the request.
- Client sends
Delete
to theDataManager
group for the chunk DataManager
responds with failure (ending the message flow) to the Client if:- there isn't an existing
StructuredData
chunk with the same name - the new chunk isn't a valid successor of the existing one
- the chunkstore can't delete the existing chunk
- there isn't an existing
- if deleting the chunk succeeds,
DataManager
responds with success to the Client
Structs
Config |
Lets a vault configure a wallet address and storage limit. |
Vault |
Main struct to hold all personas and Routing instance |
Constants
GROUP_SIZE |
The number of nodes in groups managing data and user accounts. |