CW-Storage-Plus: Enhanced/experimental storage engines for CosmWasm
The ideas in here are based on the cosmwasm-storage
crate. However,
after much usage, we decided a complete rewrite could allow us to add
more powerful and easy to use interfaces. Here are those interfaces.
Status: experimental
You currently should not be using this crate outside of the cosmwasm-plus
repo. This is a first draft of many types. We will update the status
after they have been used more heavily and the interfaces stabilized.
The ideas/desired functionality in here should be more or final, just the form to express them that is not final. As we add new functionality, we will continue to refine the foundations, but maintain semver.
Usage Overview
We introduce two main classes to provide a productive abstraction
on top of cosmwasm_std::Storage
. They are Item
, which is
a typed wrapper around one database key, providing some helper functions
for interacting with it without dealing with raw bytes. And Map
,
which allows you to store multiple unique typed objects under a prefix,
indexed by a simple (&[u8]
) or compound (eg. (&[u8], &[u8])
) primary key.
These correspond to the concepts represented in cosmwasm_storage
by
Singleton
and Bucket
, but with a re-designed API and implementation
to require less typing for developers and less gas usage in the contracts.
Item
The usage of an Item
is pretty straight-forward.
You must simply provide the proper type, as well as a database key not
used by any other item. Then it will provide you with a nice interface
to interact with such data.
If you are coming from using Singleton
, the biggest change is that
we no longer store Storage
inside, meaning we don't need read and write
variants of the object, just one type. Furthermore, we use const fn
to create the Item
, allowing it to be defined as a global compile-time
constant rather than a function that must be constructed each time,
which saves gas as well as typing.
Example Usage:
// note const constructor rather than 2 functions with Singleton
const CONFIG: = new;
Map
The usage of an Map
is a little more complex, but
is still pretty straight-forward. You can imagine it as a storage-backed
BTreeMap
, allowing key-value lookups with typed values. In addition,
we support not only simple binary keys (&[u8]
), but tuples, which are
combined. This allows us to store allowances as composite keys
eg. (owner, spender)
to look up the balance.
Beyond direct lookups, we have a super power not found in Ethereum -
iteration. That's right, you can list all items in a Map
, or only
part of them. We can efficiently allow pagination over these items as
well, starting at the point the last query ended, with low gas costs.
This requires the iterator
feature to be enabled in cw-storage-plus
(which automatically enables it in cosmwasm-std
as well).
If you are coming from using Bucket
, the biggest change is that
we no longer store Storage
inside, meaning we don't need read and write
variants of the object, just one type. Furthermore, we use const fn
to create the Bucket
, allowing it to be defined as a global compile-time
constant rather than a function that must be constructed each time,
which saves gas as well as typing. In addition, the composite indexes
(tuples) is more ergonomic and expressive of intention, and the range
interface has been improved.
Here is an example with normal (simple) keys:
const PEOPLE: = new;
Composite Keys
There are times when we want to use multiple items as a key, for example, when storing allowances based on account owner and spender. We could try to manually concatenate them before calling, but that can lead to overlap, and is a bit low-level for us. Also, by explicitly separating the keys, we can easily provide helpers to do range queries over a prefix, such as "show me all allowances for one owner" (first part of the composite key). Just like you'd expect from your favorite database.
Here how we use it with composite keys. Just define a tuple as a key and use that everywhere you used a byte slice above.
// Note the tuple for primary key. We support one slice, or a 2 or 3-tuple
// adding longer tuples is quite easy but unlikely to be needed.
const ALLOWANCE: = new;
Path
Under the scenes, we create a Path
from the Map
when accessing a key.
PEOPLE.load(&store, b"jack") == PEOPLE.key(b"jack").load()
.
Map.key()
returns a Path
, which has the same interface as Item
,
reusing the calculated path to this key.
For simple keys, this is just a bit less typing and a bit less gas if you
use the same key for many calls. However, for composite keys, like
(b"owner", b"spender")
it is much less typing. And highly recommended anywhere
you will use the a composite key even twice:
const PEOPLE: = new;
const ALLOWANCE: = new;
Prefix
In addition to getting one particular item out of a map, we can iterate over the map
(or a subset of the map). This let's us answer questions like "show me all tokens",
and we provide some nice Bound
s helpers to easily allow pagination or custom ranges.
The general format is to get a Prefix
by calling map.prefix(k)
, where k
is exactly
one less item than the normal key (If map.key()
took (&[u8], &[u8])
, then map.prefix()
takes &[u8]
.
If map.key()
took &[u8]
, map.prefix()
takes ()
). Once we have a prefix space, we can iterate
over all items with range(store, min, max, order)
. It supports Order::Ascending
or Order::Descending
.
min
is the lower bound and max
is the higher bound.
If the min
and max
bounds, it will return all items under this prefix. You can use .take(n)
to
limit the results to n
items and start doing pagination. You can also set the min
bound to
eg. Bound::Exclusive(last_value)
to start iterating over all items after the last value. Combined with
take
, we easily have pagination support. You can also use Bound::Inclusive(x)
when you want to include any
perfect matches. To better understand the API, please read the following example:
const PEOPLE: = new;
const ALLOWANCE: = new;
Indexed Map
TODO: we are working on a version of a map that manages multiple secondary indexed transparently. That work is coming soon.