unidiomatic way to derive default trait implementations of TryFrom in/out of SerializedBytes
Two main reasons this was done rather than the normal Derive approach:
- Derive requires a separate crate
- Derive doesn't allow for use of $crate to set unambiguous fully qualified paths to things
Both of these limitations push dependency management downstream to consumer crates more than we want to. This implementation allows us to manage all dependencies explicitly right here in this crate.
There is a default implementation of SerializedBytes into and out of ()
this is the ONLY supported direct primitive round-trip, which maps to
nil in messagepack
for all other primitives, wrap them in a new type or enum
e.g. do NOT do this:
instead do this:
use holochain_serialized_bytes::prelude::*; #[derive(Serialize, Deserialize, Debug)] pub struct SomeType(u32); holochain_serial!(SomeType); let serialized_bytes = SerializedBytes::try_from(SomeType(50)).unwrap(); let some_type = SomeType::try_from(serialized_bytes).unwrap();
SomeType in a separate crate that can be shared by all producers and consumers of the
serialized bytes in a minimal way.
this is a bit more upfront work but it's the only way the compiler can check a type maps to
a specific serialization across different systems, crate versions, and refactors.
for example, say we changed
SomeType(u64) in the shared crate
with the new type the compiler can enforce roundtripping of bytes works everywhere
is used, provided all producers and consumers use the same version of the shared crate.
in the case where we have no
SomeType and would use integers directly, there is no safety.
the system can't tell the difference between a type mismatch (e.g. you just wrote u32 in the
wrong spot in one of the systems) and a serialization mismatch (e.g. the serialized bytes
produced by some system were consumed by another system using a different version of the shared
crate or something).
Developers then have to manually mentally impose the meaning of primitives over the top of code across different code-bases that are ostensibly separate projects. This introduces the effect where you can't understand/refactor one component of the system without understanding and refactoring all the other components in the same PR/QA step.
An explicit goal of SerializedBytes is to introduce stability of byte-level data interfaces across systems, provided they share the same version of a shared types crate. This means that that one component can implement a new version of the shared types and know that it will be compatible with other components when they similarly upgrade AND other components are safe to delay upgrading to the latest version of the shared crate until they are ready to move. Essentially it allows for async development workflows around serialized data.
This is especially important for wasm as the wasm hosts and guests may not even be developed by the same people/organisations, so there MUST be some compiler level guarantee that at least the shared types within the same shared crate version have compatible serialization logic.
usually when working with primitives we are within a single system, i.e. a single compilation context, a single set of dependencies, a single release/QA lifecycle in this case, while we could wrap every single primitive in a new type for maximum compile time safety it is often 'overkill' and we can eat some ambiguity for the sake of ergonomics and minimising the number of parallel types/trait implementations. in the case of parallel, decoupled, independently maintiained systems that rely on byte-level canonical representation of things that will fail (e.g. cryptographically break or (de)allocate memory incorrectly) if even one byte is wrong, the guide-rails provided by new types and enums are worth the additional up-front effort of creating a few extra shared crates/types.
see the readme for more discussion around this