Expand description
TokenSpeed’s engine IPC schema and its msgpack codec.
The messages a frontend and the engine exchange over ZMQ, encoded exactly
as the Python side encodes them (msgspec structs in
python/tokenspeed/runtime/engine/io_struct.py and zmq_wire.py). Golden
fixtures written by the Python encoder keep the two sides in step: every
fixture decodes here and re-encodes byte for byte (tests/conformance.rs).
Field order of the array-layout structs is the wire contract. To add a field, append it on both sides with a default and regenerate the fixtures.
Re-exports§
pub use codec::decode_bytes;pub use codec::decode_frames;pub use codec::encode_frames;pub use codec::Dict;pub use codec::StrOrList;pub use codec::Wire;pub use control::ControlOutput;pub use control::ControlRequest;pub use error::Error;pub use error::Result;pub use frames::engine_identity;pub use frames::EngineOutput;pub use frames::EngineRequest;pub use frames::ENGINE_CORE_DEAD;pub use handshake::EngineCoreReadyResponse;pub use handshake::HandshakeAddresses;pub use handshake::HandshakeInitMessage;pub use handshake::ReadyMessage;pub use multimodal::EncodeRequest;pub use multimodal::Modality;pub use multimodal::MultimodalDataItem;pub use multimodal::MultimodalInputs;pub use multimodal::ShmTensorHandle;pub use output::BatchTokenIDOutSlim;pub use output::RequestOutput;pub use request::SessionParams;pub use request::TokenizedGenerateReqInput;pub use sampling::SamplingParams;pub use tensor::Tensor;pub use value::Value;
Modules§
- codec
- The
Wiretrait and the struct layouts msgspec uses. - control
- Control operations (
engine/io_struct.py): cache flush, pause, memory release, weight updates, profiling and the like. - error
- Decode errors.
- frames
- ZMQ message framing on the frontend–engine sockets.
- handshake
- The startup handshake (
engine/zmq_wire.py). - msgpack
- msgpack primitives that make msgspec’s encoding choices.
- multimodal
- Multimodal payloads (
multimodal/inputs.py,multimodal/shm_transport.py), and the EPD encode request that carries an item to an encode node (epd/encode_worker.py). - output
- Per-step outputs from the engine to a frontend (
engine/io_struct.py). - request
- Requests from a frontend to the engine (
engine/io_struct.py). - sampling
- Sampling parameters (
sampling/sampling_params.py). - tensor
- Tensors as the Python codec carries them (
MsgpackEncoderinio_struct.py). - value
- Untyped values, for fields the Python schema types as
dictorAny.