of_fix
of_fix is the reusable FIX tag-value codec and deterministic session
foundation for Orderflow execution adapters. It is intentionally separate from
of_execution_adapters so FIX wire handling, validation, profiles, session
coordination, and certification tooling can evolve without coupling protocol
state to one broker adapter or network stack.
First Release: 0.1.0
of_fix starts at 0.1.0 in the broader Orderflow 0.5.0 development line.
Its public surface is additive: the original codec APIs remain unchanged, and
the transport-independent session engine composes those primitives.
Included now:
- borrowed tag-value field views;
- caller-owned parse scratch buffers;
- FIX
BodyLength(9)validation; - FIX
CheckSum(10)validation; - typed FIX versions and common
MsgType(35)constants; - static dictionary/profile validation for required and disallowed tags;
- borrowed Session Reject and BusinessMessageReject parsers for counterparty diagnostics;
- reusable
FixDecoderandFixEncoderfacades; - session-state and sequence-tracking primitives;
- deterministic
FixSessionEnginecoordination for Logon/Logout, heartbeat, TestRequest, component identity, gap detection, ResendRequest, SequenceReset, duplicate handling, and custom application messages; - caller-driven monotonic timers, fixed-size TestReqID storage, typed session actions/errors, and allocation-free session metrics;
- resend-range detection for inbound sequence gaps;
- sequence-reset guardrails that reject decreasing next expected sequence;
- borrowed session identity and sequence snapshot primitives for persistence;
- owned sequence snapshot types and an atomic file-backed sequence snapshot store for durable restart/reconnect state;
- bounded in-memory resend-store primitives for replay/gap-fill planning;
- append-only file-backed durable resend-message persistence that rebuilds the bounded in-memory resend planner after restart;
- bounded in-memory transcript capture primitives for certification/audit workflows;
- possible-duplicate replay encoding with
PossDupFlag(43)andOrigSendingTime(122); - typed session/admin builders for Logon, Heartbeat, TestRequest, ResendRequest, SequenceReset gap fill, and Logout;
- typed order-entry builders for NewOrderSingle, OrderCancelRequest,
OrderCancelReplaceRequest, OrderStatusRequest, OrderMassCancelRequest, and
OrderMassStatusRequest, including optional
Account(1)on single-order entry/cancel/replace messages and optionalStopPx(99)on single-order entry/replace messages; - common tag constants and extraction helpers;
- caller-owned encode buffers that fill
BodyLengthandCheckSum; - debug rendering with
|delimiters outside the live hot path.
Not included yet:
- TCP/TLS transport;
- credentials, authentication extensions, or a wall-clock/scheduler;
- automatic transmission of replay/gap-fill plans from a resend store;
- nested repeating-group dictionaries and automatic dictionary generation;
- venue certification harness;
- OMS execution mapping.
Low-Latency Design
The codec is designed for execution hot paths:
- parse from
&[u8], notString; - expose borrowed value slices instead of allocating field strings;
- use caller-provided
&mut [FixFieldView]scratch for parse output; - avoid
HashMap<Tag, String>as the primary representation; - validate
BodyLengthandCheckSumdirectly over the raw byte buffer; - keep profile rules as static borrowed slices;
- expose reject diagnostics as borrowed views;
- track inbound/outbound sequence numbers with plain integer state;
- coordinate session lifecycle without sockets, locks, background tasks, or per-timer allocation;
- snapshot sequence counters without tying the codec to a storage backend;
- persist sequence snapshots outside the per-message hot path with explicit sync policy and checksum validation;
- retain outbound resend frames behind explicit message/byte bounds;
- append original outbound frames to a durable resend log with monotonic sequence validation and checksum-chain protection;
- plan replay versus gap-fill actions into caller-owned buffers;
- capture transcript metadata with bounded optional raw retention and a deterministic rolling hash;
- rewrite replayed frames without a generic object model;
- build session/admin messages into reusable buffers without
format!; - build order-entry messages from borrowed identifiers, symbols, quantities, prices, and timestamps;
- encode into caller-owned
Vec<u8>buffers; - keep debug rendering opt-in and outside hot paths.
Public API Meaning
FixFieldView, FixMessageView, FixTag, and the decoder facade represent a
borrowed view over one validated FIX frame. They avoid allocating field names
or values and are valid only while the source bytes remain alive. Encoder
functions write into caller-owned buffers and calculate BodyLength(9) and
CheckSum(10) as part of final frame construction.
FixDictionary and profile rules validate message-specific required, optional,
and disallowed tags. Session identity, header, sequence, and action types model
administrative protocol state independently from a socket. The session engine
consumes inbound views and emits explicit actions such as send, resend, gap
fill, reject, ready, or disconnect for the host transport to carry out.
Sequence trackers, snapshots, resend stores, replay plans, and transcript capture are restart and certification surfaces. They make gaps, duplicate messages, possible-duplicate replay, and persistence boundaries observable; they do not create a scheduler or transmit bytes. Order builders provide typed FIX entry messages, while the execution adapter remains responsible for mapping those messages to the OMS.
FixRepeatingGroupDefinition describes one flat repeating group: its count
tag, the delimiter tag that begins every entry, and the tags allowed within an
entry. FixMessageView::repeating_group interprets that definition over the
already parsed field array. The returned FixRepeatingGroupView and
FixRepeatingGroup values borrow the original message and use caller-owned
FixRepeatingGroupEntry scratch for boundaries; they do not allocate or copy
tag values. This supports venue party, allocation, instrument-leg, and
routing groups on a latency-sensitive path while keeping ownership explicit.
The matching encode_message_with_repeating_group function and
FixEncoder::encode_with_repeating_group method generate the count field and
validate each entry before writing it. They append one flat group after the
ordinary body fields and retain the existing BodyLength(9) and CheckSum(10)
behavior. Nested groups, automatic data-dictionary loading, and venue
certification scenarios remain responsibilities of a profile or adapter
layer.
Repeating Group Support
FIX repeating groups are a count field followed by a sequence of entries. Each
entry starts with a delimiter field; subsequent fields belong to that entry
until the next delimiter or the end of the group. For example, a party group
may be represented by NoPartyIDs(453), followed by entries beginning with
PartyID(448) and containing PartyIDSource(447) and PartyRole(452).
of_fix represents this structure with a small explicit definition rather
than a general-purpose object tree. The definition is borrowed and can be
declared as static profile data:
use ;
const PARTY_FIELDS: & = &;
let party_group = new;
let first_party = ;
let second_party = ;
let groups = ;
let mut raw = Vecwith_capacity;
encode_message_with_repeating_group?;
let mut fields = ;
let message = parse_message?;
let mut entry_scratch = ;
let parties = message.repeating_group?;
assert_eq!;
assert_eq!;
# Ok::
The decoder first reads the count and checks that the supplied scratch has at
least that many entries. It then verifies every delimiter and every tag in
each entry. A missing count, invalid count, insufficient scratch, missing
delimiter, or group tag found outside the declared entries returns
FixGroupError. Group fields must remain contiguous: a later allowed group tag
after an unrelated field is rejected rather than silently reinterpreted.
Field values still borrow the source frame, so the frame must remain alive while
callers inspect the group.
The encoder applies the same structural contract in the opposite direction.
An empty group is valid and emits a zero count. An entry must be non-empty,
must start with the delimiter, and may contain only tags in field_tags.
Values are checked for SOH before any bytes are written. The encoder owns the
count tag and rejects a duplicate count tag in ordinary fields, repeated
delimiters inside an entry, and tags 8, 9, 35, and 10 inside group
entries. Because the group is appended after ordinary body fields, a venue
profile that requires a different field ordering should build its complete
ordered field slices above this primitive or add a dedicated typed builder.
This first API supports flat groups intentionally. Nested groups require a recursive schema and more involved boundary validation; they should be added as a separate additive API rather than changing the meaning of the existing definition or view types.
Repeating-Group Failure Modes
The strict group contract is intentional. It prevents a venue-specific field from being silently interpreted as a different group entry:
| Operation | Failure | Meaning |
|---|---|---|
| Decode | MissingCountTag or InvalidCount |
The declared entry count cannot be trusted. |
| Decode | ScratchTooSmall |
The caller must provide one boundary slot per declared entry. |
| Decode | MissingDelimiter |
An entry does not start where the profile says it must. |
| Decode | UnexpectedField |
A permitted group tag appears after the contiguous group boundary. |
| Encode | DuplicateRepeatingGroupCountTag |
The generated count would be ambiguous with a caller field. |
| Encode | MissingRepeatingGroupDelimiter |
An entry is empty or starts with the wrong tag. |
| Encode | RepeatedRepeatingGroupDelimiter |
One entry would be decoded as multiple entries. |
| Encode | InvalidRepeatingGroupField |
The entry contains a tag outside the profile definition. |
The encoder checks the group definition and group entries before clearing the output buffer. This makes structural profile errors observable without partially replacing a previously encoded frame. Applications should treat these errors as profile/configuration failures, correct the venue definition or field ordering, and retry with a new message rather than relaxing the decoder.
Decode Example
use ;
let mut raw = Vecnew;
encode_message?;
let mut scratch = ;
let msg = parse_message?;
assert_eq!;
assert_eq!;
# Ok::
Session Engine Example
The engine writes administrative frames into a reusable caller-owned buffer. The host owns TCP/TLS, timestamp formatting, persistence, and actual sends.
use ;
let config = new?;
let id = new?;
let mut session = new;
let mut outbound = Vecwith_capacity;
session.on_transport_connecting?;
let action = session.on_transport_connected?;
assert!;
// A transport test harness supplies the counterparty Logon frame.
let mut inbound = Vecwith_capacity;
encode_logon?;
let mut scratch = ;
let logon = parse_message?;
let action = session.on_inbound?;
assert_eq!;
assert_eq!;
# Ok::
Encode Example
use ;
let mut out = Vecnew;
encode_message?;
assert!;
# Ok::
Profile Validation Example
use ;
static REQUIRED: & = &;
static RULES: & = &;
let dictionary = new;
let mut raw = Vecnew;
encode_message?;
let mut scratch = ;
let msg = parse_message?;
dictionary.validate?;
# Ok::
Reject Parsing Example
use ;
let mut raw = Vecnew;
encode_message?;
let mut scratch = ;
let msg = parse_message?;
let reject = parse_session_reject?;
assert_eq!;
assert_eq!;
assert_eq!;
# Ok::
Sequence Tracking Example
use ;
let mut tracker = new;
assert_eq!;
assert!;
# Ok::
Sequence Snapshot Example
use ;
let session = new?;
let tracker = from_next;
let snapshot = tracker.snapshot?;
let restored = from_snapshot;
assert_eq!;
assert_eq!;
# Ok::
Sequence Persistence Example
FileFixSequenceSnapshotStore persists the latest FIX sequence snapshot as a
small checksum-validated binary file. It is meant for session startup,
reconnect, and end-of-day handoff logic, not for every inbound/outbound FIX
message. Hot session code should update FixSequenceTracker in memory and save
snapshots at explicit durability points chosen by the host.
use ;
let root = temp_dir.join;
let _ = remove_dir_all;
let mut store = open?;
let session = new?;
let tracker = from_next;
let snapshot = tracker.snapshot?;
let manifest = store.save_snapshot?;
assert_eq!;
let loaded = store.load_latest?.expect;
assert!;
let restored = from_snapshot;
assert_eq!;
# let _ = remove_dir_all;
# Ok::
Session Admin Builder Example
use ;
let header = new;
let mut out = Vecwith_capacity;
encode_logon?;
assert!;
# Ok::
Order Builder Example
use ;
let header = new;
let order = new
.with_price
.with_time_in_force;
let mut out = Vecwith_capacity;
encode_new_order_single?;
# Ok::
Resend Planning Example
use ;
let mut store = new;
store.record_sent?;
store.record_sent?;
store.record_sent?;
let mut actions = Vecnew;
let summary = store.plan_resend_range;
assert_eq!;
assert_eq!;
# Ok::
Durable Resend Store Example
FileFixDurableResendStore records original outbound FIX frames in an
append-only binary log. On restart, the host can reload the durable records into
FixResendStore and keep using the same resend planner. Retransmitted
PossDupFlag(43)=Y messages should not be appended as new original sends.
use ;
let path = temp_dir.join;
let _ = remove_file;
let mut durable = open?;
durable.record_sent?;
durable.record_sent?;
drop;
let durable = open?;
let mut replay = new;
let report = durable.load_into?;
assert_eq!;
let mut actions = Vecnew;
replay.plan_resend_range;
assert_eq!;
# let _ = remove_file;
# Ok::
Transcript Capture Example
use ;
let header = new;
let mut raw = Vecnew;
encode_heartbeat?;
let mut scratch = ;
let message = parse_message?;
let mut transcript =
new;
transcript.record_message?;
let metrics = transcript.metrics;
assert_eq!;
assert_eq!;
assert_ne!;
# Ok::
Possible-Duplicate Replay Example
use ;
let header = new;
let mut original = Vecnew;
encode_heartbeat?;
let mut scratch = ;
let source = parse_message?;
let mut replay = Vecnew;
encode_poss_dup_replay?;
let mut replay_scratch = ;
let replayed = parse_message?;
assert_eq!;
assert_eq!;
assert_eq!;
# Ok::
Validation Semantics
parse_message rejects frames when:
- required tags
8,9, or10are missing; - fields are malformed;
- tags are non-numeric;
- the provided scratch buffer is too small;
BodyLength(9)does not match the byte count from the body start through the delimiter before tag10;CheckSum(10)is not exactly three digits or does not equal the modulo-256 byte sum before the checksum field.
This is deliberately strict for production execution paths. If a counterparty requires a relaxed policy, that should live in a profile layer above this codec.
FixDictionary adds an optional second validation phase after the wire frame is
valid. It checks the parsed BeginString(8) against a configured
FixVersion, locates a FixMessageRule by raw MsgType(35), then verifies
required and disallowed tags using borrowed field views. It does not allocate,
does not perform transport/session validation, and does not imply venue
certification.
Repeating-group structure is validated separately through
FixMessageView::repeating_group. A FixDictionary rule still validates the
message-level required and disallowed tags; it does not infer group structure,
because count, delimiter, and ordering requirements vary between FIX
versions and venue profiles.
parse_session_reject and parse_business_message_reject expose borrowed
views over Reject <3> and BusinessMessageReject <j> diagnostics. They parse
numeric reason fields and referenced sequence/tag values, but they do not
allocate strings, classify venue severity, or decide whether trading should
stop. Session engines and adapter profiles should turn these diagnostics into
health, metrics, and fail-closed policies.
FixSequenceTracker adds deterministic sequence bookkeeping for session
engines and adapters. It accepts expected inbound sequence numbers, reports
missing ranges as FixSequenceAction::Gap, treats lower PossDupFlag=Y
messages as duplicates, flags unmarked lower sequence numbers as too-low, and
assigns outbound sequence numbers monotonically.
FixSessionId and FixSequenceSnapshot provide borrowed, storage-neutral
state snapshots. They do not write files or choose durability policy; WAL,
checkpoint, or database layers can serialize the session id, trading day, and
next inbound/outbound counters according to their own latency and durability
requirements.
FixSessionEngine composes the codec, admin builders, and sequence tracker. It
validates begin string and component identity, negotiates the configured
heartbeat interval, drives heartbeat/TestRequest timers from caller timestamps,
emits ResendRequest on gaps, applies SequenceReset, performs graceful Logout,
and reports application/custom messages only after session readiness. It does
not retain out-of-order frames; the transport host must redeliver a triggering
frame after gap recovery. It also does not own a resend store, so a
PeerResendRequested action remains an explicit handoff to the configured
in-memory/durable resend policy.
FixResendStore retains original outbound FIX frames behind explicit message
and byte budgets. It produces caller-owned resend plans with replay actions for
retained application/reject messages and gap-fill actions for administrative,
missing, aged, or evicted ranges. It is not durable storage and does not
transmit responses; a session engine remains responsible for encoding
SequenceReset gap fills, setting PossDupFlag(43)/OrigSendingTime(122) on
replayed messages where required by profile, and enforcing counterparty policy.
FixTranscriptCapture records certification/audit transcript metadata with
optional bounded raw frame retention. It updates cumulative counters and a
deterministic rolling hash even when raw bytes are not retained. It does not
write files, redact secrets, script certification scenarios, or generate
counterparty reports; those policies belong in host tooling built on top of the
capture records.
encode_poss_dup_replay handles the common replay rewrite step for retained
messages. It takes a validated borrowed source message, preserves its original
sequence number and application fields, writes PossDupFlag(43)=Y, replaces
SendingTime(52) with the current send time, writes OrigSendingTime(122),
and recomputes BodyLength(9) and CheckSum(10). It does not decide which
messages are replayable or send them on a socket.
The session/admin builders are intentionally small protocol helpers. They write
the common standard header fields and the required admin body fields into the
same strict encoder path used by encode_message; FixSessionEngine composes
them with deterministic timers but still does not manage sockets, durable
resend stores, or authentication extensions.
The order-entry builders provide common message shapes for NewOrderSingle, OrderCancelRequest, OrderCancelReplaceRequest, OrderStatusRequest, OrderMassCancelRequest, and OrderMassStatusRequest. They do not decide whether a limit price is required, whether a field can be changed on replace, how quantities are rounded, whether a mass-cancel or mass-status scope is enabled at a venue, or which party groups, clearing instructions, or custom tags a venue requires. Those checks belong in profile/certification layers above the codec.