RBE Design Document¶

Partly superseded. The Packet Composition and any_msg sections below predate the frames layer, which implements both: see the Frames reference and the framing design notes. The annotation tables are the shorter version of the Annotations reference. This page needs a rewrite; until then it is kept for the principles and non-goals.

This document describes RBE's intended design. Pieces that are not implemented yet are marked 🚧 Planned; everything else reflects the current state of the library. See Requirements for the exhaustive, numbered specification and Type Concepts for the concept hierarchy that drives dispatch between code paths.

Core Principles¶

RBE is built around declarative programming. The user describes a binary protocol through plain C++ structs and annotations; the library derives all serialization and deserialization logic from those declarations at compile time.

The library imposes no runtime overhead over a hand-written implementation. When a struct is trivially serializable (all fields are trivial types with no special annotations), serialization is implemented in terms of memcpy.

The library only supports protocols with a fixed field order. Variable-order protocols and complex nested types are out of scope.

Protocol Declaration¶

Layout is expressed through the type system rather than annotations. Field sizes and offsets are derived from the field's type; non-standard layouts use RBE-provided types instead of annotations. Annotations are reserved for protocol semantics only. See the Annotations Reference for the complete, up-to-date list.

Struct-level annotations¶

Field-level annotations¶

RBE types — 🚧 Planned¶

Non-standard layouts and wire-specific types are meant to be expressed through types provided by the library. None of these exist yet; today, non-standard widths and variable-length fields must be handled by hand.

Serialization¶

Serialization is eager: rbe::serialize executes immediately and writes the result into the provided output buffer, returning the number of bytes written.

std::size_t bytes_written = rbe::serialize(buffer, msg); // single message

When all fields in a struct are standard trivial types (no RBE types, no endianness annotation forcing a swap), serialization is reduced to a memcpy.

🚧 Planned: a variadic overload for composing a header with one or more payloads in a single call (rbe::serialize(buffer, header, payload)) is not implemented yet — see Packet Composition below.

Deserialization¶

Unlike serialization, deserialization supports three explicit strategies, selected with a tag argument:

auto eager_msg = rbe::deserialize<cboe::AddOrder>(buffer, rbe::dsrl::eager);    // full copy, upfront
auto lazy_msg  = rbe::deserialize<cboe::AddOrder>(buffer, rbe::dsrl::lazy);     // view, reads on demand
auto& inplace  = rbe::deserialize<cboe::AddOrder>(buffer, rbe::dsrl::in_place); // zero-copy bitcast

• Eager materializes the entire object upfront; works for any wirable, default-constructible type.
• Lazy returns a lightweight view (rbe::dsrl::proxy<T>) over the buffer and reads/converts each field on demand — no full copy is made:

  auto length = lazy_msg.field("length"); // compile-time error if the field does not exist
  

• In-place reinterprets the buffer directly as T via bitcast — zero-copy, zero-allocation, but only available for trivially wirable types, and does not perform endianness translation.

Endianness translation to host byte order is applied per field at read time for the eager and lazy strategies. See Type Concepts for the exact constraints each mode requires.

Packet Composition — 🚧 Planned¶

RBE is meant to provide a generic packet composition type that groups a header and one or more payloads into a single unit, with a flatten() utility to decompose a packet back into its constituent headers and payloads. Neither exists yet.

Type Erasure and Message Dispatch — any_msg — 🚧 Planned¶

When a struct carries an =rbe::id annotation, it is meant to participate in type-erased message dispatch through any_msg, which would hold a message of any registered type and support visit/match patterns so the user can process incoming messages generically without knowing the concrete type at the call site:

rbe::any_msg<cboe::msgs> msg = receive(buffer);
msg.match(
  [](dsrl::proxy<cboe::AddOrder> msg)  { /* ... */ },
  [](dsrl::proxy<cboe::ReduceSize> msg) { /* ... */ }
);

The =rbe::id annotation and rbe::msg_list/rbe::type_list (rbe/core/message_list.hpp) already exist as building blocks, but any_msg itself is not implemented yet.

Debugging Utilities¶

• Text format — implemented. Any type annotated =rbe::fmt (or =rbe::debug, its derive preset) gets a std::format/std::ostream formatter that prints every member recursively, including base classes and bit-fields (rbe/core/fmt.hpp).
• Binary format — 🚧 Planned. A raw byte dump of the wire representation is not implemented yet.

Constraints and Non-goals¶

• Field order on the wire must be fixed and match the struct declaration order.
• Complex types (nested structs, pointers, non-trivial members) are not supported.
• Variable-length fields, optional fields, and repeated fields are supported within the fixed-order constraint.
• Only GCC ≥ 16 is supported today: C++26 static reflection is not yet implemented by any released Clang or MSVC.