mstar.communication.wire

Typed msgpack encoding for everything that crosses a communicator edge. Msgpack is readable by both Python and Rust, allowing, e.g., the sender to be fully Rust and the receiver to be Python.

The encoding is type-driven: the decoder knows the declared type of every field, so the wire carries values, not type tags, except where a field is polymorphic (see _POLYMORPHIC).

Field plans are compiled once per class into a tuple of closures; resulving annotations per message is about 17x slower than pickle.

Notes: - dict keyed by a tuple (graph_timings) becomes a list of entries. - set becomes a list and is rebuilt on decode. - PublishedInfo / ResourceReqConfig / MessageBody are abstract, so

those fields carry [tag, payload].

• Loosely typed fields (bare dict, Any, model_kwargs) use plain msgpack where it round-trips exactly, and data-only extensions for tuples, sets, non-string map keys, dataclasses, torch sizes/dtypes, NumPy values, dates, and the msgpack timestamps/extensions a /generate/ws client sends. Unsupported Python objects are rejected at send time. Incoming frames never run pickle; the only classes a frame can name are registered or mstar dataclasses, built from decoded field values. A Rust sender splices these as opaque, already-encoded bytes.

Module Attributes

AUTO_PREFIX	Tag prefix for a dataclass reached through a loosely-typed field and never explicitly registered.
PICKLED_EXT	retained only to reject old frames clearly.
OPAQUE	never decoded.
DATA	Data-only envelope for a payload that is not a registered message.

Functions

decode(data)
encode(msg)
encode_field(value, hint)	One field's value, encoded as its DECLARED type, on its own.
encode_fields(values, hints)	Several field values as one msgpack array, each by its declared type.
register(tag, cls)	Register a class whose definition we do not own.
wire_type(tag)	Register a concrete class under tag for polymorphic fields.

Exceptions

WireError

mstar.communication.wire.AUTO_PREFIX = 'auto:'

Tag prefix for a dataclass reached through a loosely-typed field and never explicitly registered. Self-describing so the decoder can find it without maintenance – the curated tags in wire_types.py stay short.

mstar.communication.wire.DATA = '__data__'

Data-only envelope for a payload that is not a registered message.

mstar.communication.wire.OPAQUE = '__py__'

never decoded.

Type:

Legacy whole-message pickle tag

mstar.communication.wire.PICKLED_EXT = 80

retained only to reject old frames clearly.

Type:

Legacy pickle code

exception mstar.communication.wire.WireError

Bases: Exception

mstar.communication.wire.decode(data)

Parameters:

data (bytes)

mstar.communication.wire.encode(msg)

Return type:

bytes

mstar.communication.wire.encode_field(value, hint)

One field’s value, encoded as its DECLARED type, on its own.

For a sender that builds the rest of the frame elsewhere: the Rust graph runtime encodes what it owns and splices these in as opaque values, so it never has to know CurrentForwardPassInfo or grow a case for every new PublishedInfo subclass.

Passing the declared type is what makes the result safe to splice – it selects the same encoder _plan would have chosen for that field, so the frame decodes identically to having built the whole message here.

Return type:

bytes

mstar.communication.wire.encode_fields(values, hints)

Several field values as one msgpack array, each by its declared type.

For a payload that is a bare tuple of values rather than a message. Unlike encode, this omits the top-level envelope, so Rust can split the array and splice its fields into the frame it builds.

Return type:

bytes

mstar.communication.wire.register(tag, cls)

Register a class whose definition we do not own.

Parameters:

• tag (str)

• cls (type)

Return type:

None

mstar.communication.wire.wire_type(tag)

Register a concrete class under tag for polymorphic fields.

Parameters:

tag (str)