mstar.conductor.request_info

Functions

merge_publish_info(into, other)

Fold one resource's published state into what is already held.

Classes

CurrentForwardConductorMetadata(graph_walk, ...)	Full-model forward pass-level metadata for running the current forward pass.
CurrentForwardPassInfo(request_id, ...[, ...])	Information that is passed into the worker / engines about this request at the current forward pass
PartitionDefinition(name, graph_walks[, ...])	Defines a partition within a model's computation graph.
PartitionState(partition_name, metadata[, ...])	Per-partition conductor-level state for a request.
StreamingConnectionState(from_partition, ...)	Per-connection streaming state tracked by the conductor.

class mstar.conductor.request_info.CurrentForwardConductorMetadata(graph_walk, is_prefill, input_modalities=<factory>, output_modalities=<factory>, kwargs=<factory>)

Bases: object

Full-model forward pass-level metadata for running the current forward pass. On the conductor/model level.

Parameters:

• graph_walk (str)

• is_prefill (bool)

• input_modalities (list[str])

• output_modalities (list[str])

• kwargs (dict)

graph_walk: str

input_modalities: list[str]

is_prefill: bool

kwargs: dict

output_modalities: list[str]

class mstar.conductor.request_info.CurrentForwardPassInfo(request_id, graph_walk, fwd_index, random_seed, max_tokens, resource_configs=<factory>, step_metadata=<factory>, resource_publish_info=<factory>, partition_name='default', loop_stop_times=<factory>, dynamic_loop_iter_counts=<factory>)

Bases: object

Information that is passed into the worker / engines about this request at the current forward pass

Parameters:

• request_id (str)

• graph_walk (str)

• fwd_index (int)

• random_seed (int)

• max_tokens (int)

• resource_configs (dict[str, ResourceReqConfig])

• step_metadata (dict)

• resource_publish_info (dict[str, PublishedInfo])

• partition_name (str)

• loop_stop_times (dict[str, NestedLoopIndices])

• dynamic_loop_iter_counts (dict[str, int])

clear_loop_stop_info()

dynamic_loop_iter_counts: dict[str, int]

fwd_index: int

graph_walk: str

loop_stop_times: dict[str, NestedLoopIndices]

max_tokens: int

partition_name: str = 'default'

random_seed: int

request_id: str

resource_configs: dict[str, ResourceReqConfig]

resource_publish_info: dict[str, PublishedInfo]

step_metadata: dict

update_publish_info(other)

Parameters:

other (dict[str, PublishedInfo])

class mstar.conductor.request_info.PartitionDefinition(name, graph_walks, initial_walk=None, producer_partitions=<factory>)

Bases: object

Defines a partition within a model’s computation graph.

Each partition has its own set of graph walks and transition logic, and can run asynchronously relative to other partitions.

Parameters:

• name (str)

• graph_walks (set[str])

• initial_walk (str | None)

• producer_partitions (list[str])

graph_walks: set[str]

initial_walk: str | None = None

name: str

producer_partitions: list[str]

class mstar.conductor.request_info.PartitionState(partition_name, metadata, fwd_pass_number=0, random_seed=0, is_done=False, completed_worker_graph_ids=<factory>, current_worker_graph_ids=<factory>, wg_rank_completions=<factory>, num_output_tokens=0, curr_forward_outputs=<factory>, resource_publish_info=<factory>)

Bases: object

Per-partition conductor-level state for a request.

Parameters:

• partition_name (str)

• metadata (CurrentForwardConductorMetadata)

• fwd_pass_number (int)

• random_seed (int)

• is_done (bool)

• completed_worker_graph_ids (set[str])

• current_worker_graph_ids (set[str])

• wg_rank_completions (dict[str, int])

• num_output_tokens (int)

• curr_forward_outputs (list[str])

• resource_publish_info (dict[str, PublishedInfo])

completed_worker_graph_ids: set[str]

curr_forward_outputs: list[str]

current_worker_graph_ids: set[str]

fwd_pass_number: int = 0

is_done: bool = False

metadata: CurrentForwardConductorMetadata

num_output_tokens: int = 0

partition_name: str

random_seed: int = 0

resource_publish_info: dict[str, PublishedInfo]

wg_rank_completions: dict[str, int]

class mstar.conductor.request_info.StreamingConnectionState(from_partition, to_partition, edge_name, token_count=0, consumed_count=0, producer_done=False)

Bases: object

Per-connection streaming state tracked by the conductor.

Parameters:

• from_partition (str)

• to_partition (str)

• edge_name (str)

• token_count (int)

• consumed_count (int)

• producer_done (bool)

consumed_count: int = 0

edge_name: str

from_partition: str

producer_done: bool = False

to_partition: str

token_count: int = 0

mstar.conductor.request_info.merge_publish_info(into, other)

Fold one resource’s published state into what is already held.

Merging is the resource’s own business (a KV cache folds in another rank’s shard rather than replacing it), so an existing entry gets update and only a new key is taken wholesale.

Parameters:

• into (dict[str, PublishedInfo])

• other (dict[str, PublishedInfo])

Return type:

None