mstar.api_server.request_types

Contents

mstar.api_server.request_types#

Classes

APIServerMessage(message_type[, body])

Envelope for messages received by the API server.

ClientAppend(request_id[, file_paths, ends, ...])

One append to a running request's client streams, queued behind the request's own PreprocessInput so its chunks stay in order.

ConductorStatus(metrics)

The conductor's periodic status, sent only while metrics are on: one snapshot per family in catalog.STATUS_FAMILIES, current or cumulative values only, never per-request data.

DataWorkerProfile(request_id[, ...])

Profiling reported by the API-server data worker at preprocess finish (or failure): the timestamp at which the request was handed to the conductor and the per-modality sizes of the raw inputs.

PreprocessInput(request_id, text, ...[, ...])

RequestComplete(request_id, ...[, ...])

Signals that a request has finished processing.

RequestFailed(request_id, error_message[, ...])

Signals that a request died in the engine and will produce no result.

ResultChunk(request_id, modality, data[, ...])

One chunk of generated output for a request.

ResultTensors(request_id, modality, ...)

ResultTokens(request_ids, values, ...[, ...])

One step's client-facing scalars (sampled tokens), every request in one frame.
class mstar.api_server.request_types.APIServerMessage(message_type, body=None)[source]#

Bases: object

Envelope for messages received by the API server.

Parameters:
body: ResultTensors | ResultTokens | RequestComplete | RequestFailed | ConductorStatus | None = None#
message_type: str#
class mstar.api_server.request_types.ClientAppend(request_id, file_paths=None, ends=frozenset({}), text=None, file_streams=None, model_kwargs=None, on_late=None)[source]#

Bases: object

One append to a running request’s client streams, queued behind the request’s own PreprocessInput so its chunks stay in order. The streams in ends end after this chunk. Stays inside the API server process.

Parameters:
ends: frozenset[str] = frozenset({})#
file_paths: dict[str, list[str]] | None = None#
file_streams: dict[str, list[str | None]] | None = None#
model_kwargs: dict | None = None#
on_late: Callable[[str], None] | None = None#
request_id: str#
text: str | None = None#
class mstar.api_server.request_types.ConductorStatus(metrics)[source]#

Bases: object

The conductor’s periodic status, sent only while metrics are on: one snapshot per family in catalog.STATUS_FAMILIES, current or cumulative values only, never per-request data.

Parameters:

metrics (list[MetricSnapshot])

metrics: list[MetricSnapshot]#
class mstar.api_server.request_types.DataWorkerProfile(request_id, preprocess_finish_time=None, inputs=<factory>, dequeue_time=None, input_modalities=<factory>, upload_sizes=<factory>, failed=False)[source]#

Bases: object

Profiling reported by the API-server data worker at preprocess finish (or failure): the timestamp at which the request was handed to the conductor and the per-modality sizes of the raw inputs. (The data worker’s tx/rx are read directly from its tensor manager at request completion, not via this.)

Parameters:
dequeue_time: float | None = None#
failed: bool = False#
input_modalities: list[str]#
inputs: list[InputInfo]#
preprocess_finish_time: float | None = None#
request_id: str#
upload_sizes: dict[str, list[int]]#
class mstar.api_server.request_types.PreprocessInput(request_id: str, text: str | None, file_paths: dict[str, list[str]] | None, input_modalities: list[str], output_modalities: list[str], model_kwargs: dict, prompt_parts: list[mstar.model.multimodal.PromptPart] | None = None, client_request_id: str | None = None, ends: frozenset[str] = frozenset())[source]#

Bases: object

Parameters:
client_request_id: str | None = None#
ends: frozenset[str] = frozenset({})#
file_paths: dict[str, list[str]] | None#
input_modalities: list[str]#
model_kwargs: dict#
output_modalities: list[str]#
prompt_parts: list[PromptPart] | None = None#
request_id: str#
text: str | None#
class mstar.api_server.request_types.RequestComplete(request_id, output_chunk_counts, conductor_ingest_time, conductor_finish_time, graph_timings=<factory>, rx_info=<factory>, tx_info=<factory>, conductor_arrival_time=None)[source]#

Bases: object

Signals that a request has finished processing.

Parameters:
conductor_arrival_time: float | None = None#
conductor_finish_time: float#
conductor_ingest_time: float#
graph_timings: dict[tuple[str, str], GraphTiming]#
output_chunk_counts: dict[str, int]#
request_id: str#
rx_info: list[RxInfo]#
tx_info: list[TxInfo]#
class mstar.api_server.request_types.RequestFailed(request_id, error_message, status=500)[source]#

Bases: object

Signals that a request died in the engine and will produce no result.

One message per request, so it routes through the API server’s result loop exactly like RequestComplete does.

Parameters:

  • request_id (str)

  • error_message (str)

  • status (int)

error_message: str#
request_id: str#
status: int = 500#
class mstar.api_server.request_types.ResultChunk(request_id, modality, data, metadata=<factory>)[source]#

Bases: object

One chunk of generated output for a request.

Parameters:
data: bytes#
metadata: dict#
modality: str#
request_id: str#
class mstar.api_server.request_types.ResultTensors(request_id: str, modality: str, graph_edge: mstar.graph.base.GraphEdge, loop_indices: mstar.graph.loop_indices.NestedLoopIndices, metadata: dict = <factory>)[source]#

Bases: object

Parameters:
graph_edge: GraphEdge#
loop_indices: NestedLoopIndices#
metadata: dict#
modality: str#
request_id: str#
class mstar.api_server.request_types.ResultTokens(request_ids, values, loop_indices, signal, modality, metadata=<factory>)[source]#

Bases: object

One step’s client-facing scalars (sampled tokens), every request in one frame. Replaces one ResultTensors per request for an output whose value the worker already holds on the host: no transport tensor, no ack.

Parameters:
loop_indices: list[NestedLoopIndices | None]#
metadata: dict#
modality: str#
request_ids: list[str]#
signal: str#
values: list[int]#
Contents