mstar.observability.metrics

mstar.observability.metrics#

Prometheus metrics for the API server: --enable-metrics and GET /metrics.

Requests are counted where they leave APIServer, which /generate, /generate/ws, the OpenAI routes, the Rust-frontend bridge and the Dynamo bridge all go through. Requests the HTTP frontend refuses before that (a missing API key, a body over the limit, a bad form) are counted by route and status in mstar_http_refusals_total. Each family carries the deployment’s model_name; every other label takes values from a fixed set.

Only the API server process imports prometheus_client and renders /metrics. Other processes (data worker, conductor, workers) send plain numbers on messages they already send the API server, plus the conductor’s periodic status message, and the API server records them here; no multiprocess directory, which would only aggregate one host. Families whose values come from another process are defined in catalog.

prometheus_client is optional (pip install 'mstar[metrics]') and only needed when metrics are on.

Module Attributes

OUTCOMES

the request failed (engine, preprocessing, result delivery, conductor gone).

Functions

mark_submitted()

The HTTP request being served reached APIServer: it is counted in mstar_requests_total when it ends, so it is never a refusal.

modalities_label(modalities)

One label value per set of input or output modalities, e.g. audio,text.

require_prometheus_client()

route_template(scope, routes)

The route path the request matched (/v1/chat/completions), never the raw URL; unmatched when no route matches.

Classes

RefusalMiddleware(app, metrics, routes)

Counts mstar_http_refusals_total for the Python frontend: every HTTP response with a 4xx status for a request that never reached APIServer.submit_request (mark_submitted()).

ServingMetrics(model_name, in_flight, ...[, ...])

The API server's registry and its request metrics.
mstar.observability.metrics.OUTCOMES = ('success', 'error', 'abort', 'timeout')#

the request failed (engine, preprocessing, result delivery, conductor gone). abort: the client went away, or its finished result was dropped unread. timeout: the per-request --timeout expired.

Type:

success

Type:

the result went to the HTTP layer. error

class mstar.observability.metrics.RefusalMiddleware(app, metrics, routes)[source]#

Bases: object

Counts mstar_http_refusals_total for the Python frontend: every HTTP response with a 4xx status for a request that never reached APIServer.submit_request (mark_submitted()). A request that was submitted and then failed is counted by mstar_requests_total instead.

Sits outside the API key and body limit middlewares (their 401 and 413 are sent before routing) and inside CORS (a preflight is not a request). WebSocket handshakes are not counted. With metrics off it only checks metrics() for None.

Parameters:

  • metrics (Callable[[], ServingMetrics | None])

  • routes (Callable[[], Iterable])

class mstar.observability.metrics.ServingMetrics(model_name, in_flight, data_worker_queue_depth, local_status=None)[source]#

Bases: object

The API server’s registry and its request metrics.

Parameters:

  • model_name (str)

  • in_flight (Callable[[], int])

  • data_worker_queue_depth (Callable[[], int])

  • local_status (Callable[[], list[MetricSnapshot]] | None)

conductor_status(status)[source]#

Export the conductor’s latest status message.

Parameters:

status (ConductorStatus)

Return type:

None

record(req, outcome)[source]#

Count one request that left the API server’s table.

Parameters:

  • req (PendingRequest)

  • outcome (str)

Return type:

None

record_chunk(modality, data, metadata)[source]#

Count one result chunk that reached the API server (not an error chunk).

Parameters:
Return type:

None

record_preprocess(profile)[source]#

Record the data worker’s numbers for one request it took off its queue: its time if preprocessing finished, else a failure, and the size of each input file either way.

Parameters:

profile (DataWorkerProfile)

Return type:

None

render()[source]#
Return type:

bytes

mstar.observability.metrics.mark_submitted()[source]#

The HTTP request being served reached APIServer: it is counted in mstar_requests_total when it ends, so it is never a refusal.

Return type:

None

mstar.observability.metrics.modalities_label(modalities)[source]#

One label value per set of input or output modalities, e.g. audio,text.

Parameters:

modalities (list[str])

Return type:

str

mstar.observability.metrics.require_prometheus_client()[source]#
Return type:

None

mstar.observability.metrics.route_template(scope, routes)[source]#

The route path the request matched (/v1/chat/completions), never the raw URL; unmatched when no route matches. routes are the app’s routes, searched for a request refused before routing (the API key, a declared body size) the way the router matches: a full match before a method-only one.

Parameters:
Return type:

str