mstar.observability.catalog#
Metric families whose values come from a process other than the API server.
Dependency-free, so the conductor and workers can import it: they record into
mstar.observability.recorders by these names, and the API server builds
the Prometheus families from the same entries, so the names cannot drift.
model_name is added by the API server and is not listed in labels.
Module Attributes
How often the conductor sends its status message while metrics are on. |
|
queued at |
|
the data worker (a thread of the API server) sends the request's inputs and reads its results. |
|
Families the engine workers report to the conductor ( |
|
Families the conductor's status message carries, exported from the latest message. |
Classes
|
- mstar.observability.catalog.CONDUCTOR_REQUEST_STATES = ('waiting', 'running', 'draining')#
queued at
max_concurrent_requests, not dispatched yet.running: dispatched to workers and not finished.draining: finished (completed, aborted or failed) and waiting for every reader to release its outputs before the workers free its state; it still counts towardmax_concurrent_requests.- Type:
waiting
- class mstar.observability.catalog.Family(name: 'str', kind: 'str', help: 'str', labels: 'tuple[str, ...]', buckets: 'tuple[float, ...]' = ())[source]#
Bases:
object
- mstar.observability.catalog.STATUS_FAMILIES: dict[str, Family] = {'mstar_batch_size': Family(name='mstar_batch_size', kind='histogram', help="Engine steps: rows the step's forward ran.", labels=('component', 'graph_walk'), buckets=(1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 64.0, 128.0, 256.0, 512.0, 1024.0)), 'mstar_component_in_flight': Family(name='mstar_component_in_flight', kind='gauge', help='Running requests with a pass on this graph node that the conductor dispatched and whose completion it has not received yet.', labels=('component',), buckets=()), 'mstar_component_pending_steps': Family(name='mstar_component_pending_steps', kind='gauge', help='Requests whose next step on this graph node is ready (its inputs arrived) but not in a running step: waiting for a batch slot, or for the worker to schedule the node.', labels=('component',), buckets=()), 'mstar_conductor_requests': Family(name='mstar_conductor_requests', kind='gauge', help='Requests the conductor holds, by state.', labels=('state',), buckets=()), 'mstar_kv_pages_cached': Family(name='mstar_kv_pages_cached', kind='gauge', help='Pages of a paged KV cache its prefix cache names, including pages running requests also hold; the cache frees the others on demand.', labels=('component', 'resource'), buckets=()), 'mstar_kv_pages_total': Family(name='mstar_kv_pages_total', kind='gauge', help='Pages of a paged KV cache that requests can hold.', labels=('component', 'resource'), buckets=()), 'mstar_kv_pages_used': Family(name='mstar_kv_pages_used', kind='gauge', help='Pages of a paged KV cache not on its free list: held by requests, or kept by the prefix cache (see mstar_kv_pages_cached).', labels=('component', 'resource'), buckets=()), 'mstar_prefix_cache_hit_tokens_total': Family(name='mstar_prefix_cache_hit_tokens_total', kind='counter', help='Prompt tokens the prefix cache already held at that lookup.', labels=('component',), buckets=()), 'mstar_prefix_cache_query_tokens_total': Family(name='mstar_prefix_cache_query_tokens_total', kind='counter', help='Prompt tokens looked up in the prefix cache, once per request.', labels=('component',), buckets=()), 'mstar_step_seconds': Family(name='mstar_step_seconds', kind='histogram', help="Engine steps: from the later of the step's hand-off to the GPU thread and the end of the worker's previous step, to the end of this step's postprocess. In steady state, the interval between step completions.", labels=('component', 'graph_walk'), buckets=(0.0005, 0.001, 0.002, 0.003, 0.005, 0.0075, 0.01, 0.015, 0.02, 0.03, 0.05, 0.075, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0)), 'mstar_transfer_bytes_total': Family(name='mstar_transfer_bytes_total', kind='counter', help='Tensor bytes the consumer read from the producer over this edge.', labels=('src', 'dst', 'transport'), buckets=()), 'mstar_transfer_seconds': Family(name='mstar_transfer_seconds', kind='histogram', help="Consumer-side time to read one edge's tensors from the producer (one observation per edge read; the producer's own write or registration is not included).", labels=('src', 'dst', 'transport'), buckets=(1e-05, 2.5e-05, 5e-05, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0)), 'mstar_transfers_total': Family(name='mstar_transfers_total', kind='counter', help='Tensors the consumer read from the producer over this edge.', labels=('src', 'dst', 'transport'), buckets=())}#
Families the conductor’s status message carries, exported from the latest message.
- mstar.observability.catalog.STATUS_INTERVAL_S = 1.0#
How often the conductor sends its status message while metrics are on.
- mstar.observability.catalog.TRANSFER_API_SERVER = 'api_server'#
the data worker (a thread of the API server) sends the request’s inputs and reads its results. Every other endpoint is a graph node name.
- Type:
src/dstof a tensor edge into or out of the graph
- mstar.observability.catalog.WORKER_FAMILIES: dict[str, Family] = {'mstar_batch_size': Family(name='mstar_batch_size', kind='histogram', help="Engine steps: rows the step's forward ran.", labels=('component', 'graph_walk'), buckets=(1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 64.0, 128.0, 256.0, 512.0, 1024.0)), 'mstar_component_pending_steps': Family(name='mstar_component_pending_steps', kind='gauge', help='Requests whose next step on this graph node is ready (its inputs arrived) but not in a running step: waiting for a batch slot, or for the worker to schedule the node.', labels=('component',), buckets=()), 'mstar_kv_pages_cached': Family(name='mstar_kv_pages_cached', kind='gauge', help='Pages of a paged KV cache its prefix cache names, including pages running requests also hold; the cache frees the others on demand.', labels=('component', 'resource'), buckets=()), 'mstar_kv_pages_total': Family(name='mstar_kv_pages_total', kind='gauge', help='Pages of a paged KV cache that requests can hold.', labels=('component', 'resource'), buckets=()), 'mstar_kv_pages_used': Family(name='mstar_kv_pages_used', kind='gauge', help='Pages of a paged KV cache not on its free list: held by requests, or kept by the prefix cache (see mstar_kv_pages_cached).', labels=('component', 'resource'), buckets=()), 'mstar_prefix_cache_hit_tokens_total': Family(name='mstar_prefix_cache_hit_tokens_total', kind='counter', help='Prompt tokens the prefix cache already held at that lookup.', labels=('component',), buckets=()), 'mstar_prefix_cache_query_tokens_total': Family(name='mstar_prefix_cache_query_tokens_total', kind='counter', help='Prompt tokens looked up in the prefix cache, once per request.', labels=('component',), buckets=()), 'mstar_step_seconds': Family(name='mstar_step_seconds', kind='histogram', help="Engine steps: from the later of the step's hand-off to the GPU thread and the end of the worker's previous step, to the end of this step's postprocess. In steady state, the interval between step completions.", labels=('component', 'graph_walk'), buckets=(0.0005, 0.001, 0.002, 0.003, 0.005, 0.0075, 0.01, 0.015, 0.02, 0.03, 0.05, 0.075, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0, 30.0)), 'mstar_transfer_bytes_total': Family(name='mstar_transfer_bytes_total', kind='counter', help='Tensor bytes the consumer read from the producer over this edge.', labels=('src', 'dst', 'transport'), buckets=()), 'mstar_transfer_seconds': Family(name='mstar_transfer_seconds', kind='histogram', help="Consumer-side time to read one edge's tensors from the producer (one observation per edge read; the producer's own write or registration is not included).", labels=('src', 'dst', 'transport'), buckets=(1e-05, 2.5e-05, 5e-05, 0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0)), 'mstar_transfers_total': Family(name='mstar_transfers_total', kind='counter', help='Tensors the consumer read from the producer over this edge.', labels=('src', 'dst', 'transport'), buckets=())}#
Families the engine workers report to the conductor (
WorkerStatus); the conductor sums the workers’ series and forwards them in its status message.