mstar.engine.resources.step

mstar.engine.resources.step#

Classes

AdmitFailedReason(message)

AdmitOutcome(ok, ready, reason)

AdmitRuntimeError(message)

A resource cannot serve this request at all.

AllocationFailed(message, pages_short, ...)

BucketKey(graph_walk, bs, num_tokens[, ...])

FullAdmitOutcome(outcome[, failed_resource])

What the runner answers with: one resource's outcome, plus which resource gave it.

RequestOffloading(message, label, request_id)

The request's state is moving to host memory; retry once it is back.

ResourceStep([segments])

Segment(request_id, label, span)

One step's addition to a request's cache stream.

SlotLease(slot, bucket)

attention admit gives to inform which slot to plan and replay

StepContext(request_ids, graph_walk, slot, ...)

SubmoduleStep(steps[, segments, ...])

class mstar.engine.resources.step.AdmitFailedReason(message: str)[source]#

Bases: object

Parameters:

message (str)

message: str#
class mstar.engine.resources.step.AdmitOutcome(ok, ready, reason)[source]#

Bases: NamedTuple

Parameters:
ok: bool#

Alias for field number 0

ready: bool#

Alias for field number 1

reason: AdmitFailedReason | None#

Alias for field number 2

class mstar.engine.resources.step.AdmitRuntimeError(message)[source]#

Bases: AdmitFailedReason

A resource cannot serve this request at all.

Terminal, unlike the two above: no eviction and no reload makes it go away, so the caller’s answer is to fail the request, not to retry it.

Parameters:

message (str)

class mstar.engine.resources.step.AllocationFailed(message: str, pages_short: int, label: str, request_id: str)[source]#

Bases: AdmitFailedReason

Parameters:

  • message (str)

  • pages_short (int)

  • label (str)

  • request_id (str)

label: str#
pages_short: int#
request_id: str#
class mstar.engine.resources.step.BucketKey(graph_walk: str, bs: int, num_tokens: int, cg_key_info: Any | None = None)[source]#

Bases: object

Parameters:

  • graph_walk (str)

  • bs (int)

  • num_tokens (int)

  • cg_key_info (Any | None)

bs: int#
cg_key_info: Any | None = None#
graph_walk: str#
num_tokens: int#
class mstar.engine.resources.step.FullAdmitOutcome(outcome, failed_resource=None)[source]#

Bases: NamedTuple

What the runner answers with: one resource’s outcome, plus which resource gave it.

A resource doesn’t know the key it is registered under, so the runner — which does — names it on the way out. The caller needs it to scope an eviction to the resource that actually ran out.

Parameters:
failed_resource: str | None#

Alias for field number 1

property ok: bool#
outcome: AdmitOutcome#

Alias for field number 0

property ready: bool#
property reason: AdmitFailedReason | None#
class mstar.engine.resources.step.RequestOffloading(message, label, request_id)[source]#

Bases: AdmitFailedReason

The request’s state is moving to host memory; retry once it is back.

Distinct from AllocationFailed because the answer is different: nothing needs evicting, the caller just re-drives the step once reload has run.

Parameters:
label: str#
request_id: str#
class mstar.engine.resources.step.ResourceStep(segments: tuple[mstar.engine.resources.step.Segment, ...] | None = None)[source]#

Bases: object

Parameters:

segments (tuple[Segment, ...] | None)

segments: tuple[Segment, ...] | None = None#
class mstar.engine.resources.step.Segment(request_id, label, span)[source]#

Bases: NamedTuple

One step’s addition to a request’s cache stream.

A request contributes one segment per label active for it in a step; the batch’s ordered segment list defines the layout of per-token arrays. span may be 0: a zero-span segment reads its stream without extending it (admission reserves nothing, commit is a no-op).

A NamedTuple, not a frozen dataclass: one is built per request per step, and the frozen dataclass’s object.__setattr__-per-field __init__ is the expensive way to do that.

Parameters:
label: str#

Alias for field number 1

request_id: str#

Alias for field number 0

span: int#

Alias for field number 2

class mstar.engine.resources.step.SlotLease(slot, bucket)[source]#

Bases: object

attention admit gives to inform which slot to plan and replay

has no clean channel to plan/commit/release. see O.

Parameters:
bucket: BucketKey | None#
slot: int#
class mstar.engine.resources.step.StepContext(request_ids: collections.abc.Sequence[str], graph_walk: str, slot: int, capture: bool, is_preplan: bool = False, plan_results: dict[str, typing.Any] = <factory>, slot_lease: mstar.engine.resources.step.SlotLease | None = None, piecewise_leases: 'Mapping[str, SlotLease]' = <factory>, _padded_request_ids: collections.abc.Sequence[str] | None = None)[source]#

Bases: object

Parameters:
capture: bool#
graph_walk: str#
is_preplan: bool = False#
property padded_request_ids: Sequence[str]#
piecewise_leases: Mapping[str, SlotLease]#
plan_results: dict[str, Any]#
request_ids: Sequence[str]#
set_padded_rids(padded_rids)[source]#
Parameters:

padded_rids (Sequence[str] | None)

set_piecewise_leases(leases)[source]#
Parameters:

leases (Mapping[str, SlotLease])

slot: int#
slot_lease: SlotLease | None = None#
class mstar.engine.resources.step.SubmoduleStep(steps: dict[str, mstar.engine.resources.step.ResourceStep], segments: list[mstar.engine.resources.step.Segment] | None = None, cg_key_info: Any | None = None, _ctx: mstar.engine.resources.step.StepContext = None)[source]#

Bases: object

Parameters:
cg_key_info: Any | None = None#
property ctx#
get(key)[source]#
Parameters:

key (str)

Return type:

ResourceStep | None

keys()[source]#
Return type:

KeysView[str]

segments: list[Segment] | None = None#
set_ctx(ctx)[source]#
steps: dict[str, ResourceStep]#