mstar.conductor.lora_ops#

Loading and unloading LoRA adapters at runtime, as the conductor sees it.

The API server asks; the conductor forwards the request to every worker hosting a node with the LoRA pool and answers once all of them have acknowledged. A load is answered as done only when every worker loaded it; if any failed, the others are told to unload it again. Which names are served is the API server’s to track (ServedModels, which a request’s adapter is selected from): the conductor keeps no registry of its own, only the operations in flight, during which it refuses requests on their adapters with a 409: each worker finishes an operation in its own time, so the ranks of one lockstep instance could open such a request differently.

A worker that never answers (hung, dead) would leave the name refused with a 409 for good. Past runtime_update_timeout_s an operation is given up: a load is undone on every worker and answered with a 504, an unload (the name already out of the API server’s selection) answered as done.

Classes

LoraAdapterOps(spec, workers, send[, clock])

The conductor's half of /v1/load_lora_adapter and /v1/unload_lora_adapter.

class mstar.conductor.lora_ops.LoraAdapterOps(spec, workers, send, clock=time.perf_counter)[source]#

Bases: object

The conductor’s half of /v1/load_lora_adapter and /v1/unload_lora_adapter.

spec is a LoRA pool’s spec, whose runtime-update settings every pool shares (None for a deployment without LoRA); workers are the workers hosting the pools’ nodes; send(peer, message) is the conductor’s communicator; clock its clock (time.perf_counter, as its other timed checks).

Parameters:
ack(body)[source]#
Parameters:

body (LoraAdapterAck)

Return type:

None

expire()[source]#

Give up on the operations some worker has not answered within runtime_update_timeout_s. A late ack for one is ignored.

Return type:

None

load(body)[source]#
Parameters:

body (LoadLoraAdapter)

Return type:

None

next_deadline()[source]#

When expire next has something to give up on, on clock; None with nothing pending.

Return type:

float | None

refuse_request(configs)[source]#

Refuse (409) a request whose adapter a load or unload is in flight for.

Parameters:

configs (Iterable[ResourceReqConfig])

Return type:

None

unload(body)[source]#

Running requests on the name finish on its weights; the API server took it out of selection before asking.

Parameters:

body (UnloadLoraAdapter)

Return type:

None