mstar.worker.lora_loads#

Runtime LoRA adapter loads and unloads on a worker: a load reads in the background and is acknowledged from the main loop.

A load reads, converts, shards, pins and digests a whole adapter file. On the main loop it would stall scheduling for as long as that takes, so each pool reads on its own reader thread and the main loop acknowledges the load once every pool has it.

Functions

lora_pools(resources)

The LoRA adapter pools among an engine's resources.

Classes

LoraLoad(body[, reads, error, behind])

One load in flight: each pool's read, and what to answer once they are all done.

LoraLoads(services)

The worker's LOAD_LORA_ADAPTER and UNLOAD_LORA_ADAPTER messages, and the loads still being read.

class mstar.worker.lora_loads.LoraLoad(body, reads=<factory>, error=None, behind=<factory>)[source]#

Bases: object

One load in flight: each pool’s read, and what to answer once they are all done.

Parameters:
behind: list[LoadLoraAdapter | UnloadLoraAdapter]#
body: LoadLoraAdapter#
error: str | None#
reads: list[tuple['LoraAdapterPool', Future]]#
class mstar.worker.lora_loads.LoraLoads(services)[source]#

Bases: object

The worker’s LOAD_LORA_ADAPTER and UNLOAD_LORA_ADAPTER messages, and the loads still being read.

Each is applied as the message arrives: a load or an unload touches no slot state an admit decides from (LoraAdapterPool’s docstring), so the ranks of a lockstep instance need not apply it at the same step. A load or an unload of a name still being read waits for that load’s ack.

Parameters:

services (WorkerServices)

finish()[source]#

Acknowledge the loads whose reads completed, then apply the loads and unloads that arrived behind them, in order.

Return type:

None

load(body)[source]#

Start reading a runtime-loaded adapter into this rank’s host tier; it is acknowledged once every pool has it (finish). A pool that fails rolls back the ones that loaded, and the conductor rolls back the other workers.

Parameters:

body (LoadLoraAdapter)

Return type:

None

unload(body)[source]#

Stop serving an adapter to new requests; running ones keep it until they end.

Parameters:

body (UnloadLoraAdapter)

Return type:

None

mstar.worker.lora_loads.lora_pools(resources)[source]#

The LoRA adapter pools among an engine’s resources.

Parameters:

resources (Mapping[str, 'Resource'])

Return type:

list[‘LoraAdapterPool’]