mstar.engine.resources.lora.serving#

The serving config’s LoRA adapters, resolved once before the conductor starts.

The API server reads every adapter its resources: lora: block names, refusing the deployment if any cannot be served, and writes what it found back into the block: each adapter’s digest, which every later read must match, and the targets the adapters touch. The conductor is handed that config, and spawns the workers with it, so every process and rank builds the same pools and serves the weights the API server keyed the prefix cache under.

Functions

lora_spec(specs)

The deployment's LoRA spec among its declared resource specs (declared_resources: whole, not yet split into a pool per node).

refuse_unresolved_lora_adapters(model_config)

Refuse a serving config whose lora: block names adapters with no pinned digest: workers spawned with it would serve whatever each of their reads finds, rather than the weights the API server keyed the prefix cache under.

resolve_lora_adapters(spec, model_config)

Check every adapter spec serves (check_adapters: a bad one fails the deployment, every bad one named), write their digests and the targets they touch into model_config's resources: lora: block and onto spec, which then is what every worker builds, and return the digests.

served_lora_targets(spec)

The targets a deployment keeps when its adapters are fixed at startup (runtime updates off, and no YAML targets, which the caller checks): those one of its adapters touches, as the targets list every process then applies.

mstar.engine.resources.lora.serving.lora_spec(specs)[source]#

The deployment’s LoRA spec among its declared resource specs (declared_resources: whole, not yet split into a pool per node).

Parameters:

specs (Iterable[Any])

Return type:

LoraSpec | None

mstar.engine.resources.lora.serving.refuse_unresolved_lora_adapters(model_config)[source]#

Refuse a serving config whose lora: block names adapters with no pinned digest: workers spawned with it would serve whatever each of their reads finds, rather than the weights the API server keyed the prefix cache under.

Parameters:

model_config (Mapping[str, Any])

Return type:

None

mstar.engine.resources.lora.serving.resolve_lora_adapters(spec, model_config)[source]#

Check every adapter spec serves (check_adapters: a bad one fails the deployment, every bad one named), write their digests and the targets they touch into model_config’s resources: lora: block and onto spec, which then is what every worker builds, and return the digests.

Parameters:
Return type:

dict[str, bytes]

mstar.engine.resources.lora.serving.served_lora_targets(spec)[source]#

The targets a deployment keeps when its adapters are fixed at startup (runtime updates off, and no YAML targets, which the caller checks): those one of its adapters touches, as the targets list every process then applies. Every other target would hold zero weights in every slot, yet cost its kernels on every step and its memory (the LM head’s and the embedding’s are vocabulary-sized), and a node left with none gets no pool. None when nothing narrows.

Parameters:

spec (LoraSpec)

Return type:

list[str] | None