mstar.engine.resources.lora.serving#
The serving config’s LoRA adapters, resolved once before the conductor starts.
The API server reads every adapter its resources: lora: block names, refusing the
deployment if any cannot be served, and writes what it found back into the block: each
adapter’s digest, which every later read must match, and the targets the adapters touch.
The conductor is handed that config, and spawns the workers with it, so every process and
rank builds the same pools and serves the weights the API server keyed the prefix cache
under.
Functions
|
The deployment's LoRA spec among its declared resource specs ( |
|
Refuse a serving config whose |
|
Check every adapter |
|
The targets a deployment keeps when its adapters are fixed at startup (runtime updates off, and no YAML |
- mstar.engine.resources.lora.serving.lora_spec(specs)[source]#
The deployment’s LoRA spec among its declared resource specs (
declared_resources: whole, not yet split into a pool per node).
- mstar.engine.resources.lora.serving.refuse_unresolved_lora_adapters(model_config)[source]#
Refuse a serving config whose
lora:block names adapters with no pinned digest: workers spawned with it would serve whatever each of their reads finds, rather than the weights the API server keyed the prefix cache under.
- mstar.engine.resources.lora.serving.resolve_lora_adapters(spec, model_config)[source]#
Check every adapter
specserves (check_adapters: a bad one fails the deployment, every bad one named), write their digests and the targets they touch intomodel_config’sresources: lora:block and ontospec, which then is what every worker builds, and return the digests.
- mstar.engine.resources.lora.serving.served_lora_targets(spec)[source]#
The targets a deployment keeps when its adapters are fixed at startup (runtime updates off, and no YAML
targets, which the caller checks): those one of its adapters touches, as thetargetslist every process then applies. Every other target would hold zero weights in every slot, yet cost its kernels on every step and its memory (the LM head’s and the embedding’s are vocabulary-sized), and a node left with none gets no pool. None when nothing narrows.