mstar.engine.resources.lora.store#
The host tier of the adapter pool: adapters read, sharded for this rank, and pinned.
An adapter’s tensors are laid out exactly like one device slot (rank padded to the pool’s
max_rank with zeros), so moving it onto the device is one copy per target. The kernels
need that padding finite (ops.py).
Functions
|
The targets of |
|
A LoRA factor's shape as a linear's: a conv adapter's |
|
The adapter at |
|
Every adapter |
|
Rank, unmapped and misfitting modules of an adapter, from each module's A and B shapes (both present, as |
|
Why a mapped module's |
Classes
|
One adapter, ready to copy into a slot. |
|
Adapters held in pinned host memory. |
- class mstar.engine.resources.lora.store.LoadedAdapter(name, digest, rank, scaling, a, b, layer_ranks)[source]#
Bases:
objectOne adapter, ready to copy into a slot.
a[target]is[target layers, slices * max_rank, in]andb[target]is[target layers, out, max_rank], both already this rank’s shard; an experts target’s hold a pair per expert,[target layers, experts, ...]. A target the adapter does not touch is absent.layer_ranks[target](int32[target layers]) is the slot’s rank at each layer:rank, or 0 where this rank’s shard of the layer’s A or B is all zero (it adds nothing), as the kernels read it.digestisadapter_digestof the unsharded weights, so every rank agrees on it, computed as the API server computes the one it roots the adapter’s prefix-cache keys in.- Parameters:
- class mstar.engine.resources.lora.store.LoraAdapterStore(spec, rank=0, world_size=1, pin=None, dtype=None)[source]#
Bases:
objectAdapters held in pinned host memory.
Which adapters it holds follows from the
holdcalls alone, in their order: each makes an adapter the most recent, dropping the least recent pastcapacitybut none the call’skeepnames. A held adapter’s copy arrives later, byput, once read. So when a read lands or fails never changes what is held, and ranks that make the same calls hold the same set.- Parameters:
spec (LoraSpec)
rank (int)
world_size (int)
pin (bool | None)
dtype (torch.dtype | None)
- checked_rank(name, path, digest)[source]#
path’s rank for this spec, read and held todigestasreaddoes, with nothing staged: all a runtime load needs.
- get(name)[source]#
- Parameters:
name (str)
- Return type:
LoadedAdapter | None
- hold(name, keep=())[source]#
Make
namethe most recent adapter held, dropping the least recent pastcapacity, but none named inkeep(which may hold the tier past capacity for a while). True when it was not held, so its copy is the caller’s to read in.- Parameters:
name (str)
keep (Collection[str])
- Return type:
- load(name, path, digest)[source]#
Hold
nameand read it frompathunless its copy is in.- Parameters:
- Return type:
- put(adapter)[source]#
adapter’s copy, kept if it is still held.- Parameters:
adapter (LoadedAdapter)
- Return type:
- mstar.engine.resources.lora.store.adapter_targets(spec, path)[source]#
The targets of
specthe checkpoint atpathholds LoRA weights for.
- mstar.engine.resources.lora.store.as_linear(shape)[source]#
A LoRA factor’s shape as a linear’s: a conv adapter’s
A [r, c, *kernel]andB [out, r, 1, ...]flatten to[r, c * prod(kernel)]and[out, r], which is the delta on a conv whose kernel is its stride, over the flattened patch.
- mstar.engine.resources.lora.store.check_adapter_file(spec, name, path)[source]#
The adapter at
pathchecked againstspecand its digest, which a worker’s read holds the file to; the digest is of the read that was checked. RaisesValueErrorfor an adapter the deployment cannot serve (OSErrorfor a file that is not a LoRA adapter, asread_lora_filerefuses it).
- mstar.engine.resources.lora.store.check_adapters(spec)[source]#
Every adapter
specserves, throughcheck_adapter_file; returns each one’s digest.Raises naming every adapter that fails, so a bad adapter in the serving config stops the deployment at startup, the same whether or not a worker would have preloaded it.