mstar.engine.resources.lora.store#

The host tier of the adapter pool: adapters read, sharded for this rank, and pinned.

An adapter’s tensors are laid out exactly like one device slot (rank padded to the pool’s max_rank with zeros), so moving it onto the device is one copy per target. The kernels need that padding finite (ops.py).

Functions

adapter_targets(spec, path)

The targets of spec the checkpoint at path holds LoRA weights for.

as_linear(shape)

A LoRA factor's shape as a linear's: a conv adapter's A [r, c, *kernel] and B [out, r, 1, ...] flatten to [r, c * prod(kernel)] and [out, r], which is the delta on a conv whose kernel is its stride, over the flattened patch.

check_adapter_file(spec, name, path)

The adapter at path checked against spec and its digest, which a worker's read holds the file to; the digest is of the read that was checked.

check_adapters(spec)

Every adapter spec serves, through check_adapter_file; returns each one's digest.

classify_adapter(spec, shapes)

Rank, unmapped and misfitting modules of an adapter, from each module's A and B shapes (both present, as read_lora_file requires).

misfit(spec, module, a_shape, b_shape)

Why a mapped module's A / B shapes do not fit its target, or None.

Classes

LoadedAdapter(name, digest, rank, scaling, ...)

One adapter, ready to copy into a slot.

LoraAdapterStore(spec[, rank, world_size, ...])

Adapters held in pinned host memory.

class mstar.engine.resources.lora.store.LoadedAdapter(name, digest, rank, scaling, a, b, layer_ranks)[source]#

Bases: object

One adapter, ready to copy into a slot.

a[target] is [target layers, slices * max_rank, in] and b[target] is [target layers, out, max_rank], both already this rank’s shard; an experts target’s hold a pair per expert, [target layers, experts, ...]. A target the adapter does not touch is absent. layer_ranks[target] (int32 [target layers]) is the slot’s rank at each layer: rank, or 0 where this rank’s shard of the layer’s A or B is all zero (it adds nothing), as the kernels read it. digest is adapter_digest of the unsharded weights, so every rank agrees on it, computed as the API server computes the one it roots the adapter’s prefix-cache keys in.

Parameters:
a: dict[str, Tensor]#
b: dict[str, Tensor]#
digest: bytes#
layer_ranks: dict[str, Tensor]#
name: str#
rank: int#
scaling: float#
class mstar.engine.resources.lora.store.LoraAdapterStore(spec, rank=0, world_size=1, pin=None, dtype=None)[source]#

Bases: object

Adapters held in pinned host memory.

Which adapters it holds follows from the hold calls alone, in their order: each makes an adapter the most recent, dropping the least recent past capacity but none the call’s keep names. A held adapter’s copy arrives later, by put, once read. So when a read lands or fails never changes what is held, and ranks that make the same calls hold the same set.

Parameters:
checked_rank(name, path, digest)[source]#

path’s rank for this spec, read and held to digest as read does, with nothing staged: all a runtime load needs.

Parameters:
Return type:

int

drop(name)[source]#
Parameters:

name (str)

Return type:

None

get(name)[source]#
Parameters:

name (str)

Return type:

LoadedAdapter | None

hold(name, keep=())[source]#

Make name the most recent adapter held, dropping the least recent past capacity, but none named in keep (which may hold the tier past capacity for a while). True when it was not held, so its copy is the caller’s to read in.

Parameters:
Return type:

bool

load(name, path, digest)[source]#

Hold name and read it from path unless its copy is in.

Parameters:
Return type:

LoadedAdapter

put(adapter)[source]#

adapter’s copy, kept if it is still held.

Parameters:

adapter (LoadedAdapter)

Return type:

bool

read(name, path, digest)[source]#

path read and sharded for this rank, without holding it: a caller can read outside whatever lock guards put. Held to digest as checked_rank is, before anything is staged.

Parameters:
Return type:

LoadedAdapter

mstar.engine.resources.lora.store.adapter_targets(spec, path)[source]#

The targets of spec the checkpoint at path holds LoRA weights for.

Parameters:
Return type:

set[str]

mstar.engine.resources.lora.store.as_linear(shape)[source]#

A LoRA factor’s shape as a linear’s: a conv adapter’s A [r, c, *kernel] and B [out, r, 1, ...] flatten to [r, c * prod(kernel)] and [out, r], which is the delta on a conv whose kernel is its stride, over the flattened patch.

Return type:

tuple[int, …]

mstar.engine.resources.lora.store.check_adapter_file(spec, name, path)[source]#

The adapter at path checked against spec and its digest, which a worker’s read holds the file to; the digest is of the read that was checked. Raises ValueError for an adapter the deployment cannot serve (OSError for a file that is not a LoRA adapter, as read_lora_file refuses it).

Parameters:
Return type:

bytes

mstar.engine.resources.lora.store.check_adapters(spec)[source]#

Every adapter spec serves, through check_adapter_file; returns each one’s digest.

Raises naming every adapter that fails, so a bad adapter in the serving config stops the deployment at startup, the same whether or not a worker would have preloaded it.

Parameters:

spec (LoraSpec)

Return type:

dict[str, bytes]

mstar.engine.resources.lora.store.classify_adapter(spec, shapes)[source]#

Rank, unmapped and misfitting modules of an adapter, from each module’s A and B shapes (both present, as read_lora_file requires).

Parameters:
Return type:

AdapterInfo

mstar.engine.resources.lora.store.misfit(spec, module, a_shape, b_shape)[source]#

Why a mapped module’s A / B shapes do not fit its target, or None.

Parameters:
Return type:

str | None