mstar.engine.resources.attn.ragged.base

mstar.engine.resources.attn.ragged.base#

The cacheless attention resource: the spec-time factory.

Sibling of attn.base’s AttentionManager, for attention that is not backed by a KV cache, e.g., an encoder tower attending within variable-length segments of one packed forward. There is nothing to page, nothing to advance, and nothing to carry between steps: the resource holds only planned wrappers.

Functions

resolve_ragged_attn_backend(backend, device)

Ragged counterpart of attn.base.resolve_attn_backend.

Classes

RaggedAttnManager()

RaggedBlockCausalAttnManager()

Cacheless block-causal self-attention within each span; see RaggedBlockCausalAttentionSpec.

RaggedCrossAttnManager()

Cacheless cross-attention between two spans of each request; see RaggedCrossAttentionSpec.
class mstar.engine.resources.attn.ragged.base.RaggedAttnManager[source]#

Bases: AttentionResource

classmethod build(spec, info)[source]#
Parameters:
prefix_skip_safe: bool = True#
class mstar.engine.resources.attn.ragged.base.RaggedBlockCausalAttnManager[source]#

Bases: AttentionResource

Cacheless block-causal self-attention within each span; see RaggedBlockCausalAttentionSpec.

classmethod build(spec, info)[source]#
Parameters:
prefix_skip_safe: bool = True#
class mstar.engine.resources.attn.ragged.base.RaggedCrossAttnManager[source]#

Bases: AttentionResource

Cacheless cross-attention between two spans of each request; see RaggedCrossAttentionSpec.

classmethod build(spec, info)[source]#
Parameters:
prefix_skip_safe: bool = True#
mstar.engine.resources.attn.ragged.base.resolve_ragged_attn_backend(backend, device)[source]#

Ragged counterpart of attn.base.resolve_attn_backend.

Parameters:
Return type:

RaggedAttnBackend