mstar.model.glm5_next.config

Contents

mstar.model.glm5_next.config#

GLM-5.3-Flash (glm5_next) architecture + generation config.

Functions

build_indexer_types(num_hidden_layers)

Every MLA layer runs its own full indexer; none are shared.

build_layer_types(num_hidden_layers)

The checkpoint's attention schedule: MLA+DSA at idx % 4 == 3.

build_mlp_layer_types(num_hidden_layers, ...)

Dense MLP on the first first_k_dense_replace layers, MoE after.

Classes

Glm5NextModelConfig(vocab_size, hidden_size, ...)

class mstar.model.glm5_next.config.Glm5NextModelConfig(vocab_size: int = 154880, hidden_size: int = 4096, num_hidden_layers: int = 45, num_attention_heads: int = 64, num_key_value_heads: int = 64, rms_norm_eps: float = 1e-05, hidden_act: str = 'silu', tie_word_embeddings: bool = False, layer_types: tuple[str, ...]=<factory>, mlp_layer_types: tuple[str, ...]=<factory>, indexer_types: tuple[str, ...]=<factory>, linear_num_heads: int = 64, linear_head_dim: int = 128, linear_conv_kernel_size: int = 4, gate_lower_bound: float = -5.0, mhc: bool = True, hc_mult: int = 4, hc_sinkhorn_iters: int = 20, hc_eps: float = 1e-06, q_lora_rank: int = 1536, kv_lora_rank: int = 512, qk_nope_head_dim: int = 256, qk_rope_head_dim: int = 0, v_head_dim: int = 256, mla_use_nope: bool = True, mla_absorb: bool = True, index_n_heads: int = 32, index_head_dim: int = 128, index_topk: int = 2048, index_kpool: int = 4, index_kpool_compress: bool = True, index_kpool_always_select_tail: bool = True, index_share_for_mtp_iteration: bool = True, indexer_rope_interleave: bool = True, mtp_num_draft_tokens: int = 0, dsa_long_context: bool = False, first_k_dense_replace: int = 3, intermediate_size: int = 12288, moe_intermediate_size: int = 2048, n_routed_experts: int = 288, n_shared_experts: int = 1, num_experts_per_tok: int = 8, routed_scaling_factor: float = 2.5, scoring_func: str = 'sigmoid', topk_method: str = 'noaux_tc', norm_topk_prob: bool = True, moe_router_dtype: str = 'float32', swiglu_limit: float = 10.0, max_position_embeddings: int = 1048576, max_seq_len: int = 2048, num_nextn_predict_layers: int = 1, eos_token_ids: tuple[int, ...]=(154820, 154827, 154829), pad_token_id: int = 154820, max_output_tokens: int | None = None, quantization_config: mstar.utils.fp8_block.Fp8BlockQuantConfig | None = None, moe_fp8_resident: bool = True, moe_quant_kernel: str = 'reference', prefill_graphs: bool = False, prefill_capture_batch_sizes: list[int] | None = None)[source]#

Bases: object

Parameters:
  • vocab_size (int)

  • hidden_size (int)

  • num_hidden_layers (int)

  • num_attention_heads (int)

  • num_key_value_heads (int)

  • rms_norm_eps (float)

  • hidden_act (str)

  • tie_word_embeddings (bool)

  • layer_types (tuple[str, ...])

  • mlp_layer_types (tuple[str, ...])

  • indexer_types (tuple[str, ...])

  • linear_num_heads (int)

  • linear_head_dim (int)

  • linear_conv_kernel_size (int)

  • gate_lower_bound (float)

  • mhc (bool)

  • hc_mult (int)

  • hc_sinkhorn_iters (int)

  • hc_eps (float)

  • q_lora_rank (int)

  • kv_lora_rank (int)

  • qk_nope_head_dim (int)

  • qk_rope_head_dim (int)

  • v_head_dim (int)

  • mla_use_nope (bool)

  • mla_absorb (bool)

  • index_n_heads (int)

  • index_head_dim (int)

  • index_topk (int)

  • index_kpool (int)

  • index_kpool_compress (bool)

  • index_kpool_always_select_tail (bool)

  • index_share_for_mtp_iteration (bool)

  • indexer_rope_interleave (bool)

  • mtp_num_draft_tokens (int)

  • dsa_long_context (bool)

  • first_k_dense_replace (int)

  • intermediate_size (int)

  • moe_intermediate_size (int)

  • n_routed_experts (int)

  • n_shared_experts (int)

  • num_experts_per_tok (int)

  • routed_scaling_factor (float)

  • scoring_func (str)

  • topk_method (str)

  • norm_topk_prob (bool)

  • moe_router_dtype (str)

  • swiglu_limit (float)

  • max_position_embeddings (int)

  • max_seq_len (int)

  • num_nextn_predict_layers (int)

  • eos_token_ids (tuple[int, ...])

  • pad_token_id (int)

  • max_output_tokens (int | None)

  • quantization_config (Fp8BlockQuantConfig | None)

  • moe_fp8_resident (bool)

  • moe_quant_kernel (str)

  • prefill_graphs (bool)

  • prefill_capture_batch_sizes (list[int] | None)

cache_latent_dim: int#
dsa_long_context: bool = False#
eos_token_ids: tuple[int, ...] = (154820, 154827, 154829)#
first_k_dense_replace: int = 3#
classmethod from_hf_config(hf_config, strict=True)[source]#

Build from the checkpoint’s config.json dict.

Parameters:
Return type:

Glm5NextModelConfig

property full_attn_layer_indices: tuple[int, ...]#
gate_lower_bound: float = -5.0#
hc_eps: float = 1e-06#
hc_mult: int = 4#
hc_sinkhorn_iters: int = 20#
hidden_act: str = 'silu'#
hidden_size: int = 4096#
index_head_dim: int = 128#
index_kpool: int = 4#
index_kpool_always_select_tail: bool = True#
index_kpool_compress: bool = True#
index_n_heads: int = 32#
index_share_for_mtp_iteration: bool = True#
index_topk: int = 2048#
indexer_rope_interleave: bool = True#
indexer_types: tuple[str, ...]#
intermediate_size: int = 12288#
is_dense_mlp_layer(layer_idx)[source]#
Parameters:

layer_idx (int)

Return type:

bool

is_full_attention_layer(layer_idx)[source]#

True for MLA+DSA layers; the MTP layer (== num_hidden_layers) always is (it ships its own full indexer).

Parameters:

layer_idx (int)

Return type:

bool

kda_config()[source]#

The KDA layers’ KDAConfig, all heads (the layer divides them by TP).

property kda_layer_indices: tuple[int, ...]#
kv_lora_rank: int = 512#
property kv_rows: int#

Cache rows a request may hold. Its real context is held to max_seq_len; under MTP each verify step also leaves its rejected drafts as holes. Admission books every step’s whole block (no draft accepted), so k + 1 rows a token keep the rows from refusing a request its context holds. Pages are taken as the rows fill, so the bound allocates nothing.

layer_types: tuple[str, ...]#
property linear_conv_channels: int#

3 x qkv_dim.

Type:

Fused depthwise conv channels over cat(q, k, v)

linear_conv_kernel_size: int = 4#
linear_head_dim: int = 128#
linear_num_heads: int = 64#
property linear_qkv_dim: int#

heads x head_dim (8192 full-size).

Type:

Per-projection KDA width

max_output_tokens: int | None = None#
max_position_embeddings: int = 1048576#
max_seq_len: int = 2048#
mhc: bool = True#
mla_absorb: bool = True#
property mla_cache_kpe: int#

Rope-slot width of the MLA latent cache. NoPE needs none; MTP keeps a 64-wide slot to mask rejected draft rows.

mla_use_nope: bool = True#
mlp_layer_types: tuple[str, ...]#
moe_fp8_resident: bool = True#
moe_intermediate_size: int = 2048#
moe_quant_kernel: str = 'reference'#
moe_router_dtype: str = 'float32'#
mtp_num_draft_tokens: int = 0#
n_routed_experts: int = 288#
n_shared_experts: int = 1#
norm_topk_prob: bool = True#
num_attention_heads: int = 64#
property num_dense_layers: int#
num_experts_per_tok: int = 8#
num_hidden_layers: int = 45#
num_key_value_heads: int = 64#
num_nextn_predict_layers: int = 1#
pad_token_id: int = 154820#
property padded_head_dim: int#

qk_head_dim rounded up to a FlashInfer paged-kernel head dim (64, 128 or 256).

prefill_capture_batch_sizes: list[int] | None = None#
prefill_graphs: bool = False#
q_lora_rank: int = 1536#
property qk_head_dim: int#
qk_nope_head_dim: int = 256#
qk_rope_head_dim: int = 0#
quantization_config: Fp8BlockQuantConfig | None = None#
classmethod reduced()[source]#

Tiny-dim variant for CPU tests: same shapes family, random weights.

Return type:

Glm5NextModelConfig

classmethod reduced_fp8(block=(16, 16))[source]#
Parameters:

block (tuple[int, int])

Return type:

Glm5NextModelConfig

rms_norm_eps: float = 1e-05#
routed_scaling_factor: float = 2.5#
scoring_func: str = 'sigmoid'#
property softmax_scale: float#
swiglu_limit: float = 10.0#
tie_word_embeddings: bool = False#
topk_method: str = 'noaux_tc'#
v_head_dim: int = 256#
vocab_size: int = 154880#
mstar.model.glm5_next.config.build_indexer_types(num_hidden_layers)[source]#

Every MLA layer runs its own full indexer; none are shared.

Parameters:

num_hidden_layers (int)

Return type:

tuple[str, …]

mstar.model.glm5_next.config.build_layer_types(num_hidden_layers)[source]#

The checkpoint’s attention schedule: MLA+DSA at idx % 4 == 3.

Parameters:

num_hidden_layers (int)

Return type:

tuple[str, …]

mstar.model.glm5_next.config.build_mlp_layer_types(num_hidden_layers, first_k_dense_replace)[source]#

Dense MLP on the first first_k_dense_replace layers, MoE after.

Parameters:
  • num_hidden_layers (int)

  • first_k_dense_replace (int)

Return type:

tuple[str, …]