mstar.model.glm5_next.config#
GLM-5.3-Flash (glm5_next) architecture + generation config.
Functions
|
Every MLA layer runs its own full indexer; none are shared. |
|
The checkpoint's attention schedule: MLA+DSA at idx % 4 == 3. |
|
Dense MLP on the first |
Classes
|
- class mstar.model.glm5_next.config.Glm5NextModelConfig(vocab_size: int = 154880, hidden_size: int = 4096, num_hidden_layers: int = 45, num_attention_heads: int = 64, num_key_value_heads: int = 64, rms_norm_eps: float = 1e-05, hidden_act: str = 'silu', tie_word_embeddings: bool = False, layer_types: tuple[str, ...]=<factory>, mlp_layer_types: tuple[str, ...]=<factory>, indexer_types: tuple[str, ...]=<factory>, linear_num_heads: int = 64, linear_head_dim: int = 128, linear_conv_kernel_size: int = 4, gate_lower_bound: float = -5.0, mhc: bool = True, hc_mult: int = 4, hc_sinkhorn_iters: int = 20, hc_eps: float = 1e-06, q_lora_rank: int = 1536, kv_lora_rank: int = 512, qk_nope_head_dim: int = 256, qk_rope_head_dim: int = 0, v_head_dim: int = 256, mla_use_nope: bool = True, mla_absorb: bool = True, index_n_heads: int = 32, index_head_dim: int = 128, index_topk: int = 2048, index_kpool: int = 4, index_kpool_compress: bool = True, index_kpool_always_select_tail: bool = True, index_share_for_mtp_iteration: bool = True, indexer_rope_interleave: bool = True, mtp_num_draft_tokens: int = 0, dsa_long_context: bool = False, first_k_dense_replace: int = 3, intermediate_size: int = 12288, moe_intermediate_size: int = 2048, n_routed_experts: int = 288, n_shared_experts: int = 1, num_experts_per_tok: int = 8, routed_scaling_factor: float = 2.5, scoring_func: str = 'sigmoid', topk_method: str = 'noaux_tc', norm_topk_prob: bool = True, moe_router_dtype: str = 'float32', swiglu_limit: float = 10.0, max_position_embeddings: int = 1048576, max_seq_len: int = 2048, num_nextn_predict_layers: int = 1, eos_token_ids: tuple[int, ...]=(154820, 154827, 154829), pad_token_id: int = 154820, max_output_tokens: int | None = None, quantization_config: mstar.utils.fp8_block.Fp8BlockQuantConfig | None = None, moe_fp8_resident: bool = True, moe_quant_kernel: str = 'reference', prefill_graphs: bool = False, prefill_capture_batch_sizes: list[int] | None = None)[source]#
Bases:
object- Parameters:
vocab_size (int)
hidden_size (int)
num_hidden_layers (int)
num_attention_heads (int)
num_key_value_heads (int)
rms_norm_eps (float)
hidden_act (str)
tie_word_embeddings (bool)
linear_num_heads (int)
linear_head_dim (int)
linear_conv_kernel_size (int)
gate_lower_bound (float)
mhc (bool)
hc_mult (int)
hc_sinkhorn_iters (int)
hc_eps (float)
q_lora_rank (int)
kv_lora_rank (int)
qk_nope_head_dim (int)
qk_rope_head_dim (int)
v_head_dim (int)
mla_use_nope (bool)
mla_absorb (bool)
index_n_heads (int)
index_head_dim (int)
index_topk (int)
index_kpool (int)
index_kpool_compress (bool)
index_kpool_always_select_tail (bool)
index_share_for_mtp_iteration (bool)
indexer_rope_interleave (bool)
mtp_num_draft_tokens (int)
dsa_long_context (bool)
first_k_dense_replace (int)
intermediate_size (int)
moe_intermediate_size (int)
n_routed_experts (int)
n_shared_experts (int)
num_experts_per_tok (int)
routed_scaling_factor (float)
scoring_func (str)
topk_method (str)
norm_topk_prob (bool)
moe_router_dtype (str)
swiglu_limit (float)
max_position_embeddings (int)
max_seq_len (int)
num_nextn_predict_layers (int)
pad_token_id (int)
max_output_tokens (int | None)
quantization_config (Fp8BlockQuantConfig | None)
moe_fp8_resident (bool)
moe_quant_kernel (str)
prefill_graphs (bool)
- classmethod from_hf_config(hf_config, strict=True)[source]#
Build from the checkpoint’s config.json dict.
- Parameters:
- Return type:
- is_full_attention_layer(layer_idx)[source]#
True for MLA+DSA layers; the MTP layer (== num_hidden_layers) always is (it ships its own full indexer).
- property kv_rows: int#
Cache rows a request may hold. Its real context is held to max_seq_len; under MTP each verify step also leaves its rejected drafts as holes. Admission books every step’s whole block (no draft accepted), so k + 1 rows a token keep the rows from refusing a request its context holds. Pages are taken as the rows fill, so the bound allocates nothing.
- property linear_conv_channels: int#
3 x qkv_dim.
- Type:
Fused depthwise conv channels over cat(q, k, v)
- property mla_cache_kpe: int#
Rope-slot width of the MLA latent cache. NoPE needs none; MTP keeps a 64-wide slot to mask rejected draft rows.
- property padded_head_dim: int#
qk_head_dim rounded up to a FlashInfer paged-kernel head dim (64, 128 or 256).
- quantization_config: Fp8BlockQuantConfig | None = None#
- classmethod reduced()[source]#
Tiny-dim variant for CPU tests: same shapes family, random weights.
- Return type:
- mstar.model.glm5_next.config.build_indexer_types(num_hidden_layers)[source]#
Every MLA layer runs its own full indexer; none are shared.