mstar.model.qwen3_5.config

Contents

mstar.model.qwen3_5.config#

Qwen3.5’s text and vision configs, read off the HF config.json.

Field names mirror HF’s so from_hf can splat the checkpoint’s own dict.

Classes

Qwen3_5Config(num_hidden_layers, ...[, ...])

Qwen3_5VisionConfig(depth, hidden_size, ...)

The ViT tower's own config, read off config.json's vision_config.
class mstar.model.qwen3_5.config.Qwen3_5Config(num_hidden_layers: 'int', hidden_size: 'int', intermediate_size: 'int', layer_types: 'list[str]', num_attention_heads: 'int', num_key_value_heads: 'int', head_dim: 'int', linear_num_key_heads: 'int', linear_num_value_heads: 'int', linear_key_head_dim: 'int', linear_value_head_dim: 'int', linear_conv_kernel_dim: 'int', vocab_size: 'int', rms_norm_eps: 'float', max_position_embeddings: 'int', attn_output_gate: 'bool' = False, tie_word_embeddings: 'bool' = False, eos_token_id: 'int | list[int] | None' = None, vision_start_token_id: 'int | None' = None, vision_end_token_id: 'int | None' = None, image_token_id: 'int | None' = None, video_token_id: 'int | None' = None, rope_theta: 'float' = 10000.0, mrope_section: 'list[int] | None' = None, mrope_interleaved: 'bool' = False, partial_rotary_factor: 'float' = 1.0, mtp_num_hidden_layers: 'int' = 0, mamba_ssm_dtype: 'str' = 'float32', extra_stop_token_ids: 'tuple[int, ...]' = ())[source]#

Bases: object

Parameters:

  • num_hidden_layers (int)

  • hidden_size (int)

  • intermediate_size (int)

  • layer_types (list[str])

  • num_attention_heads (int)

  • num_key_value_heads (int)

  • head_dim (int)

  • linear_num_key_heads (int)

  • linear_num_value_heads (int)

  • linear_key_head_dim (int)

  • linear_value_head_dim (int)

  • linear_conv_kernel_dim (int)

  • vocab_size (int)

  • rms_norm_eps (float)

  • max_position_embeddings (int)

  • attn_output_gate (bool)

  • tie_word_embeddings (bool)

  • eos_token_id (int | list[int] | None)

  • vision_start_token_id (int | None)

  • vision_end_token_id (int | None)

  • image_token_id (int | None)

  • video_token_id (int | None)

  • rope_theta (float)

  • mrope_section (list[int] | None)

  • mrope_interleaved (bool)

  • partial_rotary_factor (float)

  • mtp_num_hidden_layers (int)

  • mamba_ssm_dtype (str)

  • extra_stop_token_ids (tuple[int, ...])

attn_output_gate: bool = False#
eos_token_id: int | list[int] | None = None#
property eos_token_ids: frozenset[int]#

What config.json names; one id or several.

extra_stop_token_ids: tuple[int, ...] = ()#
classmethod from_hf(path)[source]#
Parameters:

path (str | Path)

Return type:

Qwen3_5Config

property full_layer_indices: list[int]#
head_dim: int#
hidden_size: int#
image_token_id: int | None = None#
intermediate_size: int#
layer_types: list[str]#
linear_conv_kernel_dim: int#
linear_key_head_dim: int#
property linear_layer_indices: list[int]#
linear_num_key_heads: int#
linear_num_value_heads: int#
linear_value_head_dim: int#
mamba_ssm_dtype: str = 'float32'#
max_position_embeddings: int#
mrope_interleaved: bool = False#
mrope_section: list[int] | None = None#
mtp_num_hidden_layers: int = 0#
num_attention_heads: int#
num_hidden_layers: int#
num_key_value_heads: int#
partial_rotary_factor: float = 1.0#
resource_layer_index(layer_idx)[source]#

Where a stack layer sits among its own resource’s layers.

The KV cache holds only full-attention layers and the recurrent pool only linear ones; a stack index would read another layer’s state.

Parameters:

layer_idx (int)

Return type:

int

rms_norm_eps: float#
rope_theta: float = 10000.0#
property rotary_dim: int#

Only part of each head is rotated; mrope_section covers half of it.

property stop_token_ids: frozenset[int]#

Every id that ends generation — wider than eos_token_ids.

Chat turns end on <|im_end|>, but config.json names only <|endoftext|>, so the model adds the tokenizer’s eos via extra_stop_token_ids; without it every reply runs to the budget.

tie_word_embeddings: bool = False#
video_token_id: int | None = None#
vision_end_token_id: int | None = None#
vision_start_token_id: int | None = None#
vocab_size: int#
class mstar.model.qwen3_5.config.Qwen3_5VisionConfig(depth, hidden_size, intermediate_size, num_heads, in_channels, patch_size, temporal_patch_size, spatial_merge_size, num_position_embeddings, out_hidden_size, hidden_act='gelu_pytorch_tanh', rope_theta=10000.0, deepstack_visual_indexes=())[source]#

Bases: object

The ViT tower’s own config, read off config.json’s vision_config.

Parameters:

  • depth (int)

  • hidden_size (int)

  • intermediate_size (int)

  • num_heads (int)

  • in_channels (int)

  • patch_size (int)

  • temporal_patch_size (int)

  • spatial_merge_size (int)

  • num_position_embeddings (int)

  • out_hidden_size (int)

  • hidden_act (str)

  • rope_theta (float)

  • deepstack_visual_indexes (tuple[int, ...])

deepstack_visual_indexes: tuple[int, ...] = ()#
depth: int#
classmethod from_hf(path)[source]#
Parameters:

path (str | Path)

Return type:

Qwen3_5VisionConfig

classmethod from_hf_or_none(path)[source]#

None when the checkpoint is text-only.

Parameters:

path (str | Path)

Return type:

Qwen3_5VisionConfig | None

property head_dim: int#
hidden_act: str = 'gelu_pytorch_tanh'#
hidden_size: int#
in_channels: int#
intermediate_size: int#
property merge_unit: int#
property num_grid_per_side: int#

The learned position table is a square grid.

num_heads: int#
num_position_embeddings: int#
out_hidden_size: int#
patch_size: int#
rope_theta: float = 10000.0#
spatial_merge_size: int#
temporal_patch_size: int#
Contents