mstar.model.muse.config

Contents

mstar.model.muse.config#

Muse Glimmer’s text and vision configs, read off the HF config.json.

Field names mirror HF’s so from_hf can splat the checkpoint’s own dicts.

Classes

MuseGlimmerConfig(num_hidden_layers, ...[, ...])

MuseGlimmerVisionConfig(hidden_size, ...[, ...])

class mstar.model.muse.config.MuseGlimmerConfig(num_hidden_layers: 'int', hidden_size: 'int', intermediate_size: 'int', num_attention_heads: 'int', num_key_value_heads: 'int', head_dim: 'int', vocab_size: 'int', layer_types: 'list[str]', layer_rope_theta: 'list[float]', sliding_window: 'int', max_position_embeddings: 'int', rms_norm_eps: 'float', post_norm_eps: 'float', image_token_id: 'int', qk_scale_factor: 'float' = 3.87, output_multiplier: 'float' = 0.19611613513818404, final_logit_softcapping: 'float' = 20.0, rope_theta: 'float' = 500000.0, tie_word_embeddings: 'bool' = False, eos_token_id: 'int | list[int] | None' = None, generation_eos_token_ids: 'tuple[int, ...]' = (), eot_token_id: 'int | None' = None)[source]#

Bases: object

Parameters:
  • num_hidden_layers (int)

  • hidden_size (int)

  • intermediate_size (int)

  • num_attention_heads (int)

  • num_key_value_heads (int)

  • head_dim (int)

  • vocab_size (int)

  • layer_types (list[str])

  • layer_rope_theta (list[float])

  • sliding_window (int)

  • max_position_embeddings (int)

  • rms_norm_eps (float)

  • post_norm_eps (float)

  • image_token_id (int)

  • qk_scale_factor (float)

  • output_multiplier (float)

  • final_logit_softcapping (float)

  • rope_theta (float)

  • tie_word_embeddings (bool)

  • eos_token_id (int | list[int] | None)

  • generation_eos_token_ids (tuple[int, ...])

  • eot_token_id (int | None)

property attn_scale: float#

The query multiplier after QK-norm, folded into the softmax scale.

eos_token_id: int | list[int] | None = None#
eot_token_id: int | None = None#
final_logit_softcapping: float = 20.0#
classmethod from_hf(path)[source]#
Parameters:

path (str | Path)

Return type:

MuseGlimmerConfig

generation_eos_token_ids: tuple[int, ...] = ()#
head_dim: int#
hidden_size: int#
image_token_id: int#
intermediate_size: int#
property is_sliding: list[bool]#
layer_rope_theta: list[float]#
layer_types: list[str]#
max_position_embeddings: int#
num_attention_heads: int#
num_hidden_layers: int#
num_key_value_heads: int#
output_multiplier: float = 0.19611613513818404#
post_norm_eps: float#
qk_scale_factor: float = 3.87#
rms_norm_eps: float#
rope_theta: float = 500000.0#
sliding_window: int#
property stop_token_ids: frozenset[int]#

<|end_of_text|> and <|eot|>.

<|eom|> is not one: it closes the reasoning channel and all but the last of parallel tool calls, and the model goes on after it.

tie_word_embeddings: bool = False#
vocab_size: int#
class mstar.model.muse.config.MuseGlimmerVisionConfig(hidden_size: 'int', intermediate_size: 'int', num_attention_heads: 'int', num_hidden_layers: 'int', patch_size: 'int', patch_temporal: 'int', merge_size: 'int', pos_emb_height: 'int', pos_emb_width: 'int', layer_types: 'list[str]', out_hidden_size: 'int', projector_hidden_size: 'int', text_hidden_size: 'int', text_rms_norm_eps: 'float' = 1e-05, projector_hidden_act: 'str' = 'gelu', layer_norm_eps: 'float' = 1e-05, hidden_act: 'str' = 'gelu', rope_theta: 'float' = 10000.0)[source]#

Bases: object

Parameters:
  • hidden_size (int)

  • intermediate_size (int)

  • num_attention_heads (int)

  • num_hidden_layers (int)

  • patch_size (int)

  • patch_temporal (int)

  • merge_size (int)

  • pos_emb_height (int)

  • pos_emb_width (int)

  • layer_types (list[str])

  • out_hidden_size (int)

  • projector_hidden_size (int)

  • text_hidden_size (int)

  • text_rms_norm_eps (float)

  • projector_hidden_act (str)

  • layer_norm_eps (float)

  • hidden_act (str)

  • rope_theta (float)

classmethod from_hf(path)[source]#
Parameters:

path (str | Path)

Return type:

MuseGlimmerVisionConfig

property head_dim: int#
hidden_act: str = 'gelu'#
hidden_size: int#
intermediate_size: int#
layer_norm_eps: float = 1e-05#
layer_types: list[str]#
merge_size: int#
num_attention_heads: int#
property num_heads: int#
num_hidden_layers: int#
out_hidden_size: int#
property patch_dim: int#
patch_size: int#
patch_temporal: int#
pos_emb_height: int#
pos_emb_width: int#
projector_hidden_act: str = 'gelu'#
projector_hidden_size: int#
rope_theta: float = 10000.0#
text_hidden_size: int#
text_rms_norm_eps: float = 1e-05#
property window_patches: int#

A window is one learned-table’s worth of patches on a side.