mstar.model.muse.config#
Muse Glimmer’s text and vision configs, read off the HF config.json.
Field names mirror HF’s so from_hf can splat the checkpoint’s own dicts.
Classes
|
|
|
- class mstar.model.muse.config.MuseGlimmerConfig(num_hidden_layers: 'int', hidden_size: 'int', intermediate_size: 'int', num_attention_heads: 'int', num_key_value_heads: 'int', head_dim: 'int', vocab_size: 'int', layer_types: 'list[str]', layer_rope_theta: 'list[float]', sliding_window: 'int', max_position_embeddings: 'int', rms_norm_eps: 'float', post_norm_eps: 'float', image_token_id: 'int', qk_scale_factor: 'float' = 3.87, output_multiplier: 'float' = 0.19611613513818404, final_logit_softcapping: 'float' = 20.0, rope_theta: 'float' = 500000.0, tie_word_embeddings: 'bool' = False, eos_token_id: 'int | list[int] | None' = None, generation_eos_token_ids: 'tuple[int, ...]' = (), eot_token_id: 'int | None' = None)[source]#
Bases:
object- Parameters:
num_hidden_layers (int)
hidden_size (int)
intermediate_size (int)
num_attention_heads (int)
num_key_value_heads (int)
head_dim (int)
vocab_size (int)
sliding_window (int)
max_position_embeddings (int)
rms_norm_eps (float)
post_norm_eps (float)
image_token_id (int)
qk_scale_factor (float)
output_multiplier (float)
final_logit_softcapping (float)
rope_theta (float)
tie_word_embeddings (bool)
eot_token_id (int | None)
- class mstar.model.muse.config.MuseGlimmerVisionConfig(hidden_size: 'int', intermediate_size: 'int', num_attention_heads: 'int', num_hidden_layers: 'int', patch_size: 'int', patch_temporal: 'int', merge_size: 'int', pos_emb_height: 'int', pos_emb_width: 'int', layer_types: 'list[str]', out_hidden_size: 'int', projector_hidden_size: 'int', text_hidden_size: 'int', text_rms_norm_eps: 'float' = 1e-05, projector_hidden_act: 'str' = 'gelu', layer_norm_eps: 'float' = 1e-05, hidden_act: 'str' = 'gelu', rope_theta: 'float' = 10000.0)[source]#
Bases:
object- Parameters:
hidden_size (int)
intermediate_size (int)
num_attention_heads (int)
num_hidden_layers (int)
patch_size (int)
patch_temporal (int)
merge_size (int)
pos_emb_height (int)
pos_emb_width (int)
out_hidden_size (int)
projector_hidden_size (int)
text_hidden_size (int)
text_rms_norm_eps (float)
projector_hidden_act (str)
layer_norm_eps (float)
hidden_act (str)
rope_theta (float)