mstar.model.muse.processing

mstar.model.muse.processing#

Muse Glimmer’s tokenizer, chat template and image preprocessing.

Native rather than through AutoProcessor: the HF processor classes need a transformers newer than the serving venv pins. Ported from transformers’ image_processing_muse_glimmer.py and processing_muse_glimmer.py (Apache-2.0).

Functions

load_tokenizer(local_dir)

The fast tokenizer with the repo's chat template and special tokens.

preprocess_image(image[, patch_size, ...])

(pixel_values [h * w, 1176], (1, h, w)) with the grid in patches.

smart_resize(height, width, patch_size, ...)

The integer merged-patch grid closest to the aspect ratio under the token cap, in pixels.

mstar.model.muse.processing.load_tokenizer(local_dir)[source]#

The fast tokenizer with the repo’s chat template and special tokens.

Built from tokenizer.json directly: the repo’s tokenizer_config.json is in a format older transformers cannot read.

Parameters:

local_dir (str | Path)

mstar.model.muse.processing.preprocess_image(image, patch_size=14, merge_size=2, temporal_patch_size=2, max_tokens=4096)[source]#

(pixel_values [h * w, 1176], (1, h, w)) with the grid in patches.

Lanczos resize, then (x / 255 - 0.5) / 0.5. Each patch is laid out (temporal, channel, py, px), the one frame repeated across the temporal pair.

Parameters:
  • patch_size (int)

  • merge_size (int)

  • temporal_patch_size (int)

  • max_tokens (int)

Return type:

tuple[Tensor, tuple[int, int, int]]

mstar.model.muse.processing.smart_resize(height, width, patch_size, max_tokens)[source]#

The integer merged-patch grid closest to the aspect ratio under the token cap, in pixels. No floor: a small image is scaled up.

Parameters:
  • height (int)

  • width (int)

  • patch_size (int)

  • max_tokens (int)

Return type:

tuple[int, int]