mstar.model.cosmos3.components.conditioning

mstar.model.cosmos3.components.conditioning#

Conditioning-frame preprocessing for image-to-video.

Two recipes take a conditioning image to the generation size:

  • aspect_crop (the default): scale so the image covers the target (max(width / w, height / h)), antialiased bilinear resize to the ceiled size, center crop, round to 8 bits, then x / 127.5 - 1. NVIDIA’s cosmos-framework (_resize_and_center_crop in inference/vision.py) and diffusers >= 0.40 (_preprocess_conditioning_image) both do this.

  • stretch: antialiased bilinear resize straight to (height, width), aspect ratio not preserved, then the same 8-bit rounding. What diffusers 0.39 did with a PIL image; kept for A/B against that recipe.

prepare_conditioning_frames takes the frames the data worker loaded ([T, C, H, W] in [0, 1], or a single [C, H, W]) and returns [1, 3, T, height, width] in [-1, 1] for the VAE encoder.

Functions

prepare_conditioning_frames(frames, height, ...)

Resize conditioning pixels to the generation size; see the module docstring.
mstar.model.cosmos3.components.conditioning.prepare_conditioning_frames(frames, height, width, mode='aspect_crop')[source]#

Resize conditioning pixels to the generation size; see the module docstring.

Parameters:
Return type:

Tensor