mstar.model.media_limits#

Decoded-size caps for request media.

A compressed image or video can be tiny on the wire and enormous once decoded (a flat PNG compresses about 1000:1, a still video far more), so the loaders check what a file will decode to, not how big it is. A refusal is a ValueError, which the data worker answers with a 400, as is a file its decoder can’t read (decoding_upload()).

The pixel and audio caps hold for a whole request: while the data worker loads a request’s files inside RequestMediaBudget, every check also counts against what the request’s other files use, whether finished or still loading.

Module Attributes

RAW_INPUT_CHARGES

Which modalities a request may send as raw_inputs (tensors instead of files), each with its size check: the check refuses an input over the limit (for images, MSTAR_MAX_IMAGE_PIXELS) and counts it toward the request's total, as reading a file's header does.

Functions

charge_decoded_audio(audio, sample_rate)

Charge the request the samples a loader returned, as seconds at sample_rate; a file whose header understated its length or could not be read is counted here.

charge_decoded_image(image)

Hold an image that arrived decoded ([C, H, W]) to the cap a file's header is held to, and charge the request its pixels.

check_audio_duration(filepath)

Charge the request what an audio file's header declares, before the model's loader decodes it; a file whose header declares nothing is counted as it decodes (_count_audio_seconds()).

check_file_counts(counts, model_limits)

Refuse more files of a modality than the model takes (mstar.model.base.Model.max_input_files()), before any is decoded.

check_image_size(filepath[, formats])

Refuse an image whose header promises more than max_image_pixels() or that is not in formats (the ones the caller's decoder reads); returns Pillow's name for the format and whether the file holds more than one frame.

check_media_limits()

Read the limits once at startup: a bad value then stops the server, instead of turning every media request into a 400 that blames the client.

count_sound_file_seconds(filepath, max_seconds)

The length of a libsndfile file whose header gives none, counted as it decodes and keeping no samples; stops just past max_seconds, so the caller's limit check refuses a longer clip without decoding all of it.

decoding_upload(filepath, what, *[, ffmpeg])

Around a loader's decode call: the decoder refusing filepath is the client's error, raised as UndecodableMediaError.

ffmpeg_unavailable()

Why torchcodec can't decode in this process, or None if it can.

max_audio_seconds()

MSTAR_MAX_AUDIO_SECONDS: seconds a request's audio clips may decode to, summed over them.

max_image_pixels()

MSTAR_MAX_IMAGE_PIXELS: pixels a request's images may decode to, summed over them.

max_media_files()

MSTAR_MAX_MEDIA_FILES: images, audio clips and videos one request may attach, together.

max_video_pixels()

MSTAR_MAX_VIDEO_PIXELS: pixels a request's videos may decode to, summed over the frames the loader decodes from each.

open_upright_image(filepath[, formats])

The image opened with Pillow and turned upright by its EXIF orientation, as transformers' and diffusers' load_image do.

probe_audio_seconds(filepath)

The duration an audio file's header declares, or None when it declares none (a sample count of 0 or libsndfile's "unknown" value) or neither soundfile nor torchcodec can read it.

probe_video(filepath)

(num_frames, fps, height, width) from the container header, or None when neither torchcodec nor PyAV can read it.

read_image(filepath, *[, rgb])

A request image as uint8 [C, H, W], refused first if it is too large and then if it does not decode.

read_sound_file(filepath, max_seconds)

soundfile.read(filepath, dtype="float32", always_2d=True) for a header that gives no sample count, where soundfile.read would size its buffer from the header (libsndfile's 2**63 - 1 frames) and fail; decodes block by block and stops just past max_seconds, so the caller's length check refuses a longer clip.

sound_file_seconds(frames, rate)

The length a libsndfile header declares, or None when it declares none (a sample count of 0 or libsndfile's "unknown" value).

video_decode_threads()

FFmpeg threads for one video decode.

Classes

RequestMediaBudget(num_files)

Holds one request's media to the caps together, not file by file, so many files in one request cannot multiply them; refuses more than max_media_files() files before any is decoded.

VideoBudget()

Counts decoded frames against max_video_pixels(), so a loader stops before it holds more than the cap whatever the container claims.

Exceptions

UndecodableMediaError

A request file its decoder refused: the client's error, a 400.

mstar.model.media_limits.RAW_INPUT_CHARGES = {'image': <function charge_decoded_image>}#

Which modalities a request may send as raw_inputs (tensors instead of files), each with its size check: the check refuses an input over the limit (for images, MSTAR_MAX_IMAGE_PIXELS) and counts it toward the request’s total, as reading a file’s header does. The model then loads each with Model.load_decoded_<modality>.

class mstar.model.media_limits.RequestMediaBudget(num_files)[source]#

Bases: object

Holds one request’s media to the caps together, not file by file, so many files in one request cannot multiply them; refuses more than max_media_files() files before any is decoded.

The data worker loads each file inside file(); the image, video and audio checks charge the request through it.

Parameters:

num_files (int)

charge(charges, kind, amount)[source]#
Parameters:
Return type:

None

file()[source]#

Load one file. It is charged the most any single check counted for it, so a header estimate and the decode after it are not added up.

exception mstar.model.media_limits.UndecodableMediaError[source]#

Bases: ValueError

A request file its decoder refused: the client’s error, a 400.

class mstar.model.media_limits.VideoBudget[source]#

Bases: object

Counts decoded frames against max_video_pixels(), so a loader stops before it holds more than the cap whatever the container claims.

A frame counts as at least MIN_FRAME_PIXELS: each decoded frame costs about 2 KB of tensor and frame objects whatever its size, so a clip of millions of 2x2 frames would otherwise pass the cap at hundreds of GB.

add(height, width, frames=1)[source]#
Parameters:
Return type:

None

mstar.model.media_limits.charge_decoded_audio(audio, sample_rate)[source]#

Charge the request the samples a loader returned, as seconds at sample_rate; a file whose header understated its length or could not be read is counted here.

Parameters:
Return type:

None

mstar.model.media_limits.charge_decoded_image(image)[source]#

Hold an image that arrived decoded ([C, H, W]) to the cap a file’s header is held to, and charge the request its pixels.

Parameters:

image (Tensor)

Return type:

None

mstar.model.media_limits.check_audio_duration(filepath)[source]#

Charge the request what an audio file’s header declares, before the model’s loader decodes it; a file whose header declares nothing is counted as it decodes (_count_audio_seconds()).

Parameters:

filepath (str)

Return type:

None

mstar.model.media_limits.check_file_counts(counts, model_limits)[source]#

Refuse more files of a modality than the model takes (mstar.model.base.Model.max_input_files()), before any is decoded.

Parameters:
Return type:

None

mstar.model.media_limits.check_image_size(filepath, formats=IMAGE_FORMATS)[source]#

Refuse an image whose header promises more than max_image_pixels() or that is not in formats (the ones the caller’s decoder reads); returns Pillow’s name for the format and whether the file holds more than one frame.

Reads the header only. The decoders allocate what the header declares, so it cannot understate the decode.

Parameters:
Return type:

tuple[str, bool]

mstar.model.media_limits.check_media_limits()[source]#

Read the limits once at startup: a bad value then stops the server, instead of turning every media request into a 400 that blames the client.

Also lifts Pillow’s own bomb check to the image limit, which it would otherwise apply first (at about 179M pixels) whatever the limit is.

Return type:

None

mstar.model.media_limits.count_sound_file_seconds(filepath, max_seconds)[source]#

The length of a libsndfile file whose header gives none, counted as it decodes and keeping no samples; stops just past max_seconds, so the caller’s limit check refuses a longer clip without decoding all of it.

Parameters:
Return type:

float

mstar.model.media_limits.decoding_upload(filepath, what, *, ffmpeg=False)[source]#

Around a loader’s decode call: the decoder refusing filepath is the client’s error, raised as UndecodableMediaError.

Decoders raise RuntimeError (torchvision, torchcodec, libsndfile) or OSError (Pillow) for a file they can’t read, but also for the server’s own faults, which are re-raised: a failed allocation, an upload our cleanup removed and, with ffmpeg, a torchcodec that has no FFmpeg.

Parameters:
mstar.model.media_limits.ffmpeg_unavailable()[source]#

Why torchcodec can’t decode in this process, or None if it can.

Return type:

str | None

mstar.model.media_limits.max_audio_seconds()[source]#

MSTAR_MAX_AUDIO_SECONDS: seconds a request’s audio clips may decode to, summed over them.

Return type:

int

mstar.model.media_limits.max_image_pixels()[source]#

MSTAR_MAX_IMAGE_PIXELS: pixels a request’s images may decode to, summed over them.

Return type:

int

mstar.model.media_limits.max_media_files()[source]#

MSTAR_MAX_MEDIA_FILES: images, audio clips and videos one request may attach, together.

Return type:

int

mstar.model.media_limits.max_video_pixels()[source]#

MSTAR_MAX_VIDEO_PIXELS: pixels a request’s videos may decode to, summed over the frames the loader decodes from each.

Return type:

int

mstar.model.media_limits.open_upright_image(filepath, formats=None)[source]#

The image opened with Pillow and turned upright by its EXIF orientation, as transformers’ and diffusers’ load_image do. A malformed EXIF block leaves it as stored.

Parameters:
mstar.model.media_limits.probe_audio_seconds(filepath)[source]#

The duration an audio file’s header declares, or None when it declares none (a sample count of 0 or libsndfile’s “unknown” value) or neither soundfile nor torchcodec can read it.

Parameters:

filepath (str)

Return type:

float | None

mstar.model.media_limits.probe_video(filepath)[source]#

(num_frames, fps, height, width) from the container header, or None when neither torchcodec nor PyAV can read it.

Parameters:

filepath (str)

Return type:

tuple[int, float, int, int] | None

mstar.model.media_limits.read_image(filepath, *, rgb=False)[source]#

A request image as uint8 [C, H, W], refused first if it is too large and then if it does not decode.

rgb forces three channels, with any transparency composited on white as the Qwen-VL reference preprocessing does for RGBA (dropping the alpha would turn a transparent background black); otherwise the file’s own channels are kept, as torchvision.io.decode_image does. A GIF or an animated WebP is read with Pillow, as its first frame in RGB: torchvision’s GIF decoder is not safe on untrusted files before torchvision 0.28, and its WebP decoder refuses animation and ignores EXIF, so a WebP stored rotated is read with Pillow too. The image is turned upright by its EXIF orientation (see open_upright_image()).

Parameters:
Return type:

Tensor

mstar.model.media_limits.read_sound_file(filepath, max_seconds)[source]#

soundfile.read(filepath, dtype="float32", always_2d=True) for a header that gives no sample count, where soundfile.read would size its buffer from the header (libsndfile’s 2**63 - 1 frames) and fail; decodes block by block and stops just past max_seconds, so the caller’s length check refuses a longer clip.

Parameters:
mstar.model.media_limits.sound_file_seconds(frames, rate)[source]#

The length a libsndfile header declares, or None when it declares none (a sample count of 0 or libsndfile’s “unknown” value).

Parameters:
Return type:

float | None

mstar.model.media_limits.video_decode_threads()[source]#

FFmpeg threads for one video decode. torchcodec decodes single-threaded by default, which dominates a long clip’s request latency; frame threading is bit-identical. Bounded by 8 and by the CPUs the process’s plan leaves unpinned, as CpuPlan.png_threads is. The bound is per decode: concurrent decodes each start this many threads. Not tied to torch’s intra-op pool: FFmpeg’s threads are its own, and that pool is kept small for fork-join ops a decode doesn’t run.

Return type:

int