mstar.model.qwen3_5.components.vision

Qwen3.5’s vision tower: a ViT over packed image patches.

Ported from transformers’ modeling_qwen3_5.py and vision_utils.py (Apache-2.0); the grid helpers are copied because transformers.vision_utils is internal. Dropped, as Qwen3.5 does not use them: bicubic resampling, the temporal rotary axis, and deepstack.

Images arrive packed: pixel_values is [total_patches, patch_numel] and grid_thw gives each image’s (t, h, w) patch grid.

Functions

vision_interpolation(grid, ...)	Gather indices and weights that resample the square learned position table onto each image's (h, w) grid.
vision_position_ids(grid, ...)	[total_patches, 2] (h, w) indices in spatial-merge-block order.
vision_seq_lengths(grid)	One attending segment per frame, on the host: t segments of h * w per entry, planned by the ragged attention resource.

Classes

Qwen3_5VisionModel(config)
VisionAttention(config)	Full attention within each frame, over packed patches.
VisionBlock(config)
VisionMLP(config)	Plain, not gated: one up projection, an activation, one down.
VisionPatchEmbed(config)
VisionPatchMerger(config)	Folds each m x m block of patches into one LLM-width token.
VisionRotaryEmbedding(config)	Axial 2D rope: the same frequencies drive the H and W axes, and the two halves are concatenated to cover the whole head — no partial rotation.

class mstar.model.qwen3_5.components.vision.Qwen3_5VisionModel(config)

Bases: Module

Parameters:

config (Qwen3_5VisionConfig)

embed(pixel_values, indices, weights, position_ids)

Patch embed plus the resampled position table, and the rope for the blocks: everything before the block loop, per patch.

Parameters:

• pixel_values (Tensor)

• indices (Tensor)

• weights (Tensor)

• position_ids (Tensor)

Return type:

tuple[Tensor, Tensor, Tensor]

encode(hidden, cos, sin)

The block loop, [patches, hidden] in and out – the part a CUDA graph can hold (see VisionEncoderSubmodule).

Parameters:

• hidden (Tensor)

• cos (Tensor)

• sin (Tensor)

Return type:

Tensor

forward(pixel_values, indices, weights, position_ids)

[total_patches, patch_numel] in, [merged_tokens, out_hidden] out — one token per spatial_merge_size ** 2 block of patches.

Grid-derived inputs are built by the caller (VisionEncoderSubmodule.prepare_inputs): grid_thw.tolist() here would break the graph and make h, w unbacked symints, and inductor then refuses (CantSplit) to codegen packed multi-image runs.

Parameters:

• pixel_values (Tensor)

• indices (Tensor)

• weights (Tensor)

• position_ids (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionAttention(config)

Bases: Module

Full attention within each frame, over packed patches.

The ragged attention resource plans the per-frame layout, so this forward never sees the boundaries. That is what lets the tower compile: splitting per frame here gave each frame its own symint, and inductor then refused (CantSplit) to codegen packed runs of three or more images.

Parameters:

config (Qwen3_5VisionConfig)

bind_resources(resources)

See NodeSubmodule.bind_node_resources. .get: a deployment that never builds the vision node binds nothing here.

Parameters:

resources (dict)

Return type:

None

forward(hidden_states, cos, sin)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

• hidden_states (Tensor)

• cos (Tensor)

• sin (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionBlock(config)

Bases: Module

Parameters:

config (Qwen3_5VisionConfig)

forward(hidden_states, cos, sin)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

• hidden_states (Tensor)

• cos (Tensor)

• sin (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionMLP(config)

Bases: Module

Plain, not gated: one up projection, an activation, one down.

Parameters:

config (Qwen3_5VisionConfig)

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

x (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionPatchEmbed(config)

Bases: Module

Parameters:

config (Qwen3_5VisionConfig)

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

x (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionPatchMerger(config)

Bases: Module

Folds each m x m block of patches into one LLM-width token.

Parameters:

config (Qwen3_5VisionConfig)

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

x (Tensor)

Return type:

Tensor

class mstar.model.qwen3_5.components.vision.VisionRotaryEmbedding(config)

Bases: Module

Axial 2D rope: the same frequencies drive the H and W axes, and the two halves are concatenated to cover the whole head — no partial rotation.

Parameters:

config (Qwen3_5VisionConfig)

forward(position_ids)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

position_ids (Tensor)

Return type:

tuple[Tensor, Tensor]

mstar.model.qwen3_5.components.vision.vision_interpolation(grid, num_grid_per_side, spatial_merge_size, device)

Gather indices and weights that resample the square learned position table onto each image’s (h, w) grid.

Returns [total_patches, 4] each — the 2x2 bilinear neighbourhood per patch, as the outer product of the two axes’ taps.

Parameters:

• grid (list[tuple[int, int, int]])

• num_grid_per_side (int)

• spatial_merge_size (int)

• device (device)

Return type:

tuple[Tensor, Tensor]

mstar.model.qwen3_5.components.vision.vision_position_ids(grid, spatial_merge_size, device)

[total_patches, 2] (h, w) indices in spatial-merge-block order.

Block-major, so the rotary sees the order the merger will fold.

Parameters:

• grid (list[tuple[int, int, int]])

• spatial_merge_size (int)

• device (device)

Return type:

Tensor

mstar.model.qwen3_5.components.vision.vision_seq_lengths(grid)

One attending segment per frame, on the host: t segments of h * w per entry, planned by the ragged attention resource.

Parameters:

grid (list[tuple[int, int, int]])

Return type:

tuple[int, …]