mstar.model.glm5_next.fused_decode

mstar.model.glm5_next.fused_decode#

Fused Triton kernels for the glm5_next layer: mHC and the SwiGLU.

Each replaces a chain of small torch ops that is kernel-count bound at decode batch sizes. The math and the bf16 rounding points follow the torch reference in mhc.py and moe.py, which runs wherever the kernels cannot (CPU, or _ENABLED off in tests).

Functions

fused_available(t)

hc_pre(streams, fn, hc_scale, hc_base, ...)

One mHC site plus the norm that follows it, on streams (1, T, HC, HID).

prefill_sized(streams)

hc_pre and update_streams take their prefill-size path for these streams.

swiglu(gate_up, limit)

The clamped SwiGLU of a gated MLP's gate_up (T, 2 I) bf16 output, (T, I).

update_streams(residual, h, post, comb)

Fused mhc.update_streams for residual (1, T, HC, HID), h (T, HID).

mstar.model.glm5_next.fused_decode.fused_available(t)[source]#
Parameters:

t (Tensor)

Return type:

bool

mstar.model.glm5_next.fused_decode.hc_pre(streams, fn, hc_scale, hc_base, norm_w, *, rms_eps, hc_eps, iters, norm_eps, update=None)[source]#

One mHC site plus the norm that follows it, on streams (1, T, HC, HID).

Returns (post (1,T,HC), comb (1,T,HC,HC), normed (T,HID), streams). With update=(h, post, comb) the input is the residual: the previous sublayer’s update_streams is applied first and the updated streams are returned.

mstar.model.glm5_next.fused_decode.prefill_sized(streams)[source]#

hc_pre and update_streams take their prefill-size path for these streams.

Return type:

bool

mstar.model.glm5_next.fused_decode.swiglu(gate_up, limit)[source]#

The clamped SwiGLU of a gated MLP’s gate_up (T, 2 I) bf16 output, (T, I).

mstar.model.glm5_next.fused_decode.update_streams(residual, h, post, comb)[source]#

Fused mhc.update_streams for residual (1, T, HC, HID), h (T, HID).