mstar.model.glm5_next.fused_decode#
Fused Triton kernels for the glm5_next layer: mHC and the SwiGLU.
Each replaces a chain of small torch ops that is kernel-count bound at decode batch sizes.
The math and the bf16 rounding points follow the torch reference in mhc.py and moe.py,
which runs wherever the kernels cannot (CPU, or _ENABLED off in tests).
Functions
|
One mHC site plus the norm that follows it, on |
|
hc_pre and update_streams take their prefill-size path for these streams. |
|
The clamped SwiGLU of a gated MLP's |
|
Fused |
- mstar.model.glm5_next.fused_decode.hc_pre(streams, fn, hc_scale, hc_base, norm_w, *, rms_eps, hc_eps, iters, norm_eps, update=None)[source]#
One mHC site plus the norm that follows it, on
streams (1, T, HC, HID).Returns
(post (1,T,HC), comb (1,T,HC,HC), normed (T,HID), streams). Withupdate=(h, post, comb)the input is the residual: the previous sublayer’supdate_streamsis applied first and the updated streams are returned.
- mstar.model.glm5_next.fused_decode.prefill_sized(streams)[source]#
hc_pre and update_streams take their prefill-size path for these streams.
- Return type: