mstar.utils.streams

mstar.utils.streams#

CUDA stream state a worker has to manage.

What runs on a side stream can matter: work on the SMs there (a kernel, or a device-to-device copy) while a CUDA graph replays has slowed every graph node in the process from then on, so side-stream work beside a live graph is H2D / D2H copies (see mstar.utils.h2d) unless measured. Measured on a B300 with Qwen3-TTS, whose Codec runs its windows on a side stream beside the Talker’s captured decode (side_stream in its configs), the decode steps took no longer with the codec there than with it on the chain: per-step GPU time p50 6.10 and 6.16 ms against 6.09 and 6.09 ms at c=8, 4.2 against 5.0 ms at c=1, two servers per arm (benchmark/worker_phases, gpu_exec.talker_decode).

Functions

reset_device_scheduling(device)

Create and destroy a throwaway CUDA context on device.

mstar.utils.streams.reset_device_scheduling(device)[source]#

Create and destroy a throwaway CUDA context on device.

Side-stream kernels or D2D copies beside a graph replay can leave graph nodes launching late until the device’s scheduling is rebuilt, which another context arriving on the GPU does.

Return type:

None