mstar.utils.h2d

mstar.utils.h2d#

Copy-only host-to-device staging, for work that runs beside a live CUDA graph.

Pre-planning builds step N+1’s small tensors on the plan stream while step N’s graph replays. Any SM work it issues (a kernel, a fill_, a D2D copy) slows the graph; H2D copies and event waits do not. So host values go to the device in ONE copy from pinned memory (PinnedStager), padding written on the host, never via a device-side staging tensor. An H2DMirror skips re-copying values a destination already holds, as in a steady decode batch.

Classes

H2DMirror()

What the stager last copied into one device destination.

PinnedStager(dtype[, numel, depth])

A small ring of persistent pinned buffers for one dtype.
class mstar.utils.h2d.H2DMirror[source]#

Bases: object

What the stager last copied into one device destination.

Lets PinnedStager.copy_ skip a copy whose values the destination already holds. Only sound for a destination nothing but that stager writes, and that lives as long as its mirror – hence an object the owner keeps with it, not a cache keyed by address (the allocator reuses addresses).

invalidate()[source]#

The destination was written some other way.

Return type:

None

matches(key, values)[source]#
Parameters:

  • key (tuple)

  • values (ndarray)

Return type:

bool

record(key, values)[source]#
Parameters:

  • key (tuple)

  • values (ndarray)

Return type:

None

class mstar.utils.h2d.PinnedStager(dtype, numel=256, depth=8)[source]#

Bases: object

A small ring of persistent pinned buffers for one dtype.

copy_ writes values into the next buffer and issues one non-blocking H2D copy from it on the current stream. Each buffer records an event after its copy and is not rewritten until that event has completed, so a copy still in flight never sees its source change. Thread-safe: the plan thread and the GPU thread may share one.

Parameters:
copy_(dst, values, pad_value=None, mirror=None)[source]#

dst[:len(values)] = values in one non-blocking H2D copy.

With pad_value, the rest of dst is set to it by the same copy (the padding is written on the host), so dst is fully overwritten without a device-side fill. dst must be contiguous.

With mirror (dst’s, see H2DMirror), nothing is issued when dst already holds exactly this: the copy that put it there is ahead of anything that reads dst after this call.

Parameters:
Return type:

None