mstar.worker.host_pool#
Host memory a worker copies request state into, so a request can leave the device and come back.
Functions
|
|
Classes
|
A device tensor copied into the pool: a view of where it went, and the event that records the copy, None when it copied in place. |
|
|
|
- class mstar.worker.host_pool.HostCopy(view, landed)[source]#
Bases:
NamedTupleA device tensor copied into the pool: a view of where it went, and the event that records the copy, None when it copied in place.
- class mstar.worker.host_pool.HostPool(nbytes, device)[source]#
Bases:
objectnbytesof host memory allocated once, pinned wheredevicetakes pinned memory, and handed out first fit. The pool roundsnbytesup to a power of two, as the pinned allocator does, so it hands out all it pins. Copies run on streams of the pool’s own, one each way, so a copy never waits behind a later step, and a copy back to the device never waits behind copies to the host.- Parameters:
nbytes (int)
device (torch.device)
- copy_in(view)[source]#
Copy
view, a tensor in the pool whose copy from the device has landed, to the device, and return the copy with the event recorded once it lands. Where the device is the host, returnviewas it is and no event.
- copy_out(src, into, after)[source]#
Copy
srcas it is to the start ofintoonceafter(the step that wrote it) completes, running nothing on the device but the copy.
- class mstar.worker.host_pool.HostSlice(offset, nbytes)[source]#
Bases:
NamedTuple