mstar.api_server.ws_frames#
Parsing large /generate/ws frames without holding the event loop.
json.loads, msgpack.unpackb and base64 decoding hold the GIL for the
whole call, so a thread doesn’t free the loop; nor does pickling the frame
across to a process, one more whole-frame call. A large frame is therefore
handed to a worker process through a file, written from a thread in slices
short enough that the loop gets the GIL between them; the worker parses it,
saves each file’s data under the upload dir (decoding base64 as it goes) and
sends back only the message, each saved file’s data a SavedUpload. A
small frame stays on the loop, where the round trip costs more than the parse.
Taking the frame in stays on the loop: uvicorn and websockets assemble, unmask, decode and, with deflate on, inflate it there.
The pool’s processes re-import the main module, as every spawned process does:
under mstar serve that is light, under the mstar-serve script it is the
server’s entrypoint (torch included, a few hundred MB each).
Functions
|
A frame's message: msgpack for bytes, JSON for text. |
|
A frame's message. |
|
Save a file's |
|
Start the pool with the server, so its processes are starting before the first frame (they take a few seconds, spawned off the server's main module). |
|
Classes
|
A file's |
- class mstar.api_server.ws_frames.SavedUpload(path)[source]#
Bases:
objectA file’s
datathe pool already saved topath. The file is removed with this object unlessclaimtakes it first, so a message refused before its files are taken leaves nothing behind.- Parameters:
path (str)
- mstar.api_server.ws_frames.parse(payload)[source]#
A frame’s message: msgpack for bytes, JSON for text.
- async mstar.api_server.ws_frames.parse_frame(payload, upload_dir)[source]#
A frame’s message. A large frame parses in the pool, its files’ data already saved under
upload_dir(SavedUpload). Raises what the parse raises, andOSErrorwhen it could not be handed off, or a pool process died with the frame in the pool: that fails every frame the pool held, on any socket, and a new pool takes the next. A pool found dead at the hand-off held nothing of this frame’s, which goes to a new pool.
- mstar.api_server.ws_frames.save_entry(name, data, upload_dir, prefix=None)[source]#
Save a file’s
data: raw bytes, or base64 text, by the decoder the data-URL paths use (unpadded or wrapped is fine; a bad alphabet is the client’s error). A failed write leaves nothing.