spdl.pipeline.ArenaWriterProtocol

class ArenaWriterProtocol

Writer endpoint: the last step in the worker process.

Backends implement this protocol to receive offloaded binaries. A unit is one pipeline item: it groups all binaries produced by a single _offload() call so they can be reserved and returned to the arena in bulk. Methods are called from a single thread in the worker; they need not be thread-safe. In a process-region channel the sending role may instead belong to the parent.

Added in version 0.7.0.

Changed in version 0.9.0: This extension contract is publicly available from spdl.pipeline.

Methods

abort_unit()	Roll back the in-progress unit on failure.
begin_unit()	Open a new unit before any write_binary() calls.
commit_unit()	Close the current unit and return its span.
reset()	Prepare the writer for a new iteration.
write_binary(data)	Copy one binary into the arena and return its placement.

abort_unit() → None

Roll back the in-progress unit on failure.

Discards any binaries written since begin_unit() and leaves the writer outside of a unit, ready for another begin_unit(). Called when offloading raises (e.g. an unpicklable field or a reservation failure) so the writer is not left mid-unit. Previously committed units, surviving native exports and asynchronous transfer owners remain unaffected.

Returns:

None.

begin_unit() → None

Open a new unit before any write_binary() calls.

Must be paired with exactly one terminal call — commit_unit() on success or abort_unit() on failure. Calling write_binary() outside an open unit is an error.

Returns:

None.

Raises:

BufferError – Capacity is unavailable under the backend’s bounded waiting policy, or the backend is shutting down.

commit_unit() → int

Close the current unit and return its span.

Marks the unit as readable by the parent. After this call the writer is no longer in a unit; another begin_unit() is required before further writes.

Returns:

the total byte range the unit occupies in the arena. The caller (typically _offload()) embeds this in the envelope so the reader can return the whole region in one ArenaReaderProtocol.end_unit() call. The span includes any backend padding. Committed queued units must remain readable until consumed or explicitly discarded.

Return type:

The unit’s span

reset() → None

Prepare the writer for a new iteration.

Called by the worker once per START_ITERATION. Copy-out backends may reclaim fully consumed storage here, but must not discard unread committed units. Zero-copy backends must preserve any units whose restored views or native storage exports are alive across the boundary.

Returns:

None.

write_binary(data: bytes | bytearray | memoryview[bytes]) → tuple[int, int]

Copy one binary into the arena and return its placement.

Performs the single copy from data into shared memory; the caller is expected to have already exposed the source as a flat, contiguous byte view (no further copies on the source side).

Parameters:

data – Bytes-like source buffer. Must be flat and contiguous.

Returns:

(offset, nbytes) where offset is the physical byte offset of the payload within the arena’s data region and nbytes is the number of bytes written. Both values are stored by the serializer as placement metadata and used by ArenaReaderProtocol.read_binary() to retrieve the payload on the reader side. No payload-class interpretation is requested of the backend.

Raises:

BufferError – The fragment or accumulated unit exceeds the backend’s configured capacity.