Why shared arena outputs keep storage alive

Start with Shared-memory arenas for what arenas save, pool/ring choices, default CPU usage and the optional memory-pinning workflow. This Development Note focuses on the technical obligation behind those choices: do not overwrite or unmap bytes that an output or in-flight transfer still uses. Storage lifetime is not just the lifetime of the Python wrapper that first returned it; the same rule matters with or without pinning.

Pool views and ring copies have different lifetimes

An output restored from a SharedMemorySegmentPool refers to the shared unit’s storage. Reclaiming means permitting a producer to reuse those bytes for a later unit; it must wait while any consumer alias still uses them.

Consider a consumer keeping a view of an earlier batch:

cpu_batch = next(iterator)       # A restored pool-backed Torch tensor.
retained = cpu_batch.detach()    # Another Tensor using the same storage.
del cpu_batch                   # Only the original Python wrapper is gone.
# retained still reads the original values; this segment cannot be reused.
del retained                    # Reuse is possible once ALL storage aliases go.

A weakref/finalizer on cpu_batch would be the wrong signal: it could fire while retained still reads those bytes. Reshapes, untyped_storage(), set_ aliases, NumPy base chains and memoryview slices can also outlive the original wrapper. The pool therefore waits for every export of the storage, including exports that escape a failed restore operation.

Suppose unit 0 is still held by a consumer while unit 1 has already been released. When the writer wraps around, it cannot skip unit 0 and treat unit 1’s release as permission to overwrite older storage. The reclaim point advances in order only through consecutive released units. This in-order rule trades some reuse flexibility for the guarantee that an earlier live view still sees its original values.

A SharedMemoryRingBuffer instead uses a shared first-in-first-out (FIFO) byte region. The receiver copies a unit into a new owning buffer before returning its ring span for immediate reuse. An output retained after that point owns the copy—not a live view of the source ring. Pinning changes that destination to a pinned allocator-owned buffer; it does not remove ring copy-out or make retained outputs consume the old ring span.

How the storage owner follows aliases

Native-backed arena views use the optional spdl.io.lib._buffer helper. The native buffer exporter implementation is the C++ Exporter introduced for these storage paths. Its immutable object implements the buffer protocol. For shared mappings it holds an actual Py_buffer export; for allocator-owned pinned outputs it holds a hidden reference to the allocation owner. A native storage consumer that retains this exporter keeps the underlying memory alive, even when the original Tensor or view wrapper disappears.

The built-in Torch restore path retains the exporter itself for complete contiguous binaries. Independent wrappers preserve offsets for slices. Derived NumPy and memoryview aliases retain their backing export too. Once the last exporter is destroyed, it releases the backing export/owner before notifying caller-side lease bookkeeping that storage can be reused or unmapped.

Each buffer export has its own native release ticket. Setup owns rollback until the native factory adopts that ticket before returning a successful view. After adoption, a Python return error is not permission to release the hold: the error’s traceback or an escaped alias may still own the actual exported storage. Only exporter destruction can retire the native-owned ticket, exactly once.

The exporter deliberately roots its callback and allocator graph until actual storage destruction. Otherwise cyclic GC could clear a Python partial or bound owner before the native destructor calls it. This is conservative: genuine callback ownership cycles can remain retained and may need to be broken by the caller. It is not a blanket promise of cycle reclamation or a Python-finalizer fallback.

This is why the protection is more than a Python wrapper finalizer. Supported Python 3.11 has no Python-defined buffer-export hooks; Python 3.12+ provides PEP 688 hooks, but a replacement would still need a validated storage-owner design.

The helper is loaded only at actual storage export/allocation. Ordinary pipeline use has no mandatory IO/native dependency, and unpinned ring copy-out needs no helper. Native pool views, including pageable ones, and pinned ring destinations require SPDL IO. Its existing NumPy Python dependency may load on that path, but creating the buffer exporter does not load Torch, CUDA or media-native extensions. There is no unsafe finalizer fallback if the helper is unavailable.

Closing local use is not unlinking a name

Close stops new local arena use; unlink removes the shared-memory name so that new processes cannot attach by that name. Removing a name is different from removing an already-mapped region. Existing exported views must stay readable until their storage owners are released.

For a pool, live exports postpone reuse and local unmapping even after close and unlink. The final export release can finish cleanup without reopening an already-closed control segment. A pinned pool must also unregister each original mapping before unmapping it. If unregister or transfer completion is uncertain, the owner is retained for retry rather than risking memory reuse while CUDA might still use it.

A ring output already owns its copy, so closing/unlinking the source ring does not invalidate that output. Retaining many such outputs can nevertheless retain many independent allocations. A fixed source-ring capacity does not bound all consumer-held output memory.

When an arena is handed to iterate_in_subprocess, its ownership transfers to the returned iterable, which closes/unlinks it at teardown. Do not share that same arena with another boundary or reuse it after teardown. Callers using the low-level endpoints manage close/unlink themselves and must keep the storage owners, not merely raw addresses.

CUDA work has its own completion rule

Keeping an exported CPU buffer alive is not proof that a CUDA copy has finished. The native exporter does not track DMA, CUDA streams or cross-process slot accounting. The synchronized spdl.io.transfer_tensor_h2d() retains the source, destination and transfer stream until completion, including failure paths. With raw nonblocking H2D, the caller must retain source owners until the copy completes; allocator bookkeeping alone does not protect an externally registered shared mapping. See Shared-memory arenas for stream and failed-cleanup behavior.

External torch.frombuffer consumers

SPDL’s built-in Torch restore path retains the immutable exporter directly. External torch.frombuffer(memoryview) instead retains the supplied Python memoryview object; it does not protect that wrapper against explicit release() or context-manager exit. Do not release a wrapper handed to an external consumer while its native aliases still use it.

For a complete arena binary, retaining the exporter directly avoids that particular wrapper-release hazard:

tensor = torch.frombuffer(arena_view.obj, dtype=torch.uint8)
arena_view.release()  # tensor's native storage still owns the exporter.

This is only for a complete binary: arena_view.obj describes the entire export, not a sliced view’s offset or shape. For slices/casts, keep the correct layout and supply an independent memoryview(sliced_view) that is not explicitly released while Torch uses it. Ordinary garbage collection of the original wrapper is safe when a valid owning export remains. An arbitrary raw pointer is not such an ownership reference.