spdl.pipeline.StatsQueue

class StatsQueue(name: str, *, buffer_size: int = 1, interval: float = -1)

Measures the time stages are blocked on upstream/downstream stage. Extends AsyncQueue.

See also

• Collecting Runtime Statistics : How to export the runtime performance statistics.

• Analyzing the Performance : How to use the exported stats.

Parameters:

• name – The name of the queue. Assigned by PipelineBuilder.

• buffer_size – The buffer size. Assigned by PipelineBuilder.

• interval – The interval (in second) between reporting performance numbers to console.

Methods

empty()	Return True if the queue is empty, False otherwise.
full()	Return True if there are maxsize items in the queue.
get()	Remove and return an item from the queue, track the time.
get_nowait()	Remove and return an item from the queue.
interval_stats_callback(stats)	Callback for processing interval performance statistics.
join()	Block until all items in the queue have been gotten and processed.
put(item)	Remove and return an item from the queue, track the time.
put_nowait(item)	Put an item into the queue without blocking.
qsize()	Number of items in the queue.
stage_hook()	Context manager, which handles init/final logic for the stage.
task_done()	Indicate that a formerly enqueued task is complete.

Attributes

maxsize

Number of items allowed in the queue.

async get() → T

Remove and return an item from the queue, track the time.

async interval_stats_callback(stats: QueuePerfStats) → None

Callback for processing interval performance statistics.

When interval reporting is enabled, this method is periodically called with the delta metrics.

The default behavior is to log the metrics to console.

You can override this method and modify the destination of the log.

async put(item: T) → None

Remove and return an item from the queue, track the time.

stage_hook() → AsyncIterator[None]

Context manager, which handles init/final logic for the stage.