spdl.pipeline.TaskStatsHook

class TaskStatsHook(
name: str,
concurrency: int,
interval: float = -1
)[source]

Track the task runtimes and success rate.

Parameters:

  • name – Nmae of the stage. Only used for logging.

  • concurrency – Concurrency of the stage. Only used for logging.

Methods

stage_hook()

Track the stage runtime and log the task stats.

task_hook()

Track task runtime and success rate.

Attributes

num_failures

property num_failures: int[source]
stage_hook() → AsyncIterator[None][source]

Track the stage runtime and log the task stats.

task_hook() → AsyncIterator[None][source]

Track task runtime and success rate.