spdl.pipeline.run_pipeline_in_subprocess¶
- run_pipeline_in_subprocess(builder: PipelineBuilder[T, U], *, num_threads: int, max_failures: int = -1, report_stats_interval: float = -1, queue_class: type[AsyncQueue[T]] | None = None, task_hook_factory: Callable[[str], list[TaskHook]] | None = None, **kwargs: dict[str, Any]) Iterator[T][source]¶
Run the given Pipeline in a subprocess, and iterate on the result.
- Parameters:
builder – The definition of
Pipeline.num_threads – Passed to
PipelineBuilder.build().max_failures – Passed to
PipelineBuilder.build().report_stats_interval – Passed to
PipelineBuilder.build().queue_class – Passed to
PipelineBuilder.build().task_hook_factory – Passed to
PipelineBuilder.build().kwargs – Passed to
iterate_in_subprocess().
- Yields:
The results yielded from the pipeline.