spdl.pipeline.defs¶

Overview¶

This module provides building block definitions for Pipeline.

You can build a pipeline by creating a PipelineConfig, then passing it to spdl.pipeline.build_pipeline() function.

The following step explains the steps to build a pipeline using the building blocks.

Instantiate a SourceConfig by providing a Iterable or AsyncIterable.
Using Pipe(), Aggregate() or Disaggregate(), instantiate a series of PipeConfig.
Instantiate a SinkConfig.
Pass the config objects created in the previous steps to PipelineConfig.
Pass the PipelineConfig object to spdl.pipeline.build_pipeline() function.