spdl.pipeline.defs.PipelineConfig¶

class PipelineConfig(src: SourceConfig[Any] | MergeConfig, pipes: Sequence[PipeConfig[Any, Any] | AggregateConfig[Any] | DisaggregateConfig[Any]], sink: SinkConfig[U])[source]¶

A pipeline configuration.

A pipeline consists of source, a series of pipes and sink.

You can use spdl.pipeline.build_pipeline() to build a spdl.pipeline.Pipeline object.

See also

Example: Pipeline definitions: Illustrates how to build a complex pipeline.

Attributes

`src`	Source configuration.
`pipes`	Pipe configurations.
`sink`	Sink configuration.

pipes: Sequence[PipeConfig[Any, Any] | AggregateConfig[Any] | DisaggregateConfig[Any]]¶: Pipe configurations.

sink: SinkConfig[U]¶: Sink configuration.

src: SourceConfig[Any] | MergeConfig¶: Source configuration.