Skip to content

Pipeline Graph

PipelineGraph is the resolved dependency view of a live MCPipeline. A pipeline is authored as an ordered sequence of MCStep objects, but each step also names the upstream data it consumes. The graph resolves those references once so serializers and graph consumers do not re-infer wiring from kwargs.

InputEdge

@dataclass(frozen=True)
class InputEdge()

One resolved input dependency of a step.

Fields:

Name Type Description
role str Consumer leg such as "source", "residual_source", "y_source", or "X_source".
producer str Upstream step name.
channel str Channel read from the producer, such as "observables", "std_innov", or "payload".

PipelineNode

@dataclass(frozen=True)
class PipelineNode()

A live MCStep plus its resolved graph location.

Fields:

Name Type Description
step MCStep Underlying runtime step.
inputs tuple[InputEdge, ...] Resolved upstream dependencies.
children tuple[str, ...] Downstream step names.

Properties:

Name Type Description
name str Step name.
step_type str \| None Serializable step kind when known.
is_root bool True when the node has no inputs.
is_leaf bool True when no other node consumes this node.
parents tuple[str, ...] De-duplicated producer names in leg order.
primary_parent str \| None Structural parent used when serializing the graph spec.

PipelineGraph

class PipelineGraph()

The dependency DAG of a pipeline, keyed by step name.

Fields:

Name Type Description
nodes dict[str, PipelineNode] Nodes keyed by step name.
root str Root datagen step name.
order tuple[str, ...] Authored execution order, validated as a topological order.

Methods:

PipelineGraph.from_steps(steps: tuple[MCStep, ...]) -> PipelineGraph  # @classmethod
PipelineGraph.edges() -> list[tuple[str, str]]

Properties:

Name Type Description
leaves tuple[PipelineNode, ...] Nodes with no downstream consumers.
Custom source channels

Graph resolution understands the built-in source conventions. Custom operations can still run in-process, but bundle-safe graph serialization requires the step to declare a supported step_type and source payload convention.