Pipeline Step

Step

A step (operation step) is an element of a pipeline that executes a data operation.

Syntax

The syntax of steps in a pipeline manifest is the following:

kind: pipeline
spec:
  # a list of steps
  steps:
        # the name of the operation to perform
      - operation: "firstSupplierOperationName"
        # The arguments to pass to the operation
        args:
            arg1:  value1 # the first argument
            arg2:  value2 # the second argument
            ....
            argN:  valueN # the Nieme argument
        # An optional name (identifier that should not contains any space)
        # Default to step + id in the list
        name: "name"
        # An optional comment
        comment: "A long or short command/description of the step"
      - operation: "secondIntermediateOperationName"
        ....

Type

There is 2 type of steps:

• supplier step - the first step in a pipeline that supplies the pipeline in data resources
• intermediate step - the next steps in a pipeline that transform the data resources stream