define is a supplier operation that defines one or more inline data resources in a pipeline file.
Note that a select operation may not return any data resources but the define operation must return always the data resource that it defines.
This is a first step of a pipeline (known as the supplier step) because it supplies data resources to the others operations (known as intermediate) in the pipeline.
The define operation accepts the following arguments.
| Arguments | Default | Definition |
|---|---|---|
| data-resource | One data_resource_definition | |
| data-resources | A list of data_resource_definition | |
| processing-type | batch | The processing type (batch or stream) |
The following argument defines one data resource and can be set:
You can define a data resource by:
but not both
| Arguments | Mandatory | Definition |
|---|---|---|
| One resource definition by Data Uri | ||
| data-uri | only if data is not defined | The data uri is the location of the resource |
| media-type | The media type of the data uri resource | |
| One resource definition by Data | ||
| data | only if data-uri is not defined | The data as a list of records (the data are in the inline) |
| The optional data attributes | ||
| data-def | no | The data attributes in a data definition format |