Generate Supplier
About
generate is a supplier data operation that supplies data resources with generated data.
Arguments
| Name | Default | Description |
|---|---|---|
| data-selector | - | Select data resources by glob pattern |
| data-selectors | - | A list of data-selectors |
| strict-selection | true | When set to true, the select operation will throw an error if the selection does not return any data resource |
| processing-type | stream | The processing type (stream or batch) |
| stream-record-count | The number of records by data resource generated if the stream-granularity is set to record | |
| stream-granularity | record | The stream granularity |
And the stream arguments if the processing-type is set to stream.
Type
generate supports the following supplier type
Stream
In a stream processing, this supplier create data resources at the push interval
If the granulity is:
- record: new data resources are created from the same generator with stream-record-count records
- resource: the generator resources are pushed in the pipeline
Batch
In batch mode, the generator data resources are directly provided. It's equivalent / an alias of the select supplier