fill is an intermediate data operation that creates automatically data resource generator for its inputs.
When passed to the transfer operation, you will fill the defined target.
Optionally, you can define your own generator resource with the generator-selectors arguments to take control over the generated data for a specific resource
This data operation is available via the tabul data fill command
The arguments are:
| Name | Default | Description |
|---|---|---|
| max-record-count | 100 | The maximum of record generated for generators created by the fill operation |
| generator-selectors | A list of data selector that selects generator files created manually |
If you want to load dependent data resources, you need to set the the order argument of the select operation to create.
Why? Because the foreign tables needs to be loaded first.
For instance, when loading a star schema, the dimension needs to be loaded before the fact tables.
The generator files selected with the generator-selectors are mapped to the inputs:
Example: The generator file name d_time-datagen.yml would be mapped to a data resource with the logical name d_time. For instance: