Tabul - Data Fill (with generated data)

Undraw Windows

Tabul - Data Fill (with generated data)

About

The Tabul data fill command from the data module insert generated data into a content resource (file, sql table).

How do the fill command maps generators to target ?

If you specify the --generator-selector, you may select more than one generator.

These generators are mapped to the target:

  • by their logical name for a multi-fill operation
  • by position for a mono-fill operation (one generator, one target)

Syntax

tabul data fill --help
Tabul data fill
===============

Load generated data into data resources.

You select the data resources to be filled with:
   * the data selector argument (`target-selector...`)
   * and optionally the dependency option (`--with-dependencies`)

For more control on the data generated, you can add the data resource generators (`*--generator.yml`)
by selecting them with the generator selector option (--generator-selector)

This is an alias to the `transfer` command where:
  * the targets are the selected data resources
  * the sources are generated from the generator data resources and/of target metadatas

The default transfer operation is `upsert`.



Examples
--------

 1 - To load the table `D_TIME` from the connection `sqlite` with auto-generated data:


    tabul data fill D_TIME@sqlite


 2 - To load the table `D_TIME` with the data generation file `D_TIME--generator.yml` present in the current directory:


    tabul data fill --generator-selector D_TIME* D_TIME@connection


 3 - To load all data in the whole schema with the data generation file in the dir directory:


    tabul data fill --generator-selector dir/* *@connection


 4 - Load auto generated data into the table F_SALES and its dependencies


    tabul data fill --with-dependencies F_SALES@sqlite




Syntax
------


    tabul data fill [options|flags] <target-selector...>


where:


  Arguments:

    <target-selector...>                                          One or more data selectors that will select the target data resources to be filled.


  Cross Data Transfer Options:

    -bs,--buffer-size <buffer-size>                               defines the size of the memory buffer between the source and target threads

    -mdu,--metrics-data-uri <metrics-data-uri>                    defines a target data uri where the data metrics should be exported

    -out,--output-type <output-type>                              The resource that is passed as output

    -pt,--processing-type <processing-type>                       how to process the inputs (one by one or in batch)

    -sfs,--source-fetch-size <source-fetch-size>                  defines the size of the network message from the source to fetch the data

    -tbs,--target-batch-size <target-batch-size>                  defines the batch size against the target data resource

    -tcf,--target-commit-frequency <target-commit-frequency>      defines the commit frequency in number of batches against the target data resource

    -to,--target-operation <target-operation>                     defines the data operations (drop or truncate) on the existing target before transfer. A `replace` operation will drop the target.

    -twc,--target-worker-count <target-worker-count>              defines the target number of thread against the target connection

    -tmc,--transfer-mapping-columns <transfer-mapping-columns>    defines the columns mapping between the source and the target

    -tmm,--transfer-mapping-method <transfer-mapping-method>      defines the method used to map the source columns to the target columns

    -tms,--transfer-mapping-strict <transfer-mapping-strict>      defines if a map by name or position is strict

    -op,--transfer-operation <transfer-operation>                 defines the transfer operation (insert, update, delete, upsert, merge, copy).

    -tut,--transfer-upsert-type <transfer-upsert-type>            defines the type of upsert operation (merge, insert, insert-update, update-insert).

    -wp,--with-parameters                                         defines if parameters are used in the SQL statement


  Fill Options:

    -gs,--generator-selector <generatorSelector...>               One or more data selector that selects one or more data resource generator ( --generator.yml )

    -mrc,--max-record-count <maxRecordCount>                      This option defines the maximum total number of record that the data resource(s) must have when no data resource generator was found.


  Data Definition Options:

    -ta,--target-attribute <attributeName=value>                  Set a target attribute


  Selection Options:

    -wd,--with-dependencies                                       If this flag is present, the dependencies of the selected target tables (ie parent/foreign tables) will be also be filled with data


  Global Options:

    -ah,--app-home <path>                                         The app home directory (default to the .tabul.yml file directory)

    -vf,--conf <path>                                             The path to a configuration file

    -ee,--exec-env <name>                                         The execution environment (prod or dev)

    -h,--help                                                     Print this help

    -l,--log-level <error|warning|tip|info|fine>                  Set the log level

    -ns,--not-strict                                              A minor error will not stop the process.

    -odu,--output-data-uri <outputDataUri>                        defines the output data uri for the feedback data (default: console)

    -oo,--output-operation <dataOperation>                        defines the data operations (replace, truncate) on an existing output resource before transfer.

    -oop,--output-transfer-operation <transferOperation>          defines the output transfer operation (insert, update, merge, copy). Default to `copy` for a file system and `insert` for a database.

    -pp,--passphrase <passphrase>                                 A passphrase (master password) to decrypt the encrypted vault values (Env: TABUL_PASSPHRASE)

    --pipe-mode                                                   Use pipe mode if you want to pipe the output in a shell. Pipe mode will not print the headers (ie column name) and will not make the control character visible

    -v,--version                                                  Print version information




Related HowTo
Undraw Windows
How to read and write an Excel file?

This howto will demonstrate you how to read and write to an Excel resource. To following this howto, you should have followed the getting started guide to have knowledge of: Resource creation...
Undraw Windows
Learning Tabulify - Step 9 - How to fill a data resource with generated data ?

Tabulify integrates natively a data generator. You can generate realistic production data and start working on your project right away. anonymize production data in your development environment because...
Undraw Windows
SQLite - How to fill a table with a resource data generator?

This how-to will show you how to define the generation of data via a data definition file and load it into a table via the tabul data fill operation.
Undraw Windows
SQLite - How to fill a table with auto generated data

This how-to will show you how to fill a table with auto-generated data and the Tabul fill action.
Undraw Windows
Tabul - How to fill a table with a data generation file

This how-to will show you how to define the generation of data via a data definition file and load it into a table via the tabul data fill operation.
Undraw Windows
Tabul - How to fill a table with auto generated data

This how-to will show you how to fill a table with auto-generated data and the Tabul fill action.
Undraw Windows
Tabulify - How to fill a database relational schema with generated data and Tabul

This how-to will show you how to use the tabul fill command to fill all the tables of a schema with generated data.

Task Runner