enrich is an intermediate operation that will add virtual columns to its inputs thanks to data supplier.
Enrich accepts only one argument data-def where you can define extra columns called virtual columns with their respective data supplier.
A data supplier is a function that supplies a value to a column.
In the steps below, we add to the input resources:
In this example, we will showcase the enrich operation with the enrich-me.md
With the cat command
tabul data cat pipeline/enrich/enrich-me.md@howto
We can see the content:
This is a file used in the [enrich pipeline](../enrich.yml)
for demonstration
In this example, the pipeline:
kind: pipeline
spec:
steps:
- operation: 'define'
arguments:
data-resource:
data-uri: 'pipeline/enrich/enrich-me.md@howto'
- operation: 'enrich'
arguments:
data-def:
columns:
- name: file_name
data-supplier:
type: meta
arguments:
attribute: name
- name: file_extension
data-supplier:
type: expression
arguments:
column-variable: file_name
expression: "file_name.split('.').pop()"
- name: line_id
type: integer
data-supplier:
type: sequence
- operation: 'print'
By executing it, we can see the 4 columns created.
tabul flow execute --no-results pipeline/enrich.yml@howto
pipeline/enrich/enrich-me.md@howto
lines file_name file_extension line_id
----------------------------------------------------------- ------------ -------------- -------
This is a file used in the [enrich pipeline](../enrich.yml) enrich-me.md md 1
for demonstration enrich-me.md md 2