The data processing module is structured around 3 objects:
- Dataflow: This object is used to hierarchize data processing. It cannot link elements of the dictionary together, this is why it's not possible to describe data processing in it.
- Data processing: It's used to describe links between one or many objets from the dictionnary (databases, table, view...). Links are described with data processing units.
- Data processing element: It is the most detailed element of the module, it allows to describe the link operated between different objects of the dictionary (one to x bases, containers, tables, fields...). It can be a simple link, a transformation, a filter...
For example, here is a data processing ("product load") that load informations (Bill of material, product category...) from Azure Datalake into the Enterprise Datawarehouse. As we can see:
- One data processing can contain many data processings elements: linking as many inputs to as many output. Each data processing element can explain ing one transformation
- A data flow (here "Azure to EDWH") can contain many data processings.
