Collect the required data from the appropriate data sources and move the data to a single centralized data store in the cloud.
Why?
Because the data might be located in different locations, such as on-premises or in the cloud. The data might also arrive at different speed and intervals.
Steps
- Read data from source data store
- Perform processing, such as:
- Serialization/deserialization
- Compression/decompression
- Column mapping
- Write data to the destination data store (sink)