Azure Data Factory (ADF) is the cloud-based ETL and data integration services that allows you to moving data between on-premises and cloud systems as well as scheduling and orchestrating complex data flows.
Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.
Key components of Azure Data Factory
- Linked services
- Data Flows
- Integration Runtimes
Let’s discuss all components with small description-
A data factory might have one or more pipelines. A pipeline is a logical grouping of activities that performs a unit of work.
Pipeline allows you to manage the activities as a set instead of managing each one individually.
Activities represent a processing step in a pipeline. For example, you might use a copy activity to copy data from one data store to another data store.
Datasets represent data structures within the data stores, which simply point to or reference the data you want to use in your activities as inputs or outputs.
4- Linked Services
The linked service has the connection information that the Data Factory service uses at runtime to connect to it.
5- Data Flow
Create and manage graphs of data transformation logic that you can use to transform any-sized data.
Data flows allow data engineers to develop data transformation logic without writing code.
6- Integration Runtimes
An integration runtime provides the bridge between the activity and linked Services. It’s referenced by the linked service or activity, and provides the compute environment where the activity either runs on or gets dispatched from.
- What is Azure Data Factory?
- Creating Azure Data Factory
- Creating Linked Services In Azure Data Factory
- Create Datasets in Azure Data Factory
- Creating Your First Azure Data Factory Pipeline
For more detail about ADF refer Microsoft Documentation.
Recommend Azure Post: Microsoft Azure Topics
Hope you enjoyed the post. Your valuable feedback, question, or comments about this post are always welcome or you can leave us message on our Contact form , we will revert to you asap.