What is Azure Data Factory?

Azure Data Factory (ADF) is the cloud-based ETL and data integration services that allows you to moving data between on-premises and cloud systems as well as scheduling and orchestrating complex data flows.

Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.

Key components of Azure Data Factory

• Pipelines
• Activities
• Datasets
• Linked services
• Data Flows
• Integration Runtimes

Let’s discuss all components with small description-

1- Pipelines: 

A data factory might have one or more pipelines. A pipeline is a logical grouping of activities that performs a unit of work.

Pipeline allows you to manage the activities as a set instead of managing each one individually.

Create a Pipeline

2- Activity

Activities represent a processing step in a pipeline. For example, you might use a copy activity to copy data from one data store to another data store.

Add copy data activity into Pipeline

3- Datasets

Datasets represent data structures within the data stores, which simply point to or reference the data you want to use in your activities as inputs or outputs.

Datasets ADF

4- Linked Services

The linked service has the connection information that the Data Factory service uses at runtime to connect to it.

New Linked Service ADF

5- Data Flow

Create and manage graphs of data transformation logic that you can use to transform any-sized data.

Data flows allow data engineers to develop data transformation logic without writing code.

Data Flow

6- Integration Runtimes

An integration runtime provides the bridge between the activity and linked Services. It’s referenced by the linked service or activity, and provides the compute environment where the activity either runs on or gets dispatched from.

Recommended Topics-

• What is Azure Data Factory?
• Creating Azure Data Factory
• Creating Linked Services In Azure Data Factory
• Create Datasets in Azure Data Factory
• Creating Your First Azure Data Factory Pipeline

For more detail about ADF refer Microsoft Documentation.

Recommend Azure Post: Microsoft Azure Topics

Hope you enjoyed the post. Your valuable feedback, question, or comments about this post are always welcome or you can leave us message on our Contact form , we will revert to you asap.