Home » Microsoft Azure » What is Azure Data Factory?

What is Azure Data Factory?

ADF & Components

Azure Data Factory (ADF) is the cloud-based ETL and data integration services that allows you to moving data between on-premises and cloud systems as well as scheduling and orchestrating complex data flows.

Using Azure Data Factory, you can create and schedule data-driven workflows (called pipelines) that can ingest data from disparate data stores.

Key components of Azure Data Factory

  • Pipelines
  • Activities
  • Datasets
  • Linked services
  • Data Flows
  • Integration Runtimes

Let’s discuss all components with small description-

1- Pipelines: 

A data factory might have one or more pipelines. A pipeline is a logical grouping of activities that performs a unit of work.

Pipeline allows you to manage the activities as a set instead of managing each one individually.

Create a Pipeline

Create a Pipeline

2- Activity

Activities represent a processing step in a pipeline. For example, you might use a copy activity to copy data from one data store to another data store.

Add copy data activity into Pipeline

Add copy data activity into Pipeline

3- Datasets

Datasets represent data structures within the data stores, which simply point to or reference the data you want to use in your activities as inputs or outputs.

Datasets ADF

Datasets ADF

4- Linked Services

The linked service has the connection information that the Data Factory service uses at runtime to connect to it.

New Linked Service ADF

New Linked Service ADF

5- Data Flow

Create and manage graphs of data transformation logic that you can use to transform any-sized data.

Data flows allow data engineers to develop data transformation logic without writing code.

Data Flow

Data Flow

6- Integration Runtimes

An integration runtime provides the bridge between the activity and linked Services. It’s referenced by the linked service or activity, and provides the compute environment where the activity either runs on or gets dispatched from.

Recommended Topics-

For more detail about ADF refer Microsoft Documentation.

Recommend Azure Post: Microsoft Azure Topics

Hope you enjoyed the post. Your valuable feedback, question, or comments about this post are always welcome or you can leave us message on our Contact form , we will revert to you asap.

1 thought on “What is Azure Data Factory?”

Leave a Reply