Build Data Pipelines with Delta Live Tables

Learning Track

Data Engineering

Delivery methods

On-Site, Virtual

Duration

1 day

In this course, you’ll learn how to define and schedule data pipelines that incrementally ingest and process data through multiple tables in the lakehouse using Delta Live Tables (DLT) in Spark SQL and Python. The course covers how to get started with DLT, how DLT tracks data dependencies in data pipelines, how to configure and run data pipelines using the Delta Live Tables UI, how to use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables in the lakehouse using Auto Loader and DLT, how to use APPLY CHANGES INTO syntax to process Change Data Capture feeds, and how to review event logs and data artifacts created by pipelines and troubleshoot DLT syntax.

Objectives

By the end of this course, attendees should be able to:

Describe how Delta Live Tables tracks data dependencies in data pipelines
Configure and run data pipelines using the Delta Live Tables UI
Use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables in the lakehouse using Auto Loader and Delta Live Tables
Use APPLY CHANGES INTO syntax to process Change Data Capture feeds
Review event logs and data artifacts created by pipelines and troubleshoot DLT syntax

Prerequisites

Beginner-level familiarity with basic cloud concepts (virtual machines, object storage, identity management)
Ability to perform basic code development tasks (create compute, run code in notebooks, use basic notebook operations, import repos from git, etc)
Intermediate familiarity with basic SQL concepts (CREATE, SELECT, INSERT, UPDATE, DELETE, WHILE, GROUP BY, JOIN, etc.)