There are four, half-day modules that make up this course. Please see the learning objectives for each of these modules, below:

Data Ingestion with Delta Lake

• Navigate and use the Databricks Data Science and Engineering Workspace for code development tasks.
• Utilize Spark SQL and PySpark to extract data from various sources.
• Apply common data cleaning transformations using Spark SQL and PySpark.
• Manipulate complex data structures with advanced functions in Spark SQL and PySpark.

Deploy Workloads with Databricks Workflows

• Orchestrate tasks with Databricks Workflow Jobs.
• Use Databricks SQL for on-demand queries.
• Configure and schedule dashboards and alerts to reflect updates to production data pipelines.

Build Data Pipelines with Delta Live Tables

• Describe how Delta Live Tables tracks data dependencies in data pipelines
• Configure and run data pipelines using the Delta Live Tables UI
• Use Python or Spark SQL to define data pipelines that ingest and process data through multiple tables in the lakehouse using Auto Loader and Delta Live Tables
• Use APPLY CHANGES INTO syntax to process Change Data Capture feeds
• Review event logs and data artifacts created by pipelines and troubleshoot DLT syntax

Data Management and Governance with Unity Catalog

• Explain the importance of data governance and challenges in traditional data lake environments.
• Differentiate between managed and external tables, and evaluate the architecture of Unity Catalog.
• Utilize SQL commands to navigate and inspect metastore components, and assess data segregation strategies.
• Identify query lifecycle steps and Databricks roles for effective data governance and security within Unity Catalog.
• Implement privilege assignments and fine-grained access control strategies using SQL syntax and dynamic views in Databricks.
• Assess the effectiveness and implications of different privilege scenarios, inheritance models, and access control mechanisms in Unity Catalog.

• Beginner familiarity with basic cloud concepts (virtual machines, object storage, identity management)

• Ability to perform basic code development tasks (create compute, run code in notebooks, use basic notebook operations, import repos from git, etc.)

• Intermediate familiarity with basic SQL concepts (CREATE, SELECT, INSERT, UPDATE, DELETE, WHILE, GROUP BY, JOIN, etc.)

• Intermediate experience with basic SQL concepts such as SQL commands, aggregate functions, filters and sorting, indexes, tables, and views.

• Basic knowledge of Python programming, jupyter notebook interface, and PySpark fundamentals.