1. Course Syllabus

   1. Module 1: Introduction to the Databricks Lakehouse Platform (3h)
   2. Covers the Databricks Lakehouse architecture and workspace. Learners explore the Databricks UI, create clusters and notebooks, and understand how Databricks unifies data warehousing and data lakes. Key concepts include the Lakehouse platform architecture, workspace organization, and core services (Spark, Delta, MLflow). This module lays the groundwork by explaining Databricks’ capabilities and how they support end-to-end data engineering.
   3. Module 2: Databricks Workspace & Notebooks (2h)
   4. Introduces the interactive Databricks development environment. Participants learn to use Databricks notebooks — the primary tool for data workflows on Databricks. We cover creating and managing notebooks, real-time collaboration, and built-in visualizations. Notebooks support Python, SQL, Scala (and R) for data analysis. The module also covers basic workspace tasks: managing clusters, libraries, and Git integration in Databricks.
   5. Module 3: Apache Spark Fundamentals (Python & Scala) (4h)
   6. Covers the basics of Apache Spark in Databricks. Learners work with Spark RDDs and DataFrames using both PySpark and Scala, understand lazy evaluation, and perform transformations and actions on data. They practice loading data into Spark, performing aggregations, joins, and writing data back to storage. This module builds core Spark skills needed for higher-level tasks.
   7. Module 4: Spark SQL and DataFrames (3h)
   8. Focuses on using Spark’s SQL interface and DataFrame API for data analysis. Learners write Spark SQL queries to create tables, views, and execute joins/aggregations. They also perform equivalent operations with DataFrame code in Python and Scala. Databricks emphasizes SQL for data tasks (for example, the Associate exam will present SQL whenever possible), so this module ensures proficiency in Spark SQL. Topics include filtering, grouping, and caching DataFrames for performance.
   9. Module 5: Advanced Spark – Streaming and ML (3h)
   10. Explores advanced Spark capabilities. For streaming, we introduce Spark Structured Streaming for real-time data processing. Using Delta Lake with streaming, students see how a single Delta table can support both batch and streaming use cases (enabling incremental processing). We also cover Spark’s MLlib basics: building and evaluating simple machine learning models. This module may touch on using Delta Live Tables (Lakeflow) for managed streaming pipelines.
   11. Module 6: Delta Lake Fundamentals (3h)
   12. Covers Delta Lake – Databricks’ transactional storage layer. Learners see how Delta Lake provides ACID transactions, scalable metadata handling, and schema enforcement on top of cloud storage. Through hands-on examples, they create Delta tables, perform versioned writes, and use time travel to query historical data. This module explains why Delta is the default format in Databricks and how it underpins reliable data pipelines.
   13. Module 7: Building ETL Pipelines with Delta Lake (4h)
   14. Teaches end-to-end ETL pipeline development on Databricks. Using Spark SQL and Python, students ingest data from diverse sources, apply transformations, and write results to Delta tables. This aligns with the exam emphasis on ELT tasks in Spark SQL and Python (29% of exam topics). Learners practice partitioning and optimizing data, and learn to use Databricks Jobs/Workflows to orchestrate multi-step pipelines. By the end, they can build robust batch pipelines in Databricks.
   15. Module 8: Incremental Data Processing & Streaming Pipelines (3h)
   16. Covers continuous data pipelines and incremental processing. Students build Structured Streaming applications on Databricks, reading from streams and writing to Delta Lake in real time. Databricks optimizes Delta Lake for Structured Streaming, so learners see how to implement change-data capture and event-driven ETL (using Auto Loader, Structured Streaming APIs, or Delta Live Tables). This module demonstrates how to keep downstream tables updated with new data continuously.
   17. Module 9: Databricks Workflows and Jobs (2h)
   18. Teaches how to schedule and automate data pipelines. Using Databricks Workflows (Jobs UI), learners schedule notebooks and tasks to run on a timetable or by trigger. They create multi-job and multi-notebook workflows, set job parameters, and handle dependencies. As Databricks notes, notebooks can be “scheduled regularly” to run tasks and even chained together.. This module shows how to move pipelines into a production setting on Databricks.
   19. Module 10: Data Governance with Unity Catalog (2h)
   20. Focuses on Databricks Unity Catalog for data governance. Learners study the Unity Catalog architecture and use it to secure and manage data assets. Topics include creating catalogs, schemas, and tables; setting access control permissions on data; managing external tables; and data isolation techniques. This module uses labs on Databricks’ own governance solution (Unity Catalog) to ensure data security and compliance in the Lakehouse.
   21. Module 11: MLflow & Model Lifecycle Management (3h)
   22. Introduces MLflow on Databricks for managing machine learning projects. Students learn to track experiments, log parameters and metrics, and register models with MLflow. As Databricks training emphasizes, MLflow enables building end-to-end ML workflows: “Use MLflow to track the ML lifecycle, package models for deployment and manage model versions”. Hands-on exercises include training a model in Databricks, logging it in MLflow, and preparing it for deployment or further evaluation.
   23. Module 12: Databricks SQL for Data Analytics (3h)
   24. Covers the Databricks SQL workspace for analysts. Learners practice ingesting data and writing SQL queries to create dashboards and visualizations. This module mirrors the Databricks SQL courses: students “ingest data, write queries, produce visualizations and dashboards” in the Databricks SQL UI. They learn to set up alerts, use SQL functions, and build interactive dashboards – preparing for real-world analytics tasks and the Databricks Data Analyst Associate exam.
   25. Module 13: Performance Tuning & Best Practices (1h)
   26. Introduces key optimization techniques for Spark jobs on Databricks. Topics include caching strategies, partitioning and bucketing data, broadcast joins, and monitoring jobs in the Spark UI. Learners practice tuning example queries and pipelines for better performance. This module consolidates best practices that help make data jobs efficient at scale.
   27. Module 14: Capstone Project & Certification Prep (4h)
   28. In the final module, learners apply all skills in a comprehensive project. For example, they might ingest raw data, build a multi-stage ETL pipeline into Delta Lake, enable streaming updates, apply security via Unity Catalog, and train a simple model with MLflow. The capstone simulates a real-world use case end-to-end. We conclude with review sessions covering the official exam topics (reinforcing Spark SQL/Python, pipelines, governance, etc.) and provide practice questions. This ensures learners are ready to take the Databricks Data Engineer Associate certification.

Key Features

  Hands-on Guided Labs: Includes interactive Databricks Academy labs and tutorials that simulate real Databricks environments and tasks.

  Real-World Projects: Multiple capstone and practice projects give learners practical experience with data pipelines and analytics on Databricks.

  Certification Alignment: Curriculum covers the key topics of the Databricks Data Engineer Associate exam (Spark SQL, Python, Delta Lake, streaming, pipelines, governance), ensuring direct preparation for certification.

  Flexible Learning Formats: Offers a mix of self-paced video courses, instructor-led classes, and blended learning to suit different needs.

  Cloud Platform Integration: Teaches Databricks on all major clouds (AWS, Azure, GCP), tightly integrated with cloud storage and services.

  Multi-Language Development: Exercises use Python, Scala, and SQL throughout to reflect Databricks’ multi-language support (notebooks support Python, SQL, Scala, R).