Databricks Roadmap 2026
Build lakehouse pipelines on Databricks: platform, ingestion, transformation, jobs, CI/CD, monitoring and governance with Unity Catalog.
4 stages · 7 units · Intermediate · about 70 hours · Last reviewed 19 November 2026
What is a databricks?
Databricks is a cloud data and AI platform built around the lakehouse idea, where one copy of data in open Delta Lake tables serves both analytics and machine learning. Teams use it to ingest data, transform it with Spark and SQL, run scheduled pipelines, and govern access through Unity Catalog on AWS, Azure or Google Cloud.
Step-by-step databricks roadmap
- 1
Platform
Beginner
Lakehouse basics.
- Concepts
Databricks concepts are the ideas behind the platform: the lakehouse model, workspaces, clusters and serverless compute, notebooks, and Delta Lake as the table layer on cloud storage. This unit sits at the start of the roadmap because every later topic, from ingestion to governance, builds on how these parts fit together.
- Concepts
- 2
Ingestion and Transformation
Intermediate
Medallion pipelines.
- Ingestion
Databricks ingestion is the work of getting raw data into Delta tables, from files in cloud storage, databases and event streams. It sits in the second stage and feeds the bronze layer of a medallion pipeline, usually through Auto Loader, COPY INTO or Structured Streaming sources.
- Transformation
Databricks transformation is the step that turns raw bronze data into cleaned silver tables and business-ready gold tables using Spark SQL, PySpark and declarative pipelines. It sits after ingestion and applies joins, deduplication, type fixes and aggregations while keeping the output as governed Delta tables.
- Ingestion
- 3
Orchestration and Delivery
Intermediate
Production pipelines.
- Jobs
Databricks jobs are the scheduled or triggered runs of notebooks, SQL, Python scripts and pipelines, managed through Lakeflow Jobs. Jobs sit in the orchestration stage and define tasks, dependencies, compute, retries and alerts so a pipeline runs without someone clicking Run.
- CI/CD
Databricks CI/CD is the practice of keeping pipeline code in Git and promoting it through dev, staging and production automatically. It sits in the delivery stage and commonly uses Databricks Asset Bundles, a databricks.yml file, the Databricks CLI and a runner such as GitHub Actions.
- Jobs
- 4
Operations and Governance
Advanced
Run safely.
- Operations
Databricks operations covers keeping pipelines healthy after launch: reading Spark UI and job run logs, diagnosing slow or failed runs, and tuning storage and compute. It sits in the advanced stage and uses tools such as OPTIMIZE, liquid clustering, system tables and cluster sizing to control performance and cost.
- Governance
Databricks governance is the control of who can see and change data, using Unity Catalog. It organises assets as catalog, schema and table, grants privileges to users and groups, records lineage and audit logs, and applies row filters and column masks. It is the final stage because it secures everything built earlier.
- Operations
Frequently asked questions
Ready to practise for interviews?
Practise databricks questions with an AI mock interviewer and get feedback on your answers.