CI/CD for Databrickss
Databricks CI/CD is the practice of keeping pipeline code in Git and promoting it through dev, staging and production automatically. It sits in the delivery stage and commonly uses Databricks Asset Bundles, a databricks.yml file, the Databricks CLI and a runner such as GitHub Actions.
Stage: Orchestration and Delivery · Part of the Databricks roadmap · Last reviewed 19 November 2026
What to learn in CI/CD
- Describe a Databricks Asset Bundle in a databricks.yml file with targets
- Run databricks bundle validate and databricks bundle deploy from a terminal
- Write a GitHub Actions workflow that deploys a bundle on merge
- Run unit tests for PySpark code with pytest before deployment
- Separate dev and prod configuration using bundle targets and variables
CI/CD interview questions
What is a Databricks Asset Bundle?
A bundle is a project definition, written in YAML, that declares jobs, pipelines and other resources together with the source code they run. The Databricks CLI validates and deploys it to a target workspace, which makes deployments repeatable and reviewable in Git.
How would you test PySpark transformations before they reach production?
Structure logic as functions that take and return DataFrames, then test them with pytest against a small local or Databricks Connect session using fixture data. Run the tests in the CI step so a failing test blocks the bundle deployment.
Where CI/CD fits in the Databricks roadmap
CI/CD belongs to the Orchestration and Delivery stage (intermediate level). Production pipelines.
Other units in this stage
Back to the full Databricks roadmap