Basics for Apache Sparks
Spark basics covers the architecture and core abstractions of the engine: a driver program that schedules tasks on executors, RDDs as the low-level distributed collection, and DataFrames with Spark SQL as the structured API. It is the first stage of the roadmap, because every later topic builds on lazy transformations and actions.
Stage: Core Spark · Part of the Apache Spark roadmap · Last reviewed 19 November 2026
What to learn in Basics
- Explain how the driver, executors and cluster manager cooperate to run one job
- Create a DataFrame from a CSV or Parquet file and select, filter and group its columns
- Tell transformations from actions and predict when Spark actually runs the work
- Register a temporary view and answer a question with spark.sql
- Choose between an RDD and a DataFrame for a given task
Basics interview questions
Why are Spark transformations lazy?
Spark records transformations as a lineage graph and only runs it when an action such as count, collect or write is called. This lets the Catalyst optimizer see the whole plan, prune columns, push filters down and combine steps, and it lets Spark recompute lost partitions from lineage.
What is the difference between an RDD and a DataFrame?
An RDD is a distributed collection of objects with no schema, so Spark cannot optimise inside your functions. A DataFrame has named, typed columns, so Catalyst and the Tungsten engine can plan and execute it more efficiently. DataFrames are the default choice, while RDDs give low-level control.
Where Basics fits in the Apache Spark roadmap
Basics belongs to the Core Spark stage (beginner level), which takes about 23 hours. The programming model.
Other units in this stage
Back to the full Apache Spark roadmap