For interactive notebook development, Databricks distinguishes between all-purpose compute and job compute . All-purpose compute is designed for analysis, notebook development, and interactive workloads , while job compute is intended for automated scheduled or triggered jobs. Because the engineer is actively developing a proof of concept in a notebook and wants real-time feedback, an all-purpose cluster is the right compute type. Adding autoscaling makes it better suited to the observed usage spikes, because the cluster can expand when notebook execution demands more resources and scale down afterward, reducing waste compared with a large fixed-size cluster. That makes option B the best answer. Option A keeps the interactive development model, but the fixed large size is less efficient. Options C and D use job clusters, which are better aligned with production jobs rather than iterative notebook-based development. Databricks documentation consistently positions all-purpose compute for collaborative and interactive development, and autoscaling is a standard mechanism to handle variable workloads more efficiently.

=========