The query performing a streaming hop from raw data to a Bronze table is identified by using the Spark streaming read capability and then writing to a Bronze table. Let's analyze the options:
Option A: Utilizes.writeStreambut performs a complete aggregation which is more characteristic of a roll-up into a summarized table rather than a hop into a Bronze table.
Option B: Also uses.writeStreambut calculates an average, which again does not typically represent the raw to Bronze transformation, which usually involves minimal transformations.
Option C: This uses a basic.writewith.mode("append")which is not a streaming operation, and hence not suitable for real-time streaming data transformation to a Bronze table.
Option D: It employsspark.readStream.load()to ingest raw data as a stream and then writes it out with.writeStream, which is a typical pattern for streaming data into a Bronze table where raw data is captured in real-time and minimal transformation is applied. This approach aligns with the concept of a Bronze table in a modern data architecture, where raw data is ingested continuously and stored in a more accessible format.
[References:Databricks documentation on Structured Streaming: Structured Streaming in Databricks, , , ]
Submit