Summer Certification Special Limited Time 70% Discount Offer - Ends in 0d 00h 00m 00s - Coupon code: force70

Databricks Certified Data Engineer Associate Exam Databricks-Certified-Data-Engineer-Associate Question # 42 Topic 5 Discussion

Databricks Certified Data Engineer Associate Exam Databricks-Certified-Data-Engineer-Associate Question # 42 Topic 5 Discussion

Databricks-Certified-Data-Engineer-Associate Exam Topic 5 Question 42 Discussion:
Question #: 42
Topic #: 5

A data engineer is designing a cost-optimized, event-driven pipeline. They configure a Lakeflow Job with a File Arrival trigger to watch an Amazon S3 bucket. The job runs a notebook that uses Auto Loader with trigger(availableNow=True) to ingest data into a Bronze table.

What is the technical relationship between the File Arrival trigger and Auto Loader in this integration pattern?


A.

The File Arrival trigger starts the job run, while Auto Loader uses its internal checkpoint to independently identify and process only the new files that have arrived since the last successful commit.


B.

Auto Loader must be configured in File Notification mode to work with a Lakeflow Job File Arrival trigger.


C.

Using a File Arrival trigger requires the engineer to disable Auto Loader checkpointing to prevent the job from processing the same file multiple times.


D.

The File Arrival trigger automatically passes the specific file path of the new arrival to Auto Loader, allowing the engineer to omit the source-path configuration from the code.


Get Premium Databricks-Certified-Data-Engineer-Associate Questions

Contribute your Thoughts:


Chosen Answer:
This is a voting comment (?). It is better to Upvote an existing comment if you don't have anything to add.