The workflow exhibits two distinct architectural weaknesses: unnecessary sequential dependencies and the absence of durable execution checkpoints.
Option A addresses the scheduling problem by replacing a fixed sequential chain with dependency-aware task orchestration. In a dependency graph, each processing stage executes when its actual upstream requirements are satisfied. Independent stages can run concurrently, even when an unrelated source remains unavailable.
Databricks Lakeflow Jobs supports directed acyclic graphs, parallel task execution, and conditional dependencies. These capabilities allow the pipeline to preserve necessary execution ordering without imposing artificial dependencies between unrelated processing stages.
Option E addresses recovery. Durable checkpoints preserve completed outputs and execution status, enabling the workflow to resume from the last valid state following an interruption.
Databricks supports repairing failed multitask jobs without rerunning previously successful tasks. However, failed tasks may restart from their beginning, so application-level checkpointing and idempotent writes remain necessary.
Option B increases tolerance for delays but does not eliminate redundant execution. Option C introduces event-driven triggering without necessarily resolving dependencies or lost progress. Option D incorrectly permits dependent stages to execute simultaneously.
Combining dependency-aware orchestration with checkpointing improves throughput, fault recovery, and execution consistency throughout the clinical trial processing lifecycle.
Official Databricks reference: Configure task dependencies, Control the flow of tasks within Lakeflow Jobs, and Troubleshoot and repair job failures.
────────────────────────────────────────────────────
Submit