Which of the following statements accurately identifies differences between the evaluation phase and the monitoring phase in the Generative AI application lifecycle within Databricks?
A.
The evaluation phase uses Mosaic AI Agent Evaluation and an evaluation dataset to assess an agent’s performance metrics and traces, while the monitoring phase relies on inference tables as source data for creating a metrics profile.
B.
The evaluation phase logs and traces live API calls in production, while the monitoring phase runs metrics on inference tables containing those traces.
C.
The evaluation phase ensures the agent’s responses comply with business rules in production, whereas the monitoring phase is focused on SLA and performance metrics.
D.
The evaluation phase uses all inference history to assess agent performance and readiness for production, while the monitoring phase uses only new inference-table records to monitor performance.
A best describes the dataset-based evaluation and inference-table-based monitoring workflow referenced in the question. Evaluation runs an application against selected test cases, collects traces, and calculates metrics to compare versions or detect regressions. Production monitoring examines real usage over time to identify operational or quality problems. Evaluation does not require all historical inference records, and monitoring is not limited to latency or service-level metrics; it can also assess response quality and policy adherence. Option B incorrectly treats production tracing as the defining evaluation activity. Current MLflow 3 monitoring can apply scorers directly to production traces, so inference-table metrics profiles are one documented monitoring architecture rather than a universal requirement for every modern Databricks deployment. Evaluation documentation , monitoring documentation
Contribute your Thoughts:
Chosen Answer:
This is a voting comment (?). You can switch to a simple comment. It is better to Upvote an existing comment if you don't have anything to add.
Submit