The evaluation indicates that the compaction process is losing important information despite producing summaries comfortably within the available context budget.
This is primarily a context-retention failure rather than a token-efficiency problem.
Option C correctly prioritizes improving information recall before attempting further compression.
The engineer should revise the compaction instructions to explicitly preserve previous resolutions, escalation decisions, unresolved issues, ownership changes, customer commitments, and significant troubleshooting events.
These details may remain important even when they occurred many turns earlier.
For example, an escalation decision made at turn 8 may determine how a support case should be handled at turn 40. A summarizer that prioritizes recent messages may incorrectly omit that earlier decision.
Option A would make summaries shorter and potentially worsen information loss. Option B assumes recent events are always more important. Option D changes both retention and pruning behavior simultaneously, making it harder to identify which modification improves performance.
A disciplined optimization process should first establish adequate coverage of important facts, then remove redundant language while preserving that coverage.
Databricks MLflow evaluation supports testing application behavior against representative datasets and comparing quality across configurations.
The engineer should use a benchmark containing expected escalation events and resolutions to assess completeness before optimizing summary length.
Official Databricks reference: Tracing overview, MLflow API Reference, and Debug and analyze your app with tracing.
────────────────────────────────────────────────────
Submit