Observe the ABM Agent Workflow Beyond Model Calls

The decision to make

Token counts and request latency describe computation but do not explain the account decision. A workflow can finish technically while producing a brief the seller cannot use. Another can pause appropriately for account clarification and appear slow. Observability should connect operations with business states so teams distinguish failed tools, uncertain evidence, waiting review, and rejected work. The record must be sufficient to reconstruct what happened without relying on a model's private reasoning or a large unstructured transcript.

Build the practical approach

Assign a run ID at accepted intake and carry it across research, drafting, review, and execution. Record stable account IDs, stage transitions, artifact versions, source references, approval outcomes, and external action receipts. Use spans for operations with duration and timestamped events for decisions or state changes. Separate technical status from business acceptance: a retrieval call may succeed while the evidence stage remains unresolved. Maintain a bounded taxonomy including identity unresolved, source unavailable, evidence insufficient, claim rejected, review pending, execution failed, and completed without action. Record the selected route and concise operational reason rather than attempting to expose private model reasoning. Restrict routine telemetry to metadata needed for support and quality analysis; control access to any retained content. Link asynchronous jobs and retries to the originating run. Retain critical execution receipts independently of routine trace sampling. Provide operators with a view that shows the last completed stage, current blocker, owner, and safe next action. Version instrumented schemas so changes in emerging conventions do not silently break comparisons.

OpenTelemetry spans represent operations, include start and end timestamps, and can form parent-child relationships. OpenTelemetry: Traces.

The practical workflow

1. Assign a workflow run ID
2. Trace business stages
3. Record decision outcomes
4. Link artifacts and receipts
5. Review failure patterns

Compare the approaches

Work through an illustrative scenario

Illustrative scenario: sales reports that a research brief arrived after a meeting. The run shows prompt retrieval and drafting, followed by a long review wait because account ownership was missing. A retry started another research run while the first waited, creating two drafts. The operations lead must decide whether to accelerate model processing or fix queue handling. The evidence points to an ownership fallback and a rule that resumes or escalates existing work before starting a duplicate. The team retains both runs, labels the second as redundant, and connects the accepted brief to the first. No campaign action occurred, so the incident is a delivery and coordination failure rather than an execution failure.

Measure whether the work is useful

End-to-end turnaround runs from accepted intake to the terminal business state. Active processing time sums stage execution durations without human waiting; review wait measures time in a pending-review state. Accepted-output cost includes model, tool, and operator correction costs divided by accepted deliverables. Track stage failure rate as failed stage attempts divided by attempted stage entries, and report retry recovery separately. Decision traceability is executed actions with run ID, approved artifact version, and receipt divided by executed actions. Monitor unresolved blocker age and missing links. Compare latency distributions within task types rather than using a single average across briefs, campaigns, and long review pauses. Maintain business acceptance rates alongside technical completion rates.

The GenAI agent conventions define agent, workflow, and tool spans and are explicitly marked Development. OpenTelemetry: GenAI agent span conventions.

Avoid the common failure points

An HTTP success response cannot establish usable research or correct execution. Logging every prompt can expose content while still omitting the decision that matters. High-cardinality account IDs can aid traces but make metric labels difficult to manage. A sampled trace set may miss rare failures, so preserve material action receipts deliberately. Do not count a retry as another completed business task. A terminal unknown can be correct behavior; distinguish it from a crash. Changing error categories or stage boundaries without versioning creates misleading trend shifts. Alert on actionable blocked work, not every normal human pause.

Your next-action checklist

• Technical trace: Add business-stage events. Check the limitation: does not establish business quality.
• Decision log: Record reason and version. Check the limitation: requires disciplined schemas.
• Outcome scorecard: Link outcomes to runs. Check the limitation: cannot isolate every failure.

Use the comparison to choose a bounded next step. Record the evidence, the responsible owner, and the review decision before extending the play to additional accounts.

Questions this guide answers