AgentWorks documentation

Workflow Monitoring

This doc covers the workflow observability surfaces that still exist in the product today.

It is worth keeping. The workflow UI still exposes execution logs, cost analysis, evaluation reports, learnings, and run history. What changed is the architecture behind them: older validation-heavy and per-step-learning explanations are no longer the right model.

Monitoring Surfaces

There are three practical scopes:

1. Workflow-level views

• Pulse: the single agent-curated HTML log (builder/improve.html) — the primary at-a-glance surface (see below)
• Costs: aggregated token and USD usage across run folders
• Evaluation reports: benchmark-style scoring across runs, with single-run drill-down
• Learnings: current persisted learning state, including the global workflow skill

2. Run-folder views

• Execution logs: detailed logs for one selected run folder such as iteration-12 or iteration-12/group-a
• Final outputs: generated final reports for a run folder

3. Cross-workflow operational views

• Workflow overview: recent run folders, status, timestamps, costs, and evaluation presence across workflows
• Scheduled runs panel: cron job history, latest runs, live sessions, and drill-down into logs/costs/evaluation for scheduled executions

Pulse — the agent-curated log

The Pulse (builder/improve.html) is the primary workflow-level monitoring surface and the user's main window into a workflow. It is a single, self-contained HTML document the workflow's agents curate, rendered as a first-class right-panel view alongside Plan, Report, and Soul, and it follows the app's light/dark theme.

When pending input exists, Runloop renders Needs your decision first. The HTML then reads: two verdicts → a one-line status headline → active Assumptions challenged (only when consequential assumptions exist) → Today's outcome → the goal card → collapsed technical detail → recent runs → a newest-first timeline → collapsed Agent log → archive. The Agent log contains only current handoff state, ids, cursors, cadence, and evidence pointers; it never duplicates the user narrative. Every workflow is judged on two independent axes, each stamped with the run it's as-of:

• Bug — did it run correctly (errors, skipped steps, missing/empty artifacts, regressions)? Fixed by hardening.
• Goal — is it achieving its success criteria (eval scores and outcome metrics vs soul.md)? Fixed by refining or replanning.

A per-run monitor writes to the Pulse after every run: a focused pass that detects silent breakage and goal drift, refreshes the Bug/Goal pills in builder/improve.html, records a finding only when something is wrong, and sends a compact summary. Its final notify/summary step writes the builder/card.health.html dashboard card because by then harden, artifact review, cost/time, backup, and publish outcomes are known; that card is compact dashboard state, while email can carry the fuller narrative. If Pulse asks the user a question, it first logs a structured Human input requested card in builder/improve.html and only then sends the notification. It never does broad improvement work — scheduled auto-improve passes (harden for Bug, replan-proposal for Goal) do that, and each applied change is later confirmed against the run that proves it worked. Turn the monitor on with /monitor or the toolbar Monitor toggle; schedule the acting passes with /auto-improve. See the Auto-Improvement Framework for the full loop.

Execution Logs

Execution logs are still the main debugging surface for a single run folder.

Current behavior:

• The popup is opened from the workflow toolbar and is scoped to the currently selected run folder when possible.
• Backend data comes from /api/workflow/logs.
• The server scans runs/{runFolder}/logs/ and runs/{runFolder}/execution/, then maps folder names back to plan step metadata from planning/plan.json.

The log viewer still supports these file families:

• execution/execution-attempt-{A}-iteration-{I}.json
• execution/execution-attempt-{A}-iteration-{I}-conversation.json
• validation.json and validation-{N}.json
• learning-execution.json
• conditional-evaluation.json
• orchestration-execution.json
• todo-task-execution.json

Important current nuance:

• validation logs still exist in the execution log viewer, but validation is no longer the main architecture story for workflow docs
• pre-validation remains relevant runtime signal, but the canonical validation doc is pre_validation_guide.md
• execution logs are best thought of as per-run forensic data, not as the source of workflow architecture truth

Costs

Cost analysis is still current and useful.

Current behavior:

• The popup is opened from the workflow toolbar, overview page, and scheduled runs panel.
• Backend data comes from /api/workflow/costs.
• It aggregates token usage across run folders and supports drill-down into individual runs.
• The UI highlights the currently selected run folder when applicable.

The cost UI is still workflow-level, not step-config architecture.

Evaluation Reports

Evaluation reports are still current and should stay documented.

Current behavior:

• Backend data comes from /api/workflow/evaluation-reports.
• The UI supports both All Iterations and Single Iteration views.
• The active run folder is highlighted as Current when present.

This remains a separate testing and benchmarking surface, not part of the learning or pre-validation model.

Learnings

The learnings popup is still relevant, but the old explanation was stale.

Current behavior:

• Backend data comes from /api/workflow/learnings/all.
• The popup shows both step metadata and the global workflow learning entry _global.
• _global is rendered as Workflow Knowledge (Global) in the UI.
• Locking and unlocking still exist via step config updates.
• Deleting learnings for a step still exists.

What changed:

• learning is no longer best described as per-step prose memory with complexity caps and exploitation phases
• the main persistent learning surface is now the shared skill at learnings/_global/SKILL.md
• scripted steps can still have step-specific saved code such as learnings/{step-id}/main.py

The canonical architecture doc for this is learning_architecture.md.

Run Overview And Scheduled Runs

There are now two additional monitoring surfaces worth acknowledging:

Workflow overview

• lists run folders from consolidated workspace state
• reconciles run metadata with active in-memory executions
• shows status, timestamps, cost summary, and evaluation presence across workflows

Scheduled runs panel

• shows cron jobs and run history
• opens the same logs, costs, evaluation, and final-output popups for scheduled executions
• helps operational users inspect automation outcomes without opening each workflow canvas first

Keep Or Remove?

Keep it.

This doc is still relevant because the product clearly has workflow monitoring and observability features. It just needed to stop documenting the older validation-era and complexity-cap model.

Related Docs

• cost_and_log_measurement.md
• pre_validation_guide.md
• learning_architecture.md
• evaluation_system.md
• workflow_manifest_architecture.md