Skip to main content
The feedback loop uses a Duckie Assistant feedback agent to turn human review, case outcomes, and operational metrics into better workflows, agent instructions, tools, guidelines, and reporting.

​
What It Does

The feedback function:
  • Reviews runs with low confidence, tool failures, reviewer changes, or poor outcomes
  • Groups recurring failures into categories
  • Identifies missing knowledge, unclear guardrails, or weak workflow branches
  • Proposes updates to agent instructions, workflows, tools, categories, or guidelines
  • Reports improvement items to owners
  • Measures whether changes improve future runs

​
Feedback Loop

​
What to Track

Track categories and attributes that explain both business outcomes and system behavior:

​
Review Cadence

Use three loops:

​
Improvement Actions

Common improvements include:
  • Add a branch to the dispute handling workflow
  • Split a broad reason category into more specific categories
  • Add a missing evidence checklist
  • Tighten an agent instruction
  • Add a guardrail for a risky response pattern
  • Add a custom tool for repeated manual lookup work
  • Update the daily report to surface a recurring issue
The Duckie Assistant feedback agent should propose changes, but sensitive workflow, guardrail, or tool updates should follow your normal review process before going live.

Analytics

Monitor performance and trends.

Runs

Inspect the details behind each case.

Alerts

Notify the team when important patterns appear.

Replay Testing

Test changes against historical cases.