> ## Documentation Index
> Fetch the complete documentation index at: https://docs.duckie.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Feedback and Analytics

> Use outcomes, run review, and stakeholder feedback to improve the system

The feedback loop uses a Duckie Assistant feedback agent to turn human review, case outcomes, and operational metrics into better workflows, agent instructions, tools, guidelines, and reporting.

## What It Does

The feedback function:

* Reviews runs with low confidence, tool failures, reviewer changes, or poor outcomes
* Groups recurring failures into categories
* Identifies missing knowledge, unclear guardrails, or weak workflow branches
* Proposes updates to agent instructions, workflows, tools, categories, or guidelines
* Reports improvement items to owners
* Measures whether changes improve future runs

## Recommended Components

| Need                                      | Duckie component                |
| ----------------------------------------- | ------------------------------- |
| Review run history                        | Analytics runs                  |
| Group recurring issues                    | Categories and attributes       |
| Inspect and propose configuration changes | Duckie Assistant feedback agent |
| Track improvements                        | Issue tracker or ticketing tool |
| Monitor recurring failures                | Alerts and daily reports        |

## Feedback Loop

```mermaid theme={null}
flowchart TD
  Runs["Runs and tool traces"] --> Review["Duckie Assistant feedback agent"]
  Outcomes["Case outcomes"] --> Review
  HumanEdits["Reviewer changes"] --> Review
  StakeholderReports["Daily stakeholder reports"] --> Review
  Review --> Patterns["Failure patterns and gaps"]
  Patterns --> Improvements["Guidelines, workflows, agents, tools, and categories"]
  Improvements --> Testing["Replay and batch testing"]
  Testing --> Deploy["Updated deployments"]
  Deploy --> Runs
```

## What to Track

Track categories and attributes that explain both business outcomes and system behavior:

| Tracking item         | Example values                                                                          |
| --------------------- | --------------------------------------------------------------------------------------- |
| **Reason category**   | Fraud, duplicate, service not received, product not as described, credit not processed. |
| **Automation path**   | Accepted, countered, needs account info, human review, manual handling.                 |
| **Escalation reason** | Low confidence, missing evidence, deadline risk, policy question, tool failure.         |
| **Reviewer decision** | Approved, changes requested, rejected, escalated.                                       |
| **Outcome**           | Won, lost, accepted, withdrawn, closed, pending.                                        |
| **Failure type**      | Missing data, wrong classification, weak evidence, message quality, tool error.         |

## Review Cadence

Use three loops:

| Loop                        | Cadence            | Owner                                                |
| --------------------------- | ------------------ | ---------------------------------------------------- |
| **Daily operations report** | Business days      | Duckie Assistant reporting agent and operations lead |
| **Exception review**        | As cases happen    | Slack escalation agent and reviewer                  |
| **Improvement review**      | Weekly or biweekly | Duckie Assistant feedback agent and system owner     |

## Improvement Actions

Common improvements include:

* Add a branch to the dispute handling workflow
* Split a broad reason category into more specific categories
* Add a missing evidence checklist
* Tighten an agent instruction
* Add a guardrail for a risky response pattern
* Add a custom tool for repeated manual lookup work
* Update the daily report to surface a recurring issue

<Info>
  The Duckie Assistant feedback agent should propose changes, but sensitive workflow, guardrail, or tool updates should follow your normal review process before going live.
</Info>

## Related Docs

<CardGroup cols={2}>
  <Card title="Analytics" icon="chart-line" href="/analytics/overview">
    Monitor performance and trends.
  </Card>

  <Card title="Runs" icon="clock-rotate-left" href="/analytics/runs">
    Inspect the details behind each case.
  </Card>

  <Card title="Alerts" icon="bell" href="/analytics/alerts">
    Notify the team when important patterns appear.
  </Card>

  <Card title="Replay Testing" icon="clock-rotate-left" href="/testing/replay-testing">
    Test changes against historical cases.
  </Card>
</CardGroup>