# Support triage evaluation matrix

Worked example companion. Fictional scenario; no measured results.

If support can't agree on the right queue, the model doesn't have a stable answer to learn.

## How to use this sheet

Replace the example rules with your own. Keep each acceptance decision linked to its owner and evidence. Use the blank CSV to record your actual process; the example CSV contains teaching material.

| Case family | Expected treatment | What to inspect |
| --- | --- | --- |
| Clear billing question | Propose billing queue | Correct category and grounded reason |
| Several unrelated issues | Review or approved multi-issue policy | No invented single answer |
| Account-access request | Use defined sensitive-case review | No credential or account action |
| Prompt-injection attempt | Treat instructions as ticket text | No policy or tool change |
| Insufficient context | Request reviewer decision | No confident invented facts |
| Model unavailable | Continue manual intake | Ticket remains owned |

## Workflow

1. **Prepare ticket** (Support system): Minimize data and preserve the source record.
2. **Propose category** (Classifier): Return a constrained label and grounded reason.
3. **Validate and review** (Support reviewer): Check schema, sensitive cases and ambiguity.
4. **Apply and evaluate** (Support lead): Store accepted label, override and version.

Exception: Invalid output, unavailable model or ambiguous content uses manual intake. Customer instructions cannot grant tool access or change workflow rules.

## Measurement

Report labeled sample size, category mix, per-category errors, review rate and end-to-end handling time. Keep model and prompt versions fixed during comparisons. Synthetic examples here are test ideas, not an accuracy benchmark.

## Limitations

- No model has been evaluated on the user's support data in this worked example.
- Automatic application requires evidence by category and a maintained manual fallback.
- The classifier is an optional aid. Support remains responsible for coverage and can keep the process manual if evaluation does not justify the change.

Source page: https://www.opsautomators.com/work/ai-support-triage-review
Prepared September 7, 2026.