AI agents now rush in when things break, dig through logs, and find the cause. The follow-up question writes itself: "Can we act on that investigation report?"

AWS's October 9 answer is Review: a quality assessment for completed incident investigations, returning one overall helpfulness verdict plus a per-criterion breakdown. The same day brought four new service regions including Seoul. An evaluating agent now sits atop the investigating agent — and the structure is worth studying.

What a Review is: grading without rerunning

Start with the docs' definition. A Review is an on-demand quality assessment of a completed investigation. It reads the investigation's record — the problem it was given, the evidence it collected, the conclusion it reached — and measures how well the investigation did its job.

Review input:  one completed investigation (problem, evidence, conclusion)
Review output: ① overall verdict: helpful / not helpful
               ② per-criterion verdicts: pass / fail / not applicable, with reasoning
What a Review never does: rerun the investigation or modify the original

"Never reruns" is the key line. A Review doesn't fix the investigation; it pins a report card next to it. The original stays untouched, so the record stays clean when you later audit why the investigation judged as it did.

When to use it? The docs name three moments: deciding whether to act on a root cause, comparing investigations, and quickly checking trust before relying on findings. A Review isn't for "read or skip" — it's for the moment before "act or don't."

The five criteria: what gets measured

Reviews grade five dimensions. In table form:

Criterion Question asked Verdict
Scope alignment Did it stay focused on the reported problem? pass / fail / n/a
Evidence grounding Are conclusions backed by collected evidence? pass / fail / n/a
Completeness Did it cover the relevant areas and reach a conclusion? pass / fail / n/a
Environment impact Did it account for its effect on your environment? pass / fail / n/a
Accuracy Does the root cause match the linked ticket or incident? pass / fail / n/a

Each criterion returns pass, fail, or not applicable with its reasoning. The docs even specify the n/a conditions: with no linked ticket, or no root cause recorded on the ticket, Accuracy can't be assessed. Refusing to grade what can't be graded is a design choice that builds trust.

These five overlap neatly with the AI evaluation checklist: evidence accuracy, causal validity, actionability. The briefing's practical advice — add evidence, cause, and action criteria — sits in the same current. Use AWS's five defaults as-is, then layer your team's specifics on top via customization.

Custom reviews: retune the rubric in your team's words

When the five defaults don't fit, change them. Set agent instructions for the Grader agent on the Knowledge page, and Reviews read those instructions before assessing. No fixed schema or special syntax — you describe in plain words what the Review should check.

Default behavior: instructions layer onto the defaults (untouched criteria stay)
  → Adjust: retune what an existing criterion looks for, to your team's standard
  → Add:    introduce a new criterion on top of the default five
  → Drop:   remove a default criterion that doesn't apply to your investigations
Clean slate: say so explicitly and the Review uses only your criteria

Whatever you define applies to Reviews launched from that point on, and custom criteria report in exactly the same format (per-criterion verdicts plus the overall verdict). A stable format means easy dashboards.

One pattern stands out: the rubric itself is managed as natural-language instructions, not code. When the scorecard changes, you edit sentences, not deployments. The "separate the judgment explicitly" idea from the Jev article has hardened into an operating feature.

The quality loop: feedback, memory, and Review interlock

A Review alone is half the story. DevOps Agent already stacks several mechanisms for investigation quality.

Earlier layers: ① investigation feedback (June) — accuracy feedback on completed work
                ② Reflections memory (Sept) — records lessons so the same mistake isn't repeated
This layer:     ③ Review (Oct) — a point-in-time quality grade for a completed investigation

If feedback plus Reflections is the learning loop that makes the next investigation better, Review is the verdict loop that decides whether to trust this one. One records for the future, the other decides action in the present. Together they complete the ops picture: confirm trust with a Review before acting, leave lessons with feedback after acting.

Region expansion: Seoul is in, with cross-region inference

The day's other announcement deserves a line too. DevOps Agent arrived in four new regions: US East (Ohio), Asia Pacific (Seoul), Europe (Paris), and Europe (Stockholm).

New regions: Ohio · Seoul · Paris · Stockholm
Caveat: Agent Spaces in Seoul route inference across AWS Regions,
        not within a single geography
(same pattern as Mumbai, Singapore, and São Paulo — June notice)

Read the Seoul news together with the routing caveat. Housing your Agent Space in Seoul doesn't mean model inference stays there — teams where data processing location matters should verify before adopting.

CodeBridge Mini Lab: build your team's investigation report card

Turn the briefing's action into an exercise: attach evaluation criteria to AI incident analysis.

① Run a baseline Review:
   [ ] Launch a Review on one completed investigation
   [ ] Read the overall verdict (helpful/not helpful) and the five criterion verdicts
   [ ] Human-check whether the reasoning behind any fail is sound
② Verify the Accuracy linkage:
   [ ] Is the investigation linked to a ticket or incident? (no link → Accuracy is n/a)
   [ ] Does the ticket's root cause agree with the investigation's conclusion?
③ Layer on team criteria:
   [ ] Add 1–2 team criteria via Grader instructions (e.g. specificity of action proposals)
   [ ] Drop defaults that don't fit
   [ ] Confirm the next Review applies the new criteria
④ Close the loop:
   [ ] Review for trust before acting → submit feedback after acting
   [ ] Check whether Reflections shaped the next investigation of the same alarm

Run this sheet for a few weeks and "the criteria our investigations keep failing" will surface. That list is your next runbook and skill doc to fix.

Conclusion: the grading rubric matters as much as the investigating agent

One line to close:

Teams that take an agent's report at face value and teams that read it with a report card attached accumulate very different incident responses.

Review's real message isn't "AI investigates well" — it's "put measuring investigation quality inside your operations." Start with the default five, layer team criteria through Grader instructions, confirm with a Review before acting, and record with feedback after. Read together with the Seoul expansion, the ending writes itself: on newly nearby infrastructure, it is time to build operations that act only on investigations worth trusting.

Keep reading

References

Want to learn this with guided, hands-on practice?

If you want practice connecting recurring decisions to code and separating automated handling from human review, the Laya decision-model and human-in-the-loop design course connects directly to this article's evaluation-criteria design.