How to Prove a Human Checks Your AI — A Practical Oversight Log for Charities

← Back to Blog

Almost every AI policy contains the same reassuring sentence: "All AI output is reviewed by a human." It's the right instinct. But "a human reviews it" is a claim, not evidence — and if someone ever asks you to show it, a claim won't be enough. The gap between saying you supervise your AI and being able to prove it is where most organisations are exposed.

This article shows you how to turn "a human checks it" into something you can actually point to — without drowning a small team in paperwork. It's written for people who run organisations, not for lawyers.

Why "a human reviews it" isn't enough

Human oversight runs through the whole conversation about responsible AI — in the EU AI Act, in data-protection thinking about automated decisions, and in the simple trust your funders and service users place in you. In every version, the expectation is the same: a person, not the machine, is responsible for what the AI produces.

The problem is that oversight, as usually written, is invisible. The policy says review happens. But:

• Who, specifically, does the reviewing?
• What do they actually check?
• When does the review happen — before publication, or after a complaint?
• What happens when the AI is wrong, and can you show a time it was caught?

If the honest answers are "someone, somehow, at some point," then your oversight exists in intention but not in evidence. And this is precisely the fourth of Vedomia's four pillars of transparency — Auditability: a document is not proof that a process actually runs. You have to be able to show the step was taken and leave a trace behind it.

A document that says "we supervise our AI" proves you wrote a document. A short, dated record of reviews proves you supervise your AI. Only one of those survives a hard question.

What good oversight looks like — the three parts

Real, demonstrable human oversight has three parts. Most organisations have the first, sometimes the second, and almost never the third.

1. A named responsibility (the "who")

Not "someone." A role: "the communications lead reviews all AI-assisted text before publication"; "the office coordinator reviews chatbot conversations weekly." Naming a role — not necessarily a person — is what turns a vague duty into an accountable one.

2. A defined check (the "what")

What is the reviewer actually looking for? "That it reads well" is not a check. "That facts are correct, that it contains no service-user personal data, and that the tone matches ours" — that's a check someone can perform and you can describe.

3. A trace (the "proof")

This is the missing piece. A lightweight record that the check happened: what was reviewed, by whom, when, and whether anything was corrected. Without this, parts 1 and 2 are promises. With it, they're facts.

The trace doesn't have to be heavy

The fear that stops small organisations is understandable: "we don't have time to log every single thing the AI touches." You don't have to. The trick is to match the weight of the record to the weight of the risk.

• Low-stakes, high-volume uses (AI helping draft an internal first-draft that's fully rewritten anyway) need almost nothing — a single line in your AI use guideline saying these are always rewritten by a named role is enough.
• Public-facing content (AI-assisted text you publish, campaign images) needs a light trace: who signed it off, and when. Your existing publishing workflow — an approval in your CMS, an email sign-off — often already contains this. You just have to recognise it as your oversight record and keep it.
• Anything touching a person's situation (a chatbot answering service users; anything near a decision about someone) deserves a genuine, periodic log: a weekly review with a dated note of what was checked and what was corrected.

Notice that in two of the three cases, the record either barely exists or is already sitting in a tool you use. The work is mostly recognising and keeping what you already do — not inventing a bureaucracy.

A simple oversight log you can copy

Here is a minimal format that works for the higher-stakes uses. It's not a form you submit anywhere — it's your own internal evidence, kept wherever your governance records live. One row whenever a periodic review happens.

A four-step plan to make oversight provable

Step 1 — List your AI uses and rank them by stakes

Reuse the AI inventory you (hopefully) already have. Sort each use into low / public-facing / person-affecting. This ranking tells you how much of a trace each one needs.

Step 2 — For each use, name the "who" and the "what"

Write one plain sentence per use: who reviews it and what they check. If you can't write that sentence, you've found a gap — that's a use where oversight is assumed but not assigned.

Step 3 — Decide the lightest trace that fits the risk

Low-risk: a line in your guideline. Public content: keep your existing sign-off. Person-affecting: a periodic log like the one above. Don't over-engineer the low-risk ones — the effort should follow the risk.

Step 4 — Put it where you'd find it in five minutes

Evidence you can't locate isn't evidence. Store the log and the guideline with your other governance records, so that if a funder, auditor or board member asks "show me how you oversee your AI," you can answer in minutes, not days.

How Vedomia can help

Vedomia is an Irish company focused on one thing: helping organisations make their processes and their use of AI visible, explainable and auditable. We work mainly with charities, nonprofits and publicly funded organisations. Auditability — being able to show that a step happened — is one of our four pillars, and human oversight of AI is exactly where it bites.

• Transparency Self-Audit (free). Around 20 questions across four pillars — Visibility, Sequence, Justification, Auditability. It surfaces where you can claim oversight but not yet show it.
• Transparency Gap Report. A structured read-out of where your evidence is thin, which gaps to close first, and what records you should be keeping — in language your board and funders understand.

To be clear about what we do and don't do: Vedomia supports readiness, helps you document your AI use, and identifies transparency gaps. We do not certify compliance, and nothing we provide guarantees legal conformity. This is not legal advice.

You almost certainly already say a human reviews your AI. The small, worthwhile step is making that true on paper — so the next time someone asks, you can show them, not just tell them.