AI reliability
We plan for the AI being wrong.
AI is not the right tool for every task. For each one we ask what a mistake would cost, then recommend who should do it. You make the final decision.
Who does what
| If a mistake is | The work goes to |
|---|---|
| Impossible, because the answer is certain | Fixed rules |
| Cheap, visible and easy to undo | AI alone |
| More costly | AI prepares it, a person approves |
| Costly, irreversible or regulated | A person |
If one mistake in a hundred would be costly, the task stays with a person, however accurate the AI is.
Seven checks before AI acts alone
We either make a task safe to get wrong, or make a wrong answer impossible to act on. For work AI does alone, every item passes seven checks. Any "no" goes to a person.
Do we understand it?
- 1Is it one request, of a type we have tested?
- 2Is all the information there, and does it match your records?
Is it safe to automate?
- 3Is the person or source verified?
- 4Is it free of anything sensitive, such as a complaint, a bereavement or a vulnerable person?
- 5Is a mistake harmless or easy to undo?
- 6Is AI permitted to decide this, under the law, your regulator and your own policy?
Is the AI right?
- 7Does the AI give the same answer on repeat runs, and do rule checks confirm it?
Outcomes
All seven yes: AI acts alone, using a response or action approved for that scenario.
Any no: A person handles it, and the service is the same as today.
Before a scenario goes live
It is tested on past cases, run alongside your staff until its answers match theirs, and signed off by your compliance and data protection teams.
After AI acts
The result is checked in the system and logged. A person reviews a sample. Limits stop the same action repeating. It can be switched off at once. Customers are told when a reply comes from AI, and can ask for a person.
At the start, most tasks go to a person. We widen what AI handles alone one scenario at a time.
How it keeps improving
- 1Control set. A fixed set of past examples with agreed right answers.
- 2Test. Every version must reach the agreed accuracy on the control set before it goes live.
- 3Run. In live use, uncertain items go to people.
- 4Flag. Staff tag any result that was wrong.
- 5Review. We group flagged items by cause: a missing rule, an unclear category or a new type of request.
- 6Fix and re-test. Each fix runs against the whole control set, so it cannot break what already worked. The flagged items join the set, and the loop returns to step 3.
Bring us one process.
Pick the process your team complains about most. Start with a 30-minute call.