Automation does what you tell it, quickly and over and over. The same goes for an error. So before go-live it isn’t enough that the normal path works. It must be clear what happens when something doesn’t fit.

Illustrative example: a supplier changes its invoice layout. Processing correctly reads the amount but misreads the due date. That’s not only a recognition error; the check on the result is missing too. The log records it, but without anyone reviewing deviations nobody is alerted, and payments go out wrongly for weeks.

Seven design questions

  1. What identity does the solution run under? Its own service identity where the system allows it, not the personal account of an employee who will one day leave. Permissions should fit the task and have an administrator.
  2. What may it do? Set out the types of operation, limits and prohibited steps. “Prepares an order” is a different permission from “sends an order”.
  3. When does a person decide? Weigh impact, uncertainty and the options for correction. Not every write to a system needs the same approval.
  4. What is recorded? You need to trace the case, the action taken, the rule applied and any approval. Also agree who may read the records and how long they are kept.
  5. How will you spot a problem? Define the deviations you track, the alerts and who receives them. An alert with no recipient is just a log entry.
  6. Who owns the process? It must be clear who changes the rules and decides on exceptions. That is usually someone from operations, not IT.
  7. How do you stop and correct it? Prepare a pause, a fallback procedure and a way to find the affected cases.

Four levels of autonomy

It helps to be precise about what the system should do:

  • Reads information: the lowest risk, though which data it can access still matters.
  • Prepares a recommendation: a person sees the proposal and decides.
  • Carries out an approved action: within predefined limits and with a record.
  • A step a person must decide: such as payments, external commitments or decisions about people.

Start lower and widen permissions based on results. Not everything has to end up at the highest level.

Controls should match the risk

Too much confirmation leads to rubber-stamping. Someone who clicks through two hundred proposals a day stops reading. Too little control lets an error spread. The design should separate routine cases from situations with greater impact, for example by amount, new supplier or low recognition confidence.

What to test before go-live

  • a missing or unreadable document,
  • a duplicate case,
  • an unavailable downstream system,
  • a change in input format,
  • stopping and returning to the manual process.

After deployment, keep reviewing the rules against real results. The world changes, suppliers change their invoices and models get updated.

How we approach this at ENTER

We answer these seven questions with the client before anything gets built. In the Integration phase we then set up service identities, least-privilege permissions, approval steps and a record of important actions. Sensitive steps are approved by a person. We roll out in stages on top of existing systems, with agreed testing and transition into operation. Our Responsible AI principles set this out in detail.

Related: Security · Responsible AI · How we measure