The AI Is Not the System: Engineering Reliable Agent Harness

Ask an AI a hard business question and you get an answer that sounds fluent and certain, and that nobody can check. It might even be right. You have no way of knowing, and no company can sign off on a decision on that basis.

This talk follows one business question from the moment it is asked to the moment someone is willing to put their name on the answer, building up the engineering that has to sit around the model: getting to the right data, checking generated queries before they run, holding the output to business rules the model knows nothing about, making every number traceable back to where it came from, and treating approval as part of the workflow. Each layer shows up for the same reason it does in real projects, because the previous one failed.

That changes the question. Not whether AI can be trusted, but how much confidence we have in this one result, and on what grounds. Measured confidence then sets how much review is needed and where autonomy stops.

You will leave with a working picture of how these systems are built, a way to match autonomy to the cost of being wrong, and language for discussing AI risk in numbers instead of adjectives. With a live demo, run end to end.

The model is not the system. Reliability comes from what you build around it, and from what happens when it turns out to be wrong.