Most AI demos look great. The agent answers the five questions the team tried, and everyone is happy. Then real users arrive with thousands of questions nobody tried, and the agent makes things up, calls the wrong tool, or costs ten times the budget.
A better prompt rarely fixes this. What helps is testing the agent the way you would test any other software that makes decisions. This is how we audit an agent in an AI System Audit. We test and recommend; your team makes the changes.
1. Agree what "correct" and "safe" mean
On the scoping call, we ask the people who own the process to write down measurable success criteria. For example:
- The agent answers billing questions using only the customer's own records.
- It never issues a refund above a set amount without a human approving it.
- Ninety-five percent of answers must match what a trained staff member would say.
- Each conversation must cost less than a fixed amount.
If nobody can write these down, that is a useful finding on its own: there is no way yet to tell whether the agent is working.
2. Map what the agent can reach
We list every tool the agent can call, the credentials each one uses, and the data it can read or change. Most serious risks come from this list, not from the model: an agent that can send emails, issue refunds or edit records with no limits is a risk however good its prompt is (OWASP LLM06, excessive agency).
3. Test it against real cases
With your team, we turn real examples into test cases with an expected result: past tickets, emails, documents and the answers your best people gave. We add the hard ones on purpose: vague questions, questions with no answer in your data, and questions the agent must refuse. Then we measure accuracy, refusals and cost per conversation.
4. Attack it, safely
We write test cases that act like an attacker: instructions hidden in a document or email, requests for another customer's data, and attempts to make the agent call a tool it should not use. Testing runs on what you authorized, preferably in staging, and any test that could have side effects is agreed in advance. If we find something critical, we stop testing that item and call you the same business day.
5. Check where the rules live
A language model is good at understanding what someone wants. It is not good at enforcing a refund limit every single time. So we check whether dates, amounts, permissions and approvals are enforced by plain, tested code, and whether the model's output is checked before it reaches code, SQL, HTML or a tool (OWASP LLM05). A rule that exists only in the prompt is a finding.
6. Check the limits and the logs
We look for rate limits, budgets and spend alerts, limits on loops and retries, and logs that would let your team investigate an incident without storing personal data they don't need (OWASP LLM10).
What you get
- A written report with every finding mapped to the OWASP Top 10 for LLM Applications 2025, with risk ratings and evidence
- Prioritized recommendations your team can start on the same day, step by step for your stack
- The red-team and evaluation test cases from the audit, to rerun on every change to the prompt, model, tools or data
- A review call, and one re-test of critical and high findings within 60 days to confirm your fixes
The test cases keep working after we leave. A change that improves one thing and breaks three others shows up in your next run, before your customers notice.
If you run an AI agent, or have one that behaves differently in production than in the demo, an AI System Audit checks exactly this. It starts with a free scoping call and a fixed quote.