Back to articles// AI Agents, AI governance, Safety

    How do you make sure an AI agent is safe?

    The short answer: by being able to say, at any moment, what each agent does, which model it runs on, which data it works with and since when. If you don't know that, you can't call an agent safe, however good the technology around it is. You capture that overview in a passport per agent, which you update with every change and review every quarter. Below is how that works and why it is more than paperwork.

    Job van den Berg Published 14 oktober 2026 6 min read
    Illustration of a human supervisor holding a magnifying glass above autonomous AI agents on conveyor belts

    The short answer: by being able to say, at any moment, what each agent does, which model it runs on, which data it works with and since when. If you don't know that, you can't call an agent safe, however good the technology around it is. You capture that overview in a passport per agent, which you update with every change and review every quarter. Below is how that works and why it is more than paperwork.

    Safety starts with overview, not technology

    When someone asks how to make an AI agent safe, the conversation quickly turns to technology: guardrails, access management, monitoring, an environment hosted in the EU. All of that is necessary, but it is not where it starts. The biggest risk in most organisations is that nobody knows exactly which agents are running, what they do and what they are connected to.

    That is because right now everything revolves around one thing: as many agents as possible, live as fast as possible. Show traction, keep momentum, prove that something is really happening with AI. Understandable, because whoever waits falls behind. But it is like running a team by pulling random people off the street, giving them a desk and a login, and wishing them luck. No job description, no agreement on who decides what, no overview of who has access to what. Nobody would call that safe. Yet that is exactly how many organisations treat their agents today.

    Step 1: can you answer the four questions?

    The first test of an agent's safety is whether you can answer four questions within a minute. What exactly does this agent do? Which model does it run on? Which data does it work with? And since when has it been live? These are precisely the questions that always get asked the moment something goes wrong: a claim approved by mistake, an email sent to the wrong customer, advice based on outdated data.

    If you can't answer those questions right away, you don't have an agent in production, you have a risk in production. The answer is scattered across a Slack channel, a note from the developer who has since moved on, and the memory of whoever once set up the connection. The moment you need the answer, it isn't there. And that is unsafe by definition, regardless of how good the model itself is.

    Step 2: give every agent a passport

    The way to always be able to answer those questions is an agent passport: one card per agent that records everything in one place. It starts with a unique Agent ID, so that everyone in the company is talking about the same agent, and a status that shows whether the agent is in development, in testing or in production. It also records when the agent was created and when it was last updated, so it is immediately clear how long it has been running and when something last changed.

    The most important part describes what the agent is built on, because that is where the risk sits. Which model does it use, which version, and where is that model hosted, for example within the EU? What is its task, in one or two sentences that someone outside the team can understand, and when does it hand over to a human? And which systems and data sources does it depend on? That last question determines what happens when something changes or goes wrong somewhere.

    Take AG-017, our Warranty Claim Agent, as an example. It was created on 12 March 2026, last updated on 2 October 2026, and is in production. It runs on a mid-size model, hosted in the EU, version 2026-09. Its task is to check dealer warranty claims against the current policy and route exceptions to a human. For that it relies on three sources: the dealer system, the warranty policies and the parts catalogue. Six fields on one card, and anyone can see within a minute what this agent does, on what basis and since when. If the parts catalogue changes, you immediately know which agent is affected.

    Step 3: keep the passport alive

    A passport that is filled in once and then disappears into a folder makes nothing safer. That is why we attach three rules to it. The first is simple: no passport, no production. An agent without a completed passport does not go live, however convincing the demo was. That feels strict, but it is the same threshold you apply to a new employee.

    The second rule is that the passport is updated with every change of model or data. Whoever switches to a new model version or connects an additional data source updates the passport first. That way the card stays a reflection of reality rather than of how things were once intended. Most incidents don't happen at launch, but after a change nobody recorded.

    The third rule is that all passports are reviewed every quarter, exactly as you periodically check access rights. Does this agent still do what it says? Is the owner still with the company? Has the data source it relies on been replaced? Those questions belong in a fixed rhythm, not at the moment something goes wrong.

    Why this is safety, not paperwork

    It is tempting to dismiss this as governance, an extra layer of paperwork for the compliance department. But the passport is, at its core, a safety measure, for two reasons.

    The first is that agents increasingly work alongside people in the team. They pick up work, hand over work and make decisions a colleague used to make. When you work with someone, you want to know what that person does and where their responsibility ends. The same applies to an agent. A colleague who doesn't know the agent routes exceptions to them, or who thinks the agent checks something it doesn't, is a gap in your process. The passport closes that gap, including for those who didn't build it.

    The second reason weighs heavier. A person who makes a mistake makes it once and usually notices. An agent that is wired up wrongly makes the same mistake thousands of times, at scale, before anyone notices. One wrong data source, one misconfigured access, and the effect is not one wrong decision but a whole series. The more agents are running and the more they are connected to each other and to systems, the bigger that domino effect becomes. The passport makes visible where those connections are, so you can check them before they become a problem, and so that in an incident you know within a minute which agents are affected.

    The answer in one sentence

    You make sure an AI agent is safe by treating it the way you treat a new colleague: with a contract, a role and access rights from day one, recorded in a place where everyone can see it. Nobody calls that bureaucracy for an employee; it is the basis for letting someone work responsibly. There is no reason to skip that basis for agents, especially now that they do more and more work that really matters.

    So the question for everyone with agents in production: do you know, for every agent, what it does, which model it runs on, which data it works with and since when? If the answer is no, start today with the first passport. It takes an hour, and it is the difference between moving fast and moving blind.

    Job van den Berg, Mede-oprichter, AI Keynote Spreker & Techondernemer bij ai.nl

    // About the author

    Job van den Berg

    Mede-oprichter, AI Keynote Spreker & Techondernemer

    Tech-ondernemer (1989) met een achtergrond als socioloog (Research Master (MSc) in statistiek en sociologie) en een van de meest gevraagde keynote sprekers over AI en data in Nederland. Als mede-oprichter van Ai.nl, The Automation Group en Proxies leidt hij engineers die agentic AI van prototype naar productie brengen binnen enterprises. Op het podium vertaalt Job die hands-on praktijk naar concrete strategieën. Eerder was Job Chief Data bij o.a. DPG Media en Kantar. Hij is co-auteur van 5 boeken over AI waaronder 'AI Agents' en 'Handboek AI Strategie' en een veelgevraagd expert in de landelijke media.

    LinkedIn
    Newsletter

    Always up to date on AI.

    Once a month: cases, frameworks and concrete examples of what works in practice. No noise.

    No spam. Unsubscribe any time.