Pre-launch preview · Stepwell Technologies Pvt Ltd (incorporation in progress) · details marked [ ] are being finalised
Home / Insights / AI governance
AI governance

Agentic AI that survives an audit

Analysts expect many agentic AI projects to be cancelled over cost, value and risk. A practical control set mapped to the NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

Key takeaways

  • Analysts expect many agentic AI projects to be cancelled over cost, value or risk controls; two of those are engineering problems.
  • Use frameworks your auditors already know: NIST AI RMF, ISO/IEC 42001, the EU AI Act and OWASP's LLM Top 10.
  • Seven controls (register, risk assessment, least privilege, approval gates, evaluation, audit trail, kill switch) cover most buyer questions.
  • For a first release, a fixed evidence pipeline with validated citations usually beats an open-ended agent.

Agentic AI (systems that plan and take multi-step actions using tools and APIs) has moved from demos into budgets. The attrition forecast is just as clear. In June 2025 Gartner predicted that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls.[1]

Two of those three causes are engineering and governance problems, not model problems. Agents that can't show their value, or can't show they are under control, get switched off when the first audit, incident or budget review arrives. This article sets out a control set that holds up in all three.

Start from the frameworks your auditors already know

You don't need to invent a governance model. Three public references cover most of what a regulated buyer will ask:

  • NIST AI Risk Management Framework 1.0 (January 2023) organises AI risk work into four functions: Govern, Map, Measure, Manage.[2] NIST's Generative AI Profile (NIST AI 600-1, July 2024) extends it with risks specific to generative systems, such as confabulation and information security.[3]
  • ISO/IEC 42001:2023 specifies requirements for an AI management system: policy, roles, risk and impact assessment, and continual improvement. It's certifiable in the same way as ISO/IEC 27001.[4]
  • The EU AI Act entered into force on 1 August 2024. Prohibited practices and AI-literacy duties applied from 2 February 2025, general-purpose AI model obligations from 2 August 2025, and most remaining provisions from 2 August 2026. Following the AI Omnibus amendment, the European Commission lists the high-risk deadlines as 2 December 2027 (Annex III use cases) and 2 August 2028 (AI in regulated products).[5]

For security, the OWASP Top 10 for LLM Applications names the failure modes that matter most for agents, including prompt injection and excessive agency.[6]

Seven controls, mapped

ControlMinimum standardNIST AI RMFISO/IEC 42001 theme
1. Use-case registerOwner, purpose, users, data, risk tier and review date for every agentGovern, MapAI system inventory, roles
2. Impact and risk assessmentWho is affected if it's wrong? What's the worst credible action?MapAI risk and impact assessment
3. Least-privilege toolsRead-only by default; write tools scoped to one system and one actionManageOperational controls
4. Approval gatesA named human approves consequential actions; the agent proposes, people decideManageHuman oversight
5. Evaluation setReal, labelled examples; accuracy, safety and cost measured on every changeMeasurePerformance evaluation
6. Audit trailInputs, retrieved evidence, tool calls, outputs and approvals logged and replayableGovern, MeasureDocumented information, monitoring
7. Kill switch and spend limitsOne switch to disable tools; budget alerts per use caseManageOperational planning and control

The mapping is a starting point for discussion with your risk team, not a certification crosswalk.

Design patterns that make the controls cheap

Prefer a fixed pipeline over open-ended autonomy for the first release

For a first production use case, a deterministic pipeline usually beats a free-roaming agent: collect a fixed evidence pack, make one model call with a strict output schema, validate it, then ask a human. It's easier to test, cheaper to run and simpler to explain to an auditor. Add tool use only when the evaluation set shows the fixed pipeline is missing information.

Make the model cite its evidence, and check the citations in code

Every claim in an agent's output should point to an evidence ID. A validator then checks that each ID exists and that quoted numbers match the source. Unsupported claims are removed or flagged before a human sees them.

Calculate confidence with rules, not the model's opinion

Self-reported model confidence is a weak signal. Rule-based confidence ("high only if two independent sources agree and no contradicting evidence exists") is explainable and testable.

Treat retrieved content as untrusted input

Logs, tickets, emails and web pages can contain instructions. Keep them in clearly delimited data sections, never give the model a tool it doesn't need, and test with planted injection strings.

What "survives an audit" looks like

When the risk reviewer asks "what did the system see, what did it say, and who decided?", you should be able to answer for any past run in minutes. If you can, you've also answered the value question, because the same logs show how often the recommendation was accepted and how much time it saved.

The agent proposes. A named person decides. The log proves both.

A 30-day starting plan

  1. Week 1: pick one use case; write the register entry and the risk tier; agree one business measure.
  2. Week 2: build an evaluation set of 30–50 real, labelled examples, including hard and adversarial cases.
  3. Week 3: build the fixed pipeline with citations, validator, approval gate and logging.
  4. Week 4: run the evaluation; review results with engineering and risk together; decide go, fix or stop.

This article is general information, not legal, regulatory or financial advice. Figures are quoted from the public sources listed below as at the publication date; check the source for updates.

Sources

  1. Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027", press release, 25 June 2025. gartner.com
  2. NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023. nist.gov
  3. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024. doi.org/10.6028/NIST.AI.600-1
  4. ISO/IEC 42001:2023, Information technology: Artificial intelligence: Management system. iso.org
  5. European Commission, "AI Act: regulatory framework for AI" (application timeline, including AI Omnibus changes), accessed November 2026. digital-strategy.ec.europa.eu
  6. OWASP, Top 10 for Large Language Model Applications. genai.owasp.org