Regulated Enterprises Aren't Afraid of AI, They're Afraid of Decisions They Can't Explain

In late September I was invited to join the first HumanX Europe in Amsterdam, a conference built for C-level leaders from global brands. On the main stage, I sat down with Jacomo Corbo, co-founder and CEO of PhysicsX, for a panel moderated by Ron Miller called "The Deterministic Engine," which explored how to build autonomous systems that mission-critical industries can trust.

Across sessions and hallway conversations, the same three obstacles kept coming up: speed to market, a lack of trust, and legacy systems. Every leader I met wants the upside of AI, and almost none of them are being held back by the models themselves. What's in the way is everything around them.

The real reason customers ask us to switch AI off

I opened the panel with a story that still surprises people. Engineers love the AI capabilities in Axoniq and keep asking for more. Even so, some of those same customers have asked us to turn AI off entirely, because enabling it would trigger a six-month review involving 20 different people.

That isn't resistance to AI but a rational response to liability. Our customers work in finance, healthcare and government, where a decision nobody can explain quickly becomes a regulatory problem, and when the review process is that heavy, keeping AI off is the path of least resistance.

Our own research shows how widespread that pressure is. In a recent Axoniq survey of nearly 200 technology decision makers in enterprise organizations, 57% of respondents said they are concerned about AI regulation such as the EU AI Act. When we asked who should be responsible for ensuring an AI code analysis tool complies with those rules, 83% pointed inside their own organization: 47% said themselves or their team, and 36% said their legal or compliance function. Only 11% said the tool vendor.

Who is responsible for AI compliance? In Axoniq's survey, 83% of respondents said responsibility for an AI tool's compliance sits inside their own organization, with only 11% placing it on the tool vendor.

That last number should matter to anyone buying AI. EU AI Act compliance is not something you can hand off in a vendor contract, and sector rules keep raising the bar, from explainability requirements for government AI to regulated AI in financial services. If your teams will own the outcome, they need systems that show them what happened and why without a months-long investigation every time.

Legacy systems are the biggest of the AI adoption challenges

My hot take on stage was that you need a different foundation to run AI at scale in a meaningful way. Most enterprise architecture still follows patterns from the 1970s and 1980s, which store the current state and throw everything else away. That falls short when AI needs history with causality in it, meaning the sequence of decisions and the reasons behind them rather than a snapshot of the latest result.

Jacomo described the same wall from the industrial side. "We are hitting the fundamental limits of what today's software tools can do," he said, noting that much of the engineering software his customers rely on has barely changed since the late 1990s. Finance has its mainframes and COBOL, manufacturing has its own entrenched systems of record, and nobody is ripping those out overnight.

This is why I believe technical debt is now a boardroom issue. It is no longer a line item engineering manages quietly, because it directly determines how fast a company can adopt AI and how much risk it carries along the way. AI Today summed up my view well: as AI agents take on more decisions, enterprises need infrastructure that captures not just what happened, but why.

Legacy system modernization without a rip-and-replace

The tension in the room is still very real. Established companies know what works today, and they also know they have to move into a new world while keeping the old one running. My answer is to slow down to speed up by rebuilding the foundation incrementally rather than all at once.

At Axoniq, that starts with Discovery. Our agentic engine goes into legacy systems and maps the decisions that have been made there and how they connect. From that map, leaders can see which pieces are worth modernizing first and which are good enough to leave alone while still yielding insight. A discovery engine to help with legacy modernization sparked more interest in my HumanX conversations than anything else, and I think that's because it answers the first question every CIO asks: where do we start?

Decision explainability is not observability

The vendor floor told its own story. Nearly everyone was talking about agents, control planes and observability, and many of those pitches sounded interchangeable. The idea that consistently made people stop and lean in was decision explainability.

It was also the idea most often confused with observability, which tells you whether a system is healthy and where it's slow. Decision explainability tells you why a specific decision was made, including the data, rules and inputs the system or agent was working with at that moment. That's a different question, and in a regulated business it's the one your auditor, regulator and customers will actually ask. It's about the context a decision was made in rather than the inner workings of a model. That's what makes explainable enterprise AI possible, from an AI audit trail that holds up to agentic AI governance that works in practice. We should demand that every company running AI in mission-critical systems can answer one simple question: why did AI make that decision?

Lead with outcomes, then earn the right to automate

One of my most useful lessons from the week was about how we talk. When I led with outcomes such as faster AI adoption, decisions you can defend, and modernization without a rewrite, conversations moved quickly. The underlying architecture is still what makes those outcomes possible, but executives buy the outcome, and they're right to.

The same outcome-first thinking applies to guardrails. Internal workflows can usually run with more freedom, while AI that touches external or sensitive data needs tighter controls. In mission-critical settings, human in the loop AI still matters because experts need to validate and certify what the system does, and the infrastructure underneath should make that fast rather than painful.

What I'm taking home from HumanX

HumanX made it clear that the AI conversation among enterprise leaders has matured. The question is no longer whether to adopt AI but how to adopt it in a way you can stand behind. For me, this is personal. The technology behind healthcare AI, financial AI and the AI that tests the airplanes, cars and trains our loved ones ride in every day should be battle-tested and provable. I want that for myself, for our kids, and for their kids to come. I expect the organizations that pull ahead in AI governance for regulated industries won't be the ones with the flashiest agents. They'll be the ones that can account for every decision their systems make and still change direction without a rebuild.

If that's the problem on your desk, I'd love to hear your perspective. Watch our full HumanX session, or reach out to the Axoniq team to talk about where your own decisions are getting lost.