The Question Your AI Will Never Ask, But Someone Must: The Operating Model's Missing Role

In Part I, we examined six failure modes that play out when organisations deploy AI without the right governance in place: objectives that encoded the wrong outcomes, models that scaled conviction without testing assumptions, systems that enforced the documented process over operational reality, AI that optimised silos while nobody owned coherence, metrics that worked while strategy quietly eroded, and capabilities that outran the judgement required to use them responsibly.

These failures share a single root cause: no one in the organisation had both the authority and the perspective to ask whether the system’s objective was right. What follows is the case for a specific role designed to fix that, and a blueprint specific enough to implement.

PART II: THE SOLUTION - OPERATING MODEL’S MISSING ROLE

Why your existing leaders can’t do this.

The reasonable objection: “I already have people for this. My CFO challenges objectives. My COO sees across silos. My CTO understands the technology. My General Counsel owns risk.”

They do. And none of them can perform this function. Not because they lack capability; because their incentive structures, information flows, and time horizons make it structurally impossible.

The CFO sees financial risk but doesn’t interrogate model logic. They can tell you a system’s ROI is declining. They cannot tell you its objective function is encoding bias. Financial controls sit downstream of the problem.

The CTO understands model architecture but not strategic consequence. They know a model is performing well against its training data. They are not positioned to judge whether “performing well” is producing outcomes that contradict the company’s values. Technical excellence and strategic alignment are different questions answered by different people.

The COO sees operations but not the interaction effects between automated systems. The sales-versus-risk conflict above is invisible to a COO unless someone maps it. Nobody in their reporting line will.

The Chief Legal & Compliance Officer evaluates systems one at a time, for compliance against existing regulation. They are not resourced to assess whether a portfolio of AI systems is strategically coherent, or to challenge an objective before it becomes a legal problem.

This is not a criticism of these leaders. It is a description of organisational physics. The architect role exists in the gap between their perspectives, the gap where Zillow lost half a billion, where Ofqual downgraded 280,000 students, where the Post Office wrongfully convicted 900 innocent people and the Dutch government took 2,000 children into care.
 

Your governance framework probably doesn’t cover this.

Most organisations with AI at scale have a responsible AI team, a model review board, or an ethics committee. Necessary. Not sufficient.

These teams review models in isolation. They assess individual systems for bias and compliance. They rarely examine how multiple AI systems interact, whether their combined outputs are strategically coherent, or whether the objectives they’re collectively optimising for still reflect what leadership intended. They are compliance-oriented, not strategically empowered. They report to the CTO, not the CEO. They can flag a problem. They cannot stop a deployment. And when their flags are overruled, nothing on paper records the decision. The architect role is designed to invert both halves of that pattern.

Every case in this piece shares the same structural feature: not the absence of governance, but the absence of anyone with both the authority and the perspective to challenge whether the system’s objective was right. Ofqual had a regulator. The Dutch tax authority had civil servants. The Post Office had auditors. Zillow had data scientists. None were empowered to challenge the logic before it caused harm at scale.
 

What the architect does.

The architect is a senior strategic function, not a committee, not a working group, not an additional responsibility stapled to an existing role. What follows is specific enough to implement on Monday morning.

1.    Pre-deployment objective review
Every AI system above a defined risk tier requires a signed objective review before deployment. The question is not “Does the model work?” It is “What happens to the rest of the organisation when this model succeeds?” The review covers the system’s explicit objective, the incentives it creates, the populations it affects, the second-order consequences of optimisation at scale, and the interaction with other AI systems in the portfolio. The architect can approve, require modification, or escalate to the CEO. This is a gate, not a suggestion.

2.    Cross-system coherence audit
Quarterly, the architect maps every AI system operating in overlapping decision spaces and tests for definitional consistency and objective conflicts. Do the sales model and the risk model share a definition of “good customer”? Does the fraud-detection system’s threshold align with the customer-experience team’s tolerance for false positives? The output is a coherence report to the CEO and the board, with findings, risk ratings, and required actions.

3.    Deployment authority
The architect can delay any AI deployment by 48 to 72 hours pending objective review. For the highest risk tier, systems affecting customers, pricing, credit, or public-facing decisions, the architect holds a mandatory sign-off. This mirrors how chief risk officers operate in regulated financial services: not a veto over every decision, but a defined gate for material risk.

The Dutch tax authority had civil servants who were supposed to review flagged claims. They had no information and no authority to challenge the model. That was not oversight. It was theatre. The architect’s deployment authority is designed to be the opposite: informed, empowered, and accountable.

4.    Strategic translation
The architect sits in board and executive committee discussions as a strategist, not a technologist. Their job is to explain what AI systems are doing to the business, not how models work. They are the person who says: “This system is working exactly as designed, and that is the problem.”

5.    Value creation
This is not a defensive role. Cross-system mapping routinely surfaces value that siloed teams cannot see: complementary data assets, contradictory models that can be reconciled into a single better one, objectives that can be reframed to unlock revenue the current configuration leaves on the table. In the sales-versus-risk example, the architect doesn’t just identify the £25 million conflict, they design the shared customer definition that lets both systems work in concert. The role pays for itself in the gap between silos.

Where the architect sits.

The architect does not replace existing functions. They fill the structural gap between them.

How the architect is held accountable.

A role defined by accountability must itself be accountable. Five mechanisms achieve this:

Quarterly coherence report. The architect delivers a formal assessment to the CEO and the board documenting every AI system reviewed, every objective conflict identified, every action taken, and every residual risk accepted. This is an auditable record. If an objective conflict causes harm and was not flagged, that is a failure of the architect function, visible, traceable, and consequential.

Objective-change tracking. Every pre-deployment review is logged. The number of deployments that proceed unmodified, the number modified, and the number escalated are reported quarterly. A sustained rate of zero modifications is a red flag; either the architect is rubber-stamping, or they’re not seeing the right systems.

Override transparency. Any instance where the CEO or board overrides the architect’s recommendation, proceeding with a deployment despite a flagged objective risk, is formally recorded, including the rationale and the residual risk being accepted. This ensures accountability does not quietly shift upward without trace and prevents the architect role from becoming a symbolic gate that can be bypassed without consequence. It also protects the architect from carrying scapegoat responsibility for risks the organisation knowingly chose to take. Accountability is shared, but it is explicit.

Post-deployment outcome review. Six months after deployment, every high-risk system undergoes an outcome review against the objectives set at the gate. Did the system produce the consequences the architect anticipated? Were there effects nobody predicted? The architect is accountable for the quality of their analysis, not for omniscience.

Commercial contribution. The architect tracks and reports value created through cross-system optimisation: conflicts resolved, shared definitions implemented, redundant models consolidated, revenue captured through reframed objectives. This is not a cost centre. The expectation is measurable commercial value within the first year. If the cross-silo visibility doesn’t generate it, the role isn’t working.
 

Who you hire.

VP or SVP level. Reports to the CEO. Standing invitation to the board’s risk or technology committee.

Background. The ideal candidate has led cross-functional programmes at the intersection of technology and strategy, a chief risk officer from financial services, a senior product leader from a technology company, or a strategy background with deep AI implementation experience. Technical fluency is non-negotiable: they must be able to interrogate a data scientist’s objective function and understand why it matters. But they are not a machine learning engineer. They are a strategic leader who speaks technology.

What to avoid. A pure technologist who cannot operate in board-level discussions. A pure strategist who cannot interrogate model logic. A committee; committees distribute accountability; this role concentrates it. And do not layer it under the CTO, which recreates the structural problem you are trying to solve.

Compensation. Benchmark against your chief risk officer or chief strategy officer. If you are not prepared to pay at that level, you are not serious about the function.
 

One hire, then a team.

One person cannot deliver everything described here. The pre-deployment reviews, the quarterly coherence audits, the post-deployment outcome work, the strategic translation across the executive team, at scale, this is the work of a function, not an individual.

The intention is to hire the most critical role first. That hire then builds the broader team needed to deliver the rest of the functions outlined above. Without that first appointment, there is no seat at the table where these decisions get made, and no ability to shape the direction from the start. Everything else follows from getting that one hire right.
 

The question that matters.

Who in your organisation can name what each AI system is optimising for and whether that’s the right thing? Who sees across silos and is accountable for coherence? Who owns outcomes, not outputs? Who operates in the ambiguity AI cannot, not in theory, but in practice, with authority, and with their own performance measured against results?

If the answer is unclear, that’s the finding.

The biggest risk in enterprise AI is not that the technology fails. It is that it succeeds and nobody is minding the direction. Zillow’s model didn’t fail. Ofqual’s algorithm didn’t fail. The Post Office’s system didn’t fail. They all succeeded, precisely and at scale, at the wrong thing.

That is not a technical problem. It is a leadership one.

And it has a specific solution: a named individual, with defined authority, clear accountability, and a mandate to ask the one question AI cannot ask itself.

Is this the right objective?