The Enterprise AI Operating Model

The argument everyone has

Sooner or later, every AI program reaches the same meeting. One side argues for a central AI team or center of excellence: one place for expertise, standards and risk control. The other argues that AI must live in the business units, close to the work, or it will never be adopted.

Both sides are describing a real failure mode.

Fully centralized programs become bottlenecks. A central team that must build, approve and run every AI initiative turns into a queue. Business units wait months, lose interest, or route around it.

Fully decentralized programs become sprawl. Every unit buys its own tools, negotiates its own contracts, invents its own risk approach and duplicates the same data work. Nobody can answer the board's simplest question: what AI are we actually running, and what could go wrong?

The routing-around is already happening. In Microsoft and LinkedIn's 2024 Work Trend Index, 75% of knowledge workers reported using generative AI at work, and 78% of those users said they were bringing their own AI tools rather than using ones provided by their employer. Shadow AI is usually not a discipline problem. It is a signal that the official route is slower or weaker than the unofficial one.

Decide by decision, not by org chart

The way out of the argument is to stop debating structure and start allocating decisions. An AI program involves a small number of recurring decisions: what to fund, which platforms and models to use, what risk tier a system falls into, who approves go-live, who watches it in production, and who can switch it off.

Each decision has a natural home. Some need enterprise consistency. Others need proximity to the work.

McKinsey's 2025 global survey found that organizations are already splitting decisions in this way. Risk and compliance and data governance are most often fully centralized, typically in a center of excellence. Technical talent and the adoption of AI solutions are more often run through a hybrid model, with some capability held centrally and some in business units. The survey also found that CEO oversight of AI governance was one of the factors most correlated with higher reported bottom-line impact from generative AI, yet only 28% of respondents said their CEO held that responsibility. On average, respondents reported two leaders sharing it.

That last finding matters. Shared ownership often means no single person can resolve a conflict between speed and control. A good operating model names the tie-breaker.

Centralize the controls, federate the delivery

A useful rule of thumb follows from the decision map: centralize what must be consistent, federate what must be close to the work.

The central function, whether it is called an AI office, a center of excellence or something else, should be deliberately small and should run a platform, not a project queue. Its job is to make the safe path the easy path: approved model access through a common gateway, shared evaluation tooling, reusable data products, standard contracts with vendors, and a risk-tiering process that takes days, not months.

Domain teams own the use cases. They own the workflow redesign, the adoption and the business outcome. They build on the central platform rather than around it.

Make governance proportional

The fastest way to create shadow AI is to send every use case through the same heavyweight approval. A meeting-summary assistant and a credit-decision model do not deserve the same process.

Proportional governance does two things at once. It moves low-risk work quickly, which removes the incentive to bypass the official route, and it concentrates scarce expert attention on the systems that can actually cause harm. International standards point the same way. The NIST AI Risk Management Framework and ISO/IEC 42001, the management-system standard for AI, both expect controls to scale with risk rather than be applied uniformly.

The tiering decision itself should be centralized and fast. Once a system is tiered, the controls for that tier should be known in advance, so teams can design for them from the start instead of discovering them at the approval meeting.

The operating model should change as you mature

The right model for an organization launching its first production AI systems is not the right model three years later. Most organizations move through three stages, and the roadmap should plan for the transitions rather than treat the first structure as permanent.

Stage 1: Centralized launch. A small central team builds the first few production capabilities, establishes the platform and risk process, and develops reusable patterns. Centralization is right here because expertise is scarce and the controls do not exist yet.

Stage 2: Hub and spoke. Business units with enough demand get embedded AI product teams, building on the central platform. The hub shifts from building to enabling. Most large enterprises should expect to spend the longest time here.

Stage 3: Federated on a platform. Domains deliver independently. The hub runs the platform, standards, assurance and portfolio view. This only works when the platform is mature and domain teams have real capability; moving here too early recreates sprawl.

The triggers for moving between stages should be explicit, such as the number of business units with sustained demand, platform adoption, and the share of use cases passing risk review without rework, rather than left to reorganization cycles.

Leadership: who owns AI?

Governments in the Gulf have moved faster than many companies on this question. In May 2024 the UAE Cabinet approved a new chief executive role for artificial intelligence across federal ministries and entities, and Dubai appointed 22 Chief AI Officers across its government entities the following month.

A Chief AI Officer can work well, but the title is less important than three conditions:

• A single accountable executive for the AI portfolio, with authority to resolve conflicts between business units, technology and risk.
• Visible CEO sponsorship of AI governance. The McKinsey findings above suggest this is not ceremonial.
• Clear boundaries with existing roles. The Chief Data Officer, CIO, CISO and Chief Risk Officer each own part of what AI depends on. Overlaps should be resolved in the decision-rights map, not in each project.

Measure the operating model itself

Most programs measure use cases. Few measure whether the operating model is working. Five indicators reveal it quickly:

Reviewing these quarterly, alongside the value metrics described in From AI Experimentation to Enterprise Value, gives leadership an early view of whether the program is speeding up or seizing up.

What to do in the next 90 days

1. Map current AI decisions as they actually happen today, including the informal ones. Most organizations find several decisions with no owner and a few with three.
2. Publish a one-page decision-rights matrix and name the tie-breaker for conflicts between speed and control.
3. Introduce risk tiering with pre-defined controls per tier, and set a target turnaround for low-risk approvals.
4. Stand up a common model gateway and approved tool list, then measure how much AI use moves onto it.
5. Agree the stage you are in and the triggers for the next one, so the operating model evolves on purpose.
6. Confirm CEO-level sponsorship of AI governance, and the single executive accountable for the AI portfolio.

For how to fill the portfolio this operating model governs, see How to Select Enterprise AI Use Cases.

Frequently asked questions

Should AI be centralized or decentralized in an enterprise?

Neither fully. Centralize decisions that need consistency, such as risk tiering, platforms, model access and data governance, and federate delivery, workflow redesign and adoption to business units. The balance should shift as the organization matures.

What should an AI center of excellence own?

A small set of enterprise services: the AI platform and model gateway, evaluation standards, risk-tiering and policy, reusable data products, vendor contracts and literacy. It should enable domain teams rather than deliver every use case itself.

Does every company need a Chief AI Officer?

Not necessarily, but every company needs a single executive accountable for the AI portfolio, visible CEO sponsorship of AI governance, and clear boundaries with the data, technology, security and risk leaders.

How do you stop shadow AI?

Make the approved route faster and better than the unofficial one: approved tools for common tasks, quick approval for low-risk uses, and a model gateway that is easy to adopt. Policy alone rarely works when the official path is slower.

Sources and further reading

• The state of AI: How organizations are rewiring to capture value, McKinsey & Company, 12 March 2025.
• The state of AI in 2026: On the road to ROI, McKinsey & Company, 25 August 2026.
• 2024 Work Trend Index, Microsoft and LinkedIn, 8 May 2024.
• New AI chief role in UAE federal units reflects long-term strategy, The National, 21 May 2024.
• ISO/IEC 42001:2023 Artificial intelligence management system, ISO.
• NIST AI Risk Management Framework, National Institute of Standards and Technology.