Designing the Enterprise Knowledge and Context Layer

Context, not the model, is the constraint

An enterprise assistant that gives a wrong answer has usually been given the wrong context: an outdated policy, a revenue figure computed with the wrong definition, a document the user should never have seen, or the right document buried among forty irrelevant ones. Swapping the model rarely fixes any of these.

The research supports this. In July 2025 Chroma tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, and found that performance varies significantly as input length grows, even on simple tasks. Earlier work by Liu and colleagues showed that models use information at the start and end of a long context better than information in the middle. Anthropic's engineering team described the response in September 2025 as context engineering: curating the smallest set of high-signal tokens for each step, across instructions, tools, retrieved data and history, rather than only polishing the prompt.

Most enterprises today build context separately for each assistant. The HR bot has its own document index, the finance copilot its own metric definitions, the service agent its own customer lookup and its own service-account permissions. Each team solves retrieval, definitions and access control again, slightly differently.

Our thesis is that context should be treated as a shared enterprise layer, built once and governed like data, with two non-negotiable properties: every piece of context carries its meaning and its permissions, and every consumer receives it through a standard interface. The model and the agent framework then become replaceable parts. The layer is where the durable investment sits.

Five kinds of context

"Context" covers several things with different owners, sources of truth and refresh rates. Designing the layer starts with separating them.

Metadata describes data: what a table or document is, who owns it, how it is classified, where it came from. Catalogs and lineage already hold most of this, and open standards exist for exchanging it. OpenLineage, a graduated LF AI & Data project, defines a common model of datasets, jobs and runs with extensible facets. For an agent, metadata answers "can I use this, and should I trust it?"

Semantics define business meaning: what "active customer", "net revenue" or "on-time delivery" mean and how to compute them. Without a semantic layer, an agent asked for revenue will write its own SQL against raw tables and produce a plausible, unofficial number. In September 2025 Snowflake, Salesforce, dbt Labs, BlackRock and RelationalAI launched the Open Semantic Interchange to define a vendor-neutral format for metrics, dimensions and relationships. Version 1.0 was published under Apache 2.0 on 27 January 2026, and in July 2026 the project announced it had entered the Apache Incubator as Apache Ossie. That gives enterprises a realistic path to define a metric once and expose it to BI tools and agents alike.

Knowledge is the content itself: documents, records, passages, and the indexes over them, including vector and graph representations. This is the retrieval layer most teams start with.

Memory is what an assistant retains about a user, a task or past interactions. Research such as MemGPT framed it as tiers, much like an operating system manages fast and slow memory. In the enterprise, memory is also a new data store holding personal and often sensitive information, and it needs an owner, retention rules and the same access controls as any other store.

Policy is the set of rules about who may see and do what: entitlements, classifications, consent and residency. It is the one kind of context the model should never be trusted to apply on its own.

A reference architecture

The architecture has four tiers and one cross-cutting control plane.

1. Sources remain the systems of record: ERP, CRM, document management, the warehouse or lakehouse, ticketing. The context layer does not replace them. Our guide to warehouse, lake, lakehouse and mesh covers that foundation.
2. Context services turn sources into usable context: the catalog and lineage service, the semantic layer, the knowledge index, the memory service and the entitlement service. Each has a named owner and an evaluation.
3. Delivery exposes the services through a small number of standard interfaces, increasingly Model Context Protocol servers for tools and resources, alongside ordinary APIs.
4. Consumers are assistants, agents, copilots inside SaaS products and BI tools. They should hold as little private context logic as possible.

The control plane runs through every tier: identity, policy evaluation, audit logging and tracing. If an answer is wrong or a document leaks, the trace should show which context was assembled, from which sources, under whose identity.

MCP as the delivery protocol

Anthropic released the Model Context Protocol in November 2024 to address a specific problem: every new data source needed its own custom integration for every AI application. MCP defines how an application (the host) connects through clients to servers that expose three things: resources (context and data), prompts (templated workflows) and tools (functions the model can call).

Its governance has since moved from a single vendor to a neutral body. On 9 December 2025 the Linux Foundation announced the Agentic AI Foundation, with MCP contributed by Anthropic alongside Block's goose and OpenAI's AGENTS.md, and platinum members including AWS, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft and OpenAI. The announcement cited more than 10,000 published MCP servers. Google's Agent2Agent protocol, which addresses agent-to-agent communication rather than agent-to-context, had moved to the Linux Foundation in June 2025.

The specification is also maturing in ways that matter to enterprise architects. As of 19 August 2026 the current revision is dated 28 July 2026. It removes the initialization handshake and protocol-level sessions so that each request stands alone, which makes MCP servers easier to run behind load balancers. It adds OpenTelemetry trace-context conventions, cache lifetimes on list and read results, and a formal deprecation policy with a minimum twelve-month window. It also deprecates the Roots, Sampling and Logging features, so new servers should not depend on them.

Three points keep MCP in proportion:

• MCP is a delivery protocol, not a governance layer. The specification itself says it cannot enforce its security principles at the protocol level; implementers must build consent, authorization and access control.
• Its authorization model is sound when used. For HTTP transports, the MCP server acts as an OAuth 2.1 resource server, tokens must be issued specifically for that server, and servers must not pass client tokens through to downstream APIs. The project's security guidance also covers confused-deputy attacks on proxy servers and recommends minimal, progressively elevated scopes.
• A server per source is the wrong unit. A context layer exposed as fifty thin MCP wrappers over raw systems moves the integration problem into the agent. Expose governed context services, such as a semantic-layer server or a permission-aware knowledge server, rather than raw databases.

Permissions must travel with context

Permission design is where most context layers fail. The common shortcut is a service account with broad read access, with the assistant trusted to show users only what they should see. That reverses the burden of proof.

Microsoft's documentation for its Copilot assistant states the stronger principle clearly: it only surfaces organizational data to which the individual user has at least view permissions, and its index honors the user's identity-based access boundary during grounding. The same document makes the corollary plain. The assistant inherits whatever permissions the tenant has, so oversharing that was invisible in a file share becomes visible the moment an assistant can search it.

Four design rules follow:

1. Act on behalf of the user. Agents should call context services with the user's identity, or a delegated token scoped to that user, not with a shared service identity.
2. Filter before the model reads. Entitlement filtering belongs in retrieval and in the semantic layer, before any content enters the context window. Redacting the answer afterwards is too late.
3. Scope memory to its owner. Memory written during one user's session must not be retrievable in another's, and memory that summarizes restricted content inherits that restriction.
4. Log the assembled context. For audit, record which resources and tool results entered the context for each answer, linked to the user and the trace.

Context engineering at run time

The layer supplies context; the agent still has to choose what to load. Anthropic's guidance describes techniques that map well onto an enterprise layer: hold lightweight references and load content just in time, compact long histories into summaries, write structured notes to external memory, and give sub-agents clean contexts that return condensed results.

Three enterprise-specific practices add to these:

• Ask the semantic layer for numbers. Questions about metrics should go to governed metric definitions through a tool call, not to free-form SQL over raw tables, and the answer should carry the metric's definition and owner.
• Prefer references over copies. Pass document identifiers and fetch content at the moment of use, so permissions and freshness are checked then, not when an index was last built.
• Budget the context window. Set an explicit budget per task for instructions, tools, retrieved content and history, and test answer quality at the budget, not only at the maximum the model accepts.

Gulf considerations

Three regional factors shape the design for organizations in Saudi Arabia and the UAE.

Classification as policy input. Under the NDMO standards each dataset carries an impact-based classification of Top Secret, Secret, Confidential or Public. In a context layer that label should be machine-readable metadata the entitlement service evaluates on every request, not a field in a spreadsheet. Our guide to the NDMO standards covers the method.

Residency of derived stores. Vector indexes, graph stores, caches and memory are copies of source data. They carry the same personal-data obligations as the source, including the cross-border transfer controls in Saudi Arabia's PDPL, and the same hosting requirements that the source's classification imposes. In practice, place them in the same jurisdiction and security zone as the data they were built from.

A bilingual semantic layer. Business terms in Gulf organizations live in Arabic and English, and often in transliterated forms. Defining each metric and entity once, with Arabic and English labels and synonyms, does more for Arabic answer quality than most model choices.

Design principles and how to test them

These are the same disciplines that keep retrieval reliable when the model changes, discussed in RAG, fine-tuning, continued pretraining or a custom LLM, and they need a clear owner in the enterprise AI operating model.

What to do in the next 90 days

1. Inventory the context you already build. List every assistant and agent, and for each one its indexes, metric logic, memory and the identity it uses to reach data.
2. Name owners for the five kinds of context: metadata, semantics, knowledge, memory and policy. Most organizations already have owners for the first and last; the middle three are usually unowned.
3. Put the top 20 metrics into a semantic layer with Arabic and English labels where relevant, and route agent metric questions to it.
4. Replace service-account access for at least one production assistant with user-delegated access and pre-retrieval filtering, and test it with users of different entitlements.
5. Stand up two governed MCP servers, one for knowledge and one for metrics, using audience-bound OAuth tokens, and connect two different assistants to them.
6. Turn on context tracing so each answer records the resources, tool calls and identity behind it.
7. Clean up oversharing in the document stores you plan to expose, before an assistant makes it visible.

Frequently asked questions

What is an enterprise context layer?

It is the shared set of services that supply AI assistants and agents with what they need to act correctly: metadata, business definitions, knowledge, memory and access policy. It is built once, governed like data and exposed through standard interfaces so that many assistants can reuse it.

Is MCP enough to govern enterprise AI context?

No. MCP standardizes how applications connect to context and tools, and its authorization model supports audience-bound OAuth tokens. The specification states that it cannot enforce security principles at the protocol level, so access control, consent and audit must be built into the services behind it.

Who governs the Model Context Protocol now?

Anthropic contributed MCP to the Agentic AI Foundation, announced by the Linux Foundation on 9 December 2025, alongside Block's goose and OpenAI's AGENTS.md. The current specification revision, as of 19 August 2026, is dated 28 July 2026.

Why does an AI agent need a semantic layer?

Without one, an agent asked for a business metric writes its own query and may compute it differently from finance or BI. A semantic layer defines each metric once, so agents and dashboards return the same number with the same definition and owner.

Sources and further reading

• Effective context engineering for AI agents, Anthropic, 29 September 2025.
• Context Rot: How Increasing Input Tokens Impacts LLM Performance, Chroma, 14 July 2025.
• Lost in the Middle: How Language Models Use Long Contexts, Liu et al., 2023.
• Introducing the Model Context Protocol, Anthropic, 25 November 2024.
• Linux Foundation Announces the Formation of the Agentic AI Foundation, Linux Foundation, 9 December 2025.
• Model Context Protocol versioning, current revision, accessed 19 August 2026.
• MCP 2026-07-28 key changes, Model Context Protocol.
• MCP 2026-07-28 authorization, Model Context Protocol.
• MCP 2025-11-25 specification overview, including security principles.
• MCP security best practices, Model Context Protocol.
• Linux Foundation launches the Agent2Agent Protocol project, Linux Foundation, 23 June 2025.
• Open Semantic Interchange specification finalized, Snowflake, 27 January 2026.
• Apache Ossie (Incubating): the new name for Open Semantic Interchange, Apache Ossie, July 2026.
• OpenLineage documentation, LF AI & Data Foundation.
• MemGPT: Towards LLMs as Operating Systems, Packer et al., 2023.
• Data, Privacy, and Security for Microsoft Copilot, Microsoft Learn.
• Data Management and Personal Data Protection Standards, Version 1.5, National Data Management Office, SDAIA.