RAG, Fine-Tuning, Continued Pretraining, or a Custom LLM?

The short answer

Every enterprise that wants a model to "know our business" eventually faces this choice. There are five ways to adapt a large language model, and they differ by orders of magnitude in cost, effort and risk.

Provider capabilities in this article are as of 16 August 2026. They change frequently.

For most enterprise use cases the answer is the bottom two rungs: a well-designed prompt with the right context, and retrieval-augmented generation (RAG) over governed enterprise content. Fine-tuning earns its place when you need consistent behaviour at scale: a fixed output format, a classification scheme, a tone, a tool-calling pattern. Continued pretraining is for genuine language or domain gaps that retrieval cannot bridge. Training from scratch is a sovereign or strategic decision, rarely an accuracy-per-dollar one.

The rest of this article explains why, and what has changed in 2026.

Facts and behaviour are different problems

The most useful distinction is between teaching a model what is true and teaching it how to behave. The research is unusually consistent on which tool does which.

Fine-tuning is a poor way to add facts. A Microsoft study comparing knowledge-injection methods found that RAG consistently outperformed unsupervised fine-tuning, both for knowledge the model had partly seen in pretraining and for entirely new knowledge. A Google and Technion study presented at EMNLP 2024 went further: models learn new facts slowly through fine-tuning, and as they do, their tendency to hallucinate increases. The authors' conclusion is that models acquire knowledge mostly in pretraining, and fine-tuning mainly teaches them to use what they already know.

Retrieval cannot fix behaviour. RAG puts the right passages in front of the model. It does not make the model follow a 40-field output schema reliably, write in your regulatory house style, or classify tickets into your 120 categories with consistent boundaries. That is what fine-tuning is good at.

The two combine. A separate Microsoft case study in agriculture found fine-tuning added more than 6 percentage points of accuracy and RAG added a further 5 on top, so the effects were cumulative. Berkeley's RAFT work showed a related pattern: fine-tuning a model to use retrieved documents well, including ignoring irrelevant ones, beats either method alone.

The practical rule: let retrieval own facts and tuning own behaviour. Facts change, need citations and need access control, and retrieval handles all three. Behaviour is stable, and tuning makes it reliable and cheaper to run at volume.

2026: the market is making the decision for you

The fine-tuning landscape changed materially this year, and many enterprise roadmaps have not caught up.

According to OpenAI's deprecations page, self-serve fine-tuning is being wound down in steps. Since 7 May 2026, organizations that had never run fine-tuning cannot start. Since 2 July 2026, organizations without recent use of a fine-tuned model have lost the ability to create new jobs. From 6 January 2027, existing customers can no longer create new fine-tuning jobs at all. Fine-tuned models keep serving until their base model is deprecated, and several, including fine-tuned GPT-4 and GPT-4.1-nano, are scheduled to shut down on 23 October 2026, with plain base models listed as the migration path.

Anthropic does not offer fine-tuning of Claude models through its own API. The managed route has been through Amazon Bedrock, where the documentation lists Claude 3 Haiku, a 2024 model, as the tunable Claude model. Microsoft Foundry and Google's Gemini platform do still offer supervised, preference and, in some cases, reinforcement fine-tuning on selected models.

Two lessons follow:

1. A fine-tuned model on a closed platform is a depreciating asset with an expiry date set by the vendor. When the base model retires, the tuning is lost and must be rebuilt, re-evaluated and re-approved. Budget it as operating cost with a life of roughly a year, not as a durable asset.
2. Strategies that depend on fine-tuning a closed frontier model now carry platform risk. Where tuning is genuinely needed, open-weight models that you can host and version yourself are increasingly the safer foundation.

Long context and caching changed the maths

Two other shifts make the bottom rung of the ladder more capable than many teams assume.

Context windows are now very large. Leading frontier models accept around a million tokens, enough for several long manuals or a substantial policy library in a single request.

Prompt caching makes repeated context cheap. All three major API providers now discount cached input heavily, typically charging around a tenth of the normal input price, and less on some models. For a stable, moderate-sized body of content that many requests share, placing it directly in a cached prompt can be simpler and cheaper than building and operating a retrieval pipeline.

That does not make retrieval obsolete. Research on long-context models has repeatedly found that they use information at the start and end of a long prompt better than information buried in the middle. And large, frequently changing or permission-restricted content still needs retrieval, because you cannot put a million documents in a prompt, and you must not put documents a user is not entitled to see in front of the model on their behalf.

A reasonable rule of thumb:

• Small, stable, shared content (a product manual, a policy set): try long context with caching first.
• Large, changing or permissioned content (case files, contracts, customer records): use retrieval, with access control enforced at retrieval time.
• Anything that must be cited or audited: use retrieval, so every answer can point to its source.

The custom-model trap

Training your own model from scratch is the most expensive option and, for most enterprises, the least defensible. The best-documented example is Bloomberg's.

In 2023 Bloomberg published BloombergGPT, a 50-billion-parameter model trained on 569 billion tokens, including a large proprietary financial corpus, using 512 GPUs for about 53 days. It was a serious, well-executed effort by an organization with four decades of unique financial data.

Within months, researchers from JPMorgan AI Research and Queen's University compared it with GPT-4, a general model given only a few examples or none. GPT-4 matched or beat BloombergGPT on most of the finance benchmarks they compared, sometimes by a wide margin. BloombergGPT kept an edge in named-entity recognition.

The lesson is not that Bloomberg was wrong to try. It is about timing: a custom model competes not with today's general model but with the next two releases. Any business case for continued pretraining or a custom model should assume general models will keep improving, and should rest on something general models cannot provide, such as data that cannot leave your control, a language they handle poorly, or strategic independence.

Arabic, sovereignty and the Gulf

For organizations in Saudi Arabia and the UAE, two factors often decide the question before accuracy does: language and data residency.

Residency. None of the managed fine-tuning services reviewed for this article (OpenAI, Microsoft Foundry, Google's Gemini platform and Amazon Bedrock) listed a training region in Saudi Arabia or the UAE as of August 2026. Where data protection law, sector regulation or data classification requires training data to stay in-country, the realistic options are retrieval on a model served from an in-region endpoint, or tuning an open-weight model on infrastructure you control. Our guide to the NDMO standards covers the classification and personal-data obligations that drive these decisions in the Kingdom.

Language. The region now has credible Arabic-first models, and their build routes are instructive:

Notably, two of the three national efforts used continued pretraining of an existing model rather than starting from zero. If national programmes with substantial resources chose that route for language depth, it is a strong signal for enterprises: continued pretraining of an open model is the cost-effective path when a real language or domain gap exists.

For many Arabic use cases, a strong general model plus Arabic retrieval will be sufficient, particularly for Modern Standard Arabic documents. Dialect-heavy conversational use, culturally specific tone and strict in-country hosting are where regional models earn their place. In every case, test candidates head-to-head on your own Arabic evaluation set rather than relying on published leaderboards.

A decision guide

Whatever the route, three disciplines apply:

1. Build the evaluation set first. Collect real questions and tasks with known good answers before choosing a method, and keep the set independent of any vendor's tooling.
2. Separate the layers. Keep enterprise knowledge in governed, retrievable sources, and keep behaviour in prompts or tuned adapters. This lets you swap the base model when a better one arrives without losing your investment.
3. Plan for model change. Assume the base model will be replaced within a year. Retrieval indexes and prompts carry over with re-testing; fine-tuned weights on closed platforms do not.

Frequently asked questions

Is RAG or fine-tuning better for enterprise knowledge?

For facts, RAG. Research shows retrieval outperforms fine-tuning for injecting knowledge, and fine-tuning on new facts can increase hallucination. Fine-tuning is better for consistent behaviour, such as formats, classification and tone. The strongest systems often combine both.

Is OpenAI discontinuing fine-tuning?

OpenAI's deprecations page sets out a staged wind-down of self-serve fine-tuning: closed to new organizations from 7 May 2026, and no new jobs for existing customers from 6 January 2027. Existing fine-tuned models run until their base models are deprecated.

When is continued pretraining worth it?

When a language or domain gap is too large for retrieval and prompting to bridge, typically with billions of tokens of in-domain text available. Saudi Arabia's ALLaM and TII's Falcon Arabic were both built by continued training of existing models.

Should an enterprise train its own LLM from scratch?

Rarely. BloombergGPT, a well-resourced example, was matched or beaten on most finance benchmarks by GPT-4 within months. Training from scratch is usually justified by sovereignty or strategic independence rather than accuracy per dollar.

Sources and further reading

• Deprecations, OpenAI. Fine-tuning wind-down and fine-tuned model retirements, accessed 16 August 2026.
• Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs, Ovadia et al., 2023.
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?, Gekhman et al., EMNLP 2024.
• RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture, Balaguer et al., 2024.
• RAFT: Adapting Language Model to Domain Specific RAG, Zhang et al., 2024.
• Lost in the Middle: How Language Models Use Long Contexts, Liu et al., 2023.
• BloombergGPT: A Large Language Model for Finance, Wu et al., 2023.
• Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics?, Li et al., EMNLP 2023.
• ALLaM: Large Language Models for Arabic and English, Bari et al., 2024.
• Jais 2 release, Inception, Cerebras and MBZUAI, 9 December 2025.
• Falcon Arabic and Falcon-H1 launch, Technology Innovation Institute, 21 May 2025.
• Fine-tune models in Microsoft Foundry, Microsoft.
• Customize your model with fine-tuning, Amazon Bedrock documentation.