Private AI vs public AI: where your data goes

What is the difference between private and public AI?

Public AI is a model you rent through a third-party API, where your data leaves your network to be processed on shared infrastructure you neither own nor inspect. Private AI runs inside infrastructure you control, on-premises or in an isolated cloud, so sensitive data never crosses your governance boundary. For UK businesses the practical question is not which model is smarter, it is where your data goes and who can see it.

OpenKit designs and deploys private AI for UK organisations that need the capability without the exposure, with data held inside UK jurisdiction and inference logged for audit. We hold ISO 27001, ISO 9001, and Cyber Essentials, and operate to UK GDPR, and we design deployments to help you meet your own obligations. This guide compares public, private and hybrid AI on the things a skeptical CTO actually asks about, then gives you a UK data-residency and security checklist you can use in a vendor review. Last updated 10 August 2026.

Public, private and hybrid AI, compared

Most teams do not face a binary choice. The honest answer is a spectrum, and the right pick depends on data sensitivity, volume and how much control you need over the model itself. The table below sets out the trade-offs without the marketing gloss.

The pattern we see across UK engagements is hybrid: route a marketing draft or a public summary to a public model for speed, and keep anything containing personal, regulated or proprietary data on a private path. The deciding factor is the data classification, not the task.

Is our data used to train the public LLM we use?

It depends entirely on the tier and the contract, not on a single toggle. Most enterprise API tiers contractually exclude your inputs from training and offer zero-retention processing, but consumer and free tiers frequently reserve the right to use your data. The safe assumption is that anything you cannot point to in a signed data-processing agreement may be retained.

This is where public deployments get uncomfortable for a regulated buyer. A zero-retention promise is a contractual assurance rather than a technical guarantee, and a misconfiguration on either side can still expose proprietary code or customer data. Private AI removes the question: when the model runs inside your boundary, there is no third party to retain anything, and you can demonstrate that in an audit rather than assert it.

Where is our data stored if we use private AI in the UK?

With a private deployment you choose the location, and OpenKit builds in UK regions or fully on-premises so data, vector stores and inference logs stay inside UK jurisdiction. The major public providers do offer region selection and residency commitments on their enterprise tiers, so this is not the gap it once was. What differs is how much of the pipeline the guarantee covers: residency on the inference endpoint says nothing about where abuse-monitoring logs, caches or support access sit, and those are the parts a private deployment lets you point at directly.

This matters for GDPR accountability under the UK GDPR and Data Protection Act 2018. Keeping the whole pipeline in a UK boundary lets you answer the regulator’s questions plainly: where the data sits, who the processor is, how long records are kept and how they get deleted. The ICO’s guidance on AI and data protection expects exactly this kind of documented data-flow understanding, which is far easier to produce when you control the infrastructure.

A note on retrieval and the right to erasure

A common private pattern is retrieval-augmented generation, where the model answers from a separate knowledge base rather than from anything baked into its weights. That makes erasure tractable rather than automatic. Deleting someone’s records from the retrieval store stops them being retrieved, but the request is only complete once you have also worked through the copies: cached responses, search indexes, conversation logs, and backups, which usually get handled through a documented retention cycle rather than an instant delete. Design for that up front and erasure is a process you can evidence. Trying to remove specific personal data that a public model may have absorbed during training is far harder and often impractical, which is one of the strongest practical arguments for keeping sensitive data out of public training paths.

That pattern is what Marella is built on, the private knowledge platform we build and operate. Where the data sits, who can reach it and what gets logged are written down for each deployment, so a risk owner can check rather than take it on trust.

How do we prevent shadow AI?

Shadow AI is your staff pasting company data into unapproved public tools because the sanctioned route is slower or does not exist. You reduce it the same way you reduced shadow IT: give people a sanctioned alternative that is at least as easy to use, publish a short acceptable-use policy, and log usage so you can see what is actually happening. Banning tools without a replacement pushes the behaviour underground rather than stopping it.

The mistake we see most often is leading with prohibition. A flat ban tells a busy team to find a workaround, and they will. The version that works pairs a clear, sensitivity-based policy (“public data is fine in tool X, anything with customer or financial data goes through our private assistant”) with a private assistant good enough that nobody wants the workaround. Logging then turns an invisible risk into a managed one.

UK data-residency and security controls checklist

Use this in a vendor review or before signing off an internal deployment. It is the set of controls that separate a defensible AI deployment from a liability, framed for UK obligations. Where a number cannot be sourced honestly, treat it as a question to ask, not a claim to accept.

OpenKit holds ISO 27001, ISO 9001, and Cyber Essentials, and operates to UK GDPR, and we design private deployments to help UK organisations meet their own obligations. We do not claim SOC 2, HIPAA or ISO 42001 certification, and any EU AI Act or ISO 42001 work is about helping you align toward those standards rather than certifying you against them.

Security and compliance: why the choice matters

For a CTO or risk owner, the public-versus-private decision is part of your legal strategy, not just an architecture preference. When AI only summarised public articles, an API was low-risk. Now that models read internal databases, draft contracts and act on customer data, the deployment model decides whether you can stand behind the system in an audit.

The acute risks of routing sensitive data through public services are concrete: a retention misconfiguration that exposes proprietary data, prompt-injection attacks against public endpoints, and the inability to forensically investigate a bad output in an opaque service. Private deployments answer each of these by keeping the model, its data and its logs inside one controllable boundary, which is also what makes documented data governance and human oversight realistic.

Worked example: legal document review

OpenKit built BAiSICS, a private AI platform for commercial-lease and property-document review, where client confidentiality ruled out pasting documents into a public model. The system runs on bespoke OCR and a private LLM workflow hosted in an AWS UK region, with every extracted field traceable to its source so users can verify rather than trust the output.

On the same set of historical leases marked up by senior partners, BAiSICS reaches 96% extraction accuracy and cuts review time by roughly 92% against manual work, saving its customers more than £200,000 a year. The point is not the model: it is that a private architecture made an accuracy and confidentiality bar viable that generic public tools failed to clear.

Worked example: public-sector air quality

For Air Aware, built for four London boroughs and since joined by a fifth, the raw air-quality data is public but its use in public-health policy is sensitive. OpenKit built a conversational AI grounded in the boroughs’ own data and kept inside UK jurisdiction, and residents stay with it for an average of five minutes and two seconds a session. Sovereignty and usefulness were not a trade-off here; the controlled boundary was what let the boroughs put it in front of the public.

Which approach fits your business?

Work down a short decision path rather than starting from the technology. Classify the data first, because the data, not the use case, sets the floor for what you can use.

1. Is the data genuinely public, with no personal, regulated or proprietary content? Public AI is fine and usually cheapest for sporadic use.
2. Does the prompt or output contain personal, regulated or proprietary data? Treat private AI as the default and document why if you deviate.
3. Is the workload high-volume and continuous? Fixed-cost private compute often wins on total cost once usage is steady, and removes per-token budget volatility.
4. Do you have a mix of both kinds of work? A hybrid architecture that routes by sensitivity is usually the pragmatic answer.

If you want help running that classification and standing up a deployment that holds up to an audit, OpenKit’s private AI service covers on-premises builds, isolated UK cloud enclaves and zero-data-retention architectures, all designed around UK data-protection obligations.