Why most people will run AI locally

In March 2019, Google squeezed its speech recogniser into 80 MB and put it inside the phone keyboard. Nobody had to choose “local dictation”. People only noticed that talking to their phone now worked in a tunnel.

We think the AI you chat with is heading the same way. Our forecast: within a few years, most people who use AI will, most of the time, be using a model that runs on their own phone or computer, and many of them will not know it. That is a prediction, not a measurement. Nobody publishes a number today for what share of AI use is local, and we will not invent one. This post lays out the evidence we could source, says what is still missing, and ends with what would prove us wrong.

Dictation, translation and photo search already made this move

New AI features tend to start on servers, because that is where the big computers are. Then the model shrinks, the chips in phones improve, and the feature moves onto the device. It has happened often enough to count as a pattern.

Take dictation. Google’s 2019 announcement said the compressed model was 80 MB, ran faster than real-time speech on a single processor core, and was “always available, even when you are offline”. A year earlier, Google Translate’s offline mode got the same kind of model at 35 to 45 MB per language set. Later in 2019, Live Caption arrived on Android working “completely on-device”.

Apple followed the same path. Photo search has used on-device facial, object and scene recognition since iOS 10 in 2016. The Translate app launched in 2020 with an on-device mode for offline use. In 2021, Apple announced that the audio of Siri requests would be processed on the iPhone by default.

Two things stand out. First, none of these moves needed users to care about where the work happened. The feature simply got faster and stopped failing without a signal. Second, once a feature moved, it stayed. Chat-style AI is a much bigger job than dictation, which is why it started in the cloud. The question is whether it follows the same path, and the next four sections are our reasons for saying yes.

The AI you never installed already runs on your device

The strongest sign is what the companies that make operating systems and browsers have shipped. A general language model, the kind that rewrites, summarises and answers questions, now comes built into the software most people already use.

Apple’s version is the clearest. Since iOS 26, any app can send requests to a 3 billion parameter model on the device. Apple told developers the features work offline and use “AI inference that is free of cost”. This year’s release brought a new on-device model. We covered Apple’s device-and-cloud design in our Siri post.

Google did the same in the browser. Chrome’s Summarizer and Translator APIs have been stable since Chrome 138, and the general-purpose Prompt API became stable in Chrome 148 in May 2026. A web page can now ask a model inside the browser to do the work, and Chrome’s documentation says “no data is sent to Google or any third party when using the model”. Firefox translates pages on the device too.

This matters for the forecast because of who decides. When a notes app adds a “summarise” button, its developer picks where the summary runs. The built-in model costs that developer nothing per use and needs no server. Most people will never pick local AI. The apps they already use will pick it for them.

There is early evidence that people use AI without knowing where it runs, or even that it is AI. Samsung says Galaxy AI is on more than 400 million devices. A survey of 2,000 Americans that Samsung commissioned found 90% use AI features on their phone and only 38% realise they do. Galaxy AI mixes on-device and cloud features, and Samsung does not publish the split, so treat this as a sign of invisible use, not of local use.

New devices qualify by default, and old ones do not

A built-in model is only useful on a device that can run it. Here the picture is mixed, and it is the main reason our forecast says “within a few years” and not “today”.

The good news is that the bar is now part of the standard product. Every iPhone since the iPhone 16 qualifies. Microsoft’s Copilot+ label requires a neural chip and 16 GB of RAM, and Windows features such as Recall are built so that “no internet or cloud connections are required”. Nobody has to shop for an “AI computer”. Buying a current one is enough.

The bar is also coming down. Chrome first needed a capable graphics chip, then added support for ordinary processors in Chrome 140. Microsoft’s Edge team is previewing a model that runs on devices without a GPU. Google says the newest Gemini Nano is up to 4 times faster and uses up to 60% less battery than earlier versions.

The bad news is the “leaves out” column. Budget phones, older iPhones and 8 GB laptops are a large share of what people own, and they are replaced slowly. Analysts publish forecasts for how fast AI-capable devices will spread. We could not read the primary releases ourselves, so we are not quoting them. If you want to know where your own machine stands, our memory guide for local AI has the numbers.

People who get a choice are already picking local

Built-in models show what vendors are doing. A different group shows what users want: people who went out and installed a local model themselves, when a free cloud chatbot was one click away.

These numbers describe an early, technical crowd. Ollama is a tool for running open models on your own computer, and it reports serving 8.9 million developers. Its code has more than 186,000 stars on GitHub. Google says its open Gemma models have been downloaded over 400 million times.

None of this proves that most people will follow. Developers adopt things early that the public never touches. It does show real demand from people who have used both kinds of AI and still chose to run their own. In the 2025 Stack Overflow survey, Ollama was also the most-used framework among the 3,398 developers building AI agents, at 51%.

Small models became good enough for everyday jobs

A model has to fit in a device’s memory to run there, so local AI depends on small models being useful. Model size is counted in parameters, the internal numbers a model learns. Fewer parameters means less memory and less power.

Stanford’s 2025 AI Index tracked the smallest model able to score above 60% on MMLU, a broad knowledge test. In 2022 that was a model with 540 billion parameters. By 2024, Microsoft’s Phi-3-mini did it with 3.8 billion, which the report calls a 142-fold reduction. The first model needed a data centre. The second fits on a phone.

That is the technical reason the built-in models above exist at all. Apple’s on-device model has 3 billion parameters. Rewriting an email, summarising a page or pulling dates out of a document does not need the largest model in the world, and those jobs are most of what people ask for. We tested how far laptop-sized models go in our GPT-4-class laptop post.

Small does not mean equal. The 2026 AI Index reports that as of March 2026 the top closed model led the top open one by 3.3%, up from 0.5% in August 2024. The best cloud models are still ahead, and lately the gap has widened. Our forecast rests on everyday jobs, not on local models winning at the top.

Free, private, offline and unlimited is hard to beat

Ask why a feature would move to the device and you get four plain answers. Each use costs nothing. It works with no signal. The data stays put. Nobody can cap how often you use it.

Apple’s developer material states the last point outright. In a session this year, it said its on-device model “has no request limits”, while its server-side model “offers a daily limit per user”. That is the trade in one line. A server costs its owner money every time you use it, so somebody has to meter it. A model on your device does not.

Privacy is the driver people talk about most, and the evidence is indirect. We found no published survey that asks people whether they prefer AI that runs on their device. What exists measures worry. In Deloitte’s 2025 survey of 3,524 US consumers, the share worried about data privacy and security rose from 60% to 70%. Pew found that 50% of US adults are more concerned than excited about AI in daily life, up from 37% in 2021.

Companies show the same pattern. In Cisco’s 2025 privacy study of 2,600 professionals, 64% worried about accidentally sharing sensitive information through AI tools. Worry is not the same as choice. It does mean that when an app can say “this never leaves your device”, it is answering a concern most users already have. For the cost side, see our post on cost per task.

The cloud keeps the hardest work, so most people will use both

“Most use will be local” does not mean the cloud loses. The same vendors shipping on-device models keep their best work on servers. Apple’s newest image model runs on its Private Cloud Compute servers, not on the phone. Microsoft’s Paint Cocreator needs an internet connection even on a Copilot+ PC.

Cloud AI is also getting cheaper fast. The 2025 AI Index found the cost of running a model at GPT-3.5’s level fell more than 280-fold between November 2022 and October 2024. Cheap cloud AI weakens the cost argument for local, and we should say so.

So the likely shape is a split. The many small, private, frequent jobs run on the device, and the few hard ones go to a server, often inside the same app. Counted by how often people use AI, local wins. Counted by raw computing power, the cloud probably stays ahead for years.

A forecast should say how it could fail. Ours is wrong if three things happen. First, if built-in models stay stuck in beta features that few apps call. Second, if cheaper devices keep shipping without the memory to run them. Third, if everyday expectations rise as fast as the cloud models do, so that “good enough” always sits just out of a phone’s reach.

You can check the forecast against your own day. Dictate a note, caption a video or translate a page with your connection switched off, and count what still works. That list is longer than it was two years ago. Then try a local model on purpose: our guide to open models covers what fits on which machine, and CSuite runs them next to cloud models in one app, so you can send each job to the place that suits it. We think the device will get most of them.