How one app talks to both cloud AI and local AI

The user picks a model, types a prompt, and hits Generate. Behind that one button, our desktop app might open an HTTPS connection to a GPU rented from Replicate, or push a task down a WebSocket to Runware, or call an HTTP server on localhost that it spawned itself, or make no connection at all and run an ONNX model inside its own process. Same button, same progress bar, same file in the project folder at the end. Four backends that agree on almost nothing.

This post is the engineering tour of how one app, our own CSuite, treats cloud APIs and local runtimes as the same thing behind one interface. It is told through the five problems that actually made it hard: describing the models, routing the request, filtering the parameters, streaming the result, and keeping the trust boundaries intact. The pattern matters beyond this product. Every AI app that outgrows a single provider becomes a router, and the design question is what keeps the router from collapsing into a pyramid of special cases.

A “model” means four different machines

Start with how little the four backends share. Replicate is a classic cloud API: you create a prediction over HTTPS, it moves through statuses (starting, processing, succeeded, failed), you poll or stream, and a run can take up to 30 minutes. Runware prefers a persistent WebSocket: you authenticate once per connection, fire tasks tagged with UUIDs, and match responses back by tag as they arrive out of order.

The local half is stranger. Ollama is an HTTP server that happens to live on your own machine; CSuite bundles its own copy and runs it on port 11435, one off from Ollama’s default 11434, so it never fights an install the user already has. transformers.js removes the server entirely: it runs ONNX-exported models through ONNX Runtime inside a child process of the app. No socket, no port, no network.

The asymmetries go all the way down. Cloud models need an API key; local ones need gigabytes of disk and enough RAM. Cloud failures look like rate limits and timeouts; local failures look like out-of-memory kills. A cloud model is available the moment you know its name; a local model has to be downloaded, version-pinned, and health-checked before the first token. An abstraction that pretends these differences don’t exist will leak them at the worst moments. The goal is to contain them in known places instead.

One catalog beats four model lists

The tempting first design is one model list per provider: the Replicate models, the Runware models, the Ollama models. It works for a demo and then rots. The same model appears twice with different names, every picker needs merge logic, and every new provider multiplies the surfaces to update.

CSuite inverted it: one catalog file, organized by modality, where each model is a single entry that declares which platforms can run it. As of August 2026 that file describes 107 models (62 text, 17 image, 13 audio, 15 video) through 151 platform blocks: 49 on Replicate, 49 on Runware, 47 on Ollama, 6 on the embedded HuggingFace runtime. A model offered by two clouds is one entry with two blocks, not two rows. A small model that runs on both local runtimes is also one entry: the catalog id is the Ollama tag, and the block for the HF runtime points at the ONNX export of the same weights.

What a block carries depends on what its platform needs to know. Cloud blocks hold the provider’s routing slug, the parameters the model accepts, and a pricing basis (per token, per image, per second of video). Local blocks hold download size, a minimum-RAM floor for gating, and capability flags like vision input or native tool calling. Everything downstream reads this one file: the model picker, the settings panel, the cost estimates, and the dispatch itself.

That is the payoff: routing stops being a decision tree and becomes a lookup. Local runtimes get fixed sentinel ids; cloud providers are stored credentials with UUIDs. The request handler checks for a sentinel, otherwise resolves the credential and switches on its kind. Four executors per modality, each owning one backend’s weirdness. The router itself stays boring, because every model-specific decision was already made by the catalog before the request reached it.

The catalog, not the UI, is the parameter contract

Models disagree about parameters even more than platforms disagree about transports. Across CSuite’s catalog, the model entries declare 559 input parameters between them, and the settings panel is rendered from those declarations: a model with no temperature parameter simply shows no temperature slider. The same schema does the reverse job at the boundary. Before a request leaves the app, everything the target model didn’t declare is dropped, so the UI cannot send what the model cannot accept.

Declaring the parameter is half the contract; placing it is the other half, and that lesson cost us a production break. Runware’s text API nests sampling options inside a settings object and rejects the whole request when one arrives at the top level. Our app once sent maxTokens top-level, and every Runware text generation for models declaring that parameter failed until the mapping layer learned to fold catalog names into Runware’s envelope. Text-to-speech was stricter still: one model’s speed belongs inside a speech object, its turbo flag inside settings, and a misplaced parameter that carried a default value failed every single generation, not just the ones where a user touched the control.

Two lessons survived. First, strict APIs are a gift: an error that names the offending key is the best failure mode an integration can have, far better than a lenient API silently ignoring what you send. Second, capability is per model, not per platform. One flagship model rejects the temperature parameter its siblings accept; reasoning-effort enums differ between models of the same family. The catalog carries a per-model enum for each of those rather than assuming a family resemblance, because the API told us otherwise.

Stream the whole string, not the diff

Streaming text is where the transports diverge most. Replicate streams through its SDK. Runware’s socket returns whole task results. Ollama emits newline-delimited JSON objects, each carrying a small delta of the reply. The app’s interface process needs one contract for all of them, and CSuite’s is deliberately redundant: every chunk pushed to the UI carries the full string so far, and the renderer replaces what it has rather than appending.

Deltas are the textbook answer because they minimize bytes. But at chat scale the bytes are trivial: even resending the whole string on every chunk, a long reply moves a few megabytes between two processes on the same machine. Deltas buy their efficiency with fragility: they assume nothing is ever dropped, duplicated, or reordered between producer and screen. Cumulative chunks are self-healing. A lost message costs one skipped repaint, and the next message repairs the screen completely. Runware’s own protocol gestures at the same reality from the other side: its API buffers undelivered results for 120 seconds so a client that reconnects can collect what it missed. When the payload is small, design for recovery, not for byte savings.

Cancel means four different things

The Cancel button is one control in the interface and four different operations underneath. CSuite keeps a single abort registry: the interface mints a request id for every generation, and cancel aborts by id. What that abort does depends entirely on the backend it reaches.

The last row is the point of the table. A unified interface makes it tempting to promise unified behavior, and sometimes the truthful answer is that one backend cannot honor the promise yet. Shipping the button that works on three backends, and being honest about the fourth, beats either hiding the button everywhere or faking a cancel that leaves a model spinning CPU in the background.

API keys never meet the interface

A desktop app has a trust boundary that web apps don’t make you think about. In Electron’s process model, the main process has full system access while the sandboxed renderer, the part that draws the interface and runs the most third-party code, does not. CSuite treats that line as the security architecture: every provider call, credential read, and file write happens on the privileged side, and the interface asks for work over a validated message channel instead of doing it.

Keys are the clearest example. They are stored only on the privileged side, encrypted at rest with Electron’s safeStorage, which delegates to the OS keyring: Keychain on macOS, DPAPI on Windows, Secret Service on Linux. The interface never receives a key at all; it renders a last-four-characters hint. Decryption is deferred and per-provider, so the macOS Keychain prompt fires when a stored provider is first used, not as a wall of prompts at launch. The same discipline applies to disk: every path the interface asks to write is resolved and checked against the project folder, so a compromised or simply buggy renderer cannot name a file outside it. If you bring your own keys to AI tools, this is the shape to look for; we’ve written before about why BYOK changes the deal.

What the abstraction costs, honestly

None of this is free, and the costs deserve the same candor as the design. The catalog is a maintenance commitment: every new model means researching its real parameter surface, and published schemas are not the whole contract, so we probe live APIs before trusting them. The lowest-common-denominator temptation is constant; the honest fix is per-model capability flags rather than flattening every backend to the features all four share. And the local half carries weight the cloud half never does: a pinned runtime downloaded at first setup, version markers that force clean re-downloads, and models measured in gigabytes where a cloud model is just a name in a request.

Some asymmetries refuse to hide, and the right move is to stop hiding them. Local text generation can’t be cancelled. One cloud platform can drive tool-calling chat and the other can’t, so the chat picker only offers models that can actually do the job. An abstraction earns trust by being honest about its edges, not by sanding them off.

If you’re building the same shape, the checklist is short. Describe models as data, and make every surface read the same file. Route by lookup: sentinels for local runtimes, credentials for cloud. Filter parameters at the boundary so the UI can’t lie to a model. Stream cumulative state when payloads are small, and keep one abort registry even though every backend cancels differently. Keep secrets in the privileged process, always. The reward for the discipline is that the desktop app thesis actually works: the same button serves a frontier cloud model today and a small local model tomorrow, and the user never has to care which machine answered.