How to Ship In-Browser AI Apps With No Server and No Inference Bill

The cheapest AI app to run is one where the user’s own machine does the work. Your server hands out files, the browser downloads a small model once, and every inference after that happens on the visitor’s GPU or CPU. You pay for static hosting. That’s it.

This used to be a demo trick. In 2026 it’s a reasonable way to ship a real tool. WebGPU now runs in Chrome, Edge, Safari 26 and up (iPhone and iPad included) and Firefox on Windows and Apple Silicon Macs. Transformers.js v4 runs vision, speech, embedding and classification models on WebGPU and falls back to WebAssembly when there’s no GPU path. The pieces are boring enough to build on.

What follows is a setup that works for small, single-purpose tools: a face blurrer, a PDF table extractor, a private transcriber, a screenshot secret scanner. It is deliberately strict. Every rule exists because breaking it either costs money or breaks the privacy promise.

The ground rules

Ten rules, and a tool that needs to break one should be designed differently.

1. The model runs on the user’s device. No server-side inference in the free product.
2. Nothing the user loads leaves the device. No uploads, no logging of inputs.
3. Zero running cost. Static hosting and free object storage only, no Worker, no database, no API keys in the product.
4. The output is a file or a table. A cleaned image, a CSV, a redacted PDF. Chat is where the big hosted models win, so stay out of it.
5. One job per tool, using the smallest model that does it well enough.
6. No account. Open the page, drop the file, get the result.
7. It works offline after the first visit.
8. It falls back instead of failing: WebGPU first, WebAssembly second, a plain message third.
9. Open source code, and every model’s licence checked for commercial use and redistribution.
10. The test suite proves the privacy claim, not just the features.

The stack and where the files live

Transformers.js is the default engine. It sits on ONNX Runtime Web, caches model files in the browser on its own, and the same code runs in Node, which matters for testing later. For the rare tool that truly needs to write text, WebLLM covers chat-style models, but it’s WebGPU only and even a 1B model is a download of roughly 700 MB. MediaPipe Tasks handles fast face, hand and pose detection with tiny models. Chrome’s built-in summarizer and translator APIs are a nice extra on desktop Chrome, never something to depend on, since Safari and Firefox users get nothing from them.

Everything else is plain web platform: Canvas and OffscreenCanvas for crops and blurs, pdf.js and pdf-lib for documents, a Vite build that outputs static files. The model always runs in a Web Worker so the page never freezes.

Picking the device takes a few lines:

import { pipeline } from "@huggingface/transformers";

const adapter = navigator.gpu ? await navigator.gpu.requestAdapter() : null;
const device = adapter ? "webgpu" : "wasm";

const detector = await pipeline("object-detection", "Xenova/detr-resnet-50", { device });

Model weights are just files, and where they live depends only on size.

A few habits keep this sane. Put the model name, version and a short hash in every file name, so a published file never changes and can be cached forever. Check each download against its SHA-256. Keep weights out of git and let a script in the repo fetch, quantise and upload them. Lock the bucket to your own domains with CORS so nobody hotlinks your weights. And give every tool page a button that clears the downloaded models.

On budgets, a page should open in under a second, the first result from a small model should land in under ten seconds on broadband including the download, and repeat visits should answer in under two seconds from cache. Warn the user before any model over 50 MB starts downloading, and show progress in megabytes. For phones, plan on staying under about 300 MB of model.

One gotcha for offline mode: cache the app shell with a service worker, but keep inference in a dedicated worker. Firefox doesn’t expose WebGPU inside service workers.

Making the privacy promise checkable

“Your files never leave your device” is easy to write and easy to fake. The setup should make it hard to break by accident and easy for a skeptic to verify.

The strongest piece is a Content-Security-Policy that tells the browser it may only talk to your own domains. On Cloudflare Pages that’s one rule in the _headers file:

/*
  Content-Security-Policy: default-src 'self'; connect-src 'self' https://models.example.com; script-src 'self' 'wasm-unsafe-eval'; worker-src 'self' blob:

With that in place, even a buggy dependency can’t post a file to some other host. Bundle every library and serve it from your own domain instead of loading it from a public CDN at run time. Build results on the device and save them with an ordinary download. Skip analytics on tool pages. Then write a short “how to verify” page: open the network tab and watch nothing go out, or turn off Wi-Fi once the model loads and see that the tool still works.

Testing closes the loop. Run the accuracy tests in Node in CI, using the same Transformers.js code the browser runs, and report the miss rate on a labelled test set as the headline number. Build that set from your own files, public-domain material and synthetic documents, never from real people’s private data. Run the page in headless Chromium against a sample file, and add a privacy test that fails the build if any request leaves the allowed hosts while a file is being processed. Headless CI usually has no GPU, so it checks the WebAssembly path, and the WebGPU path gets checked on real devices before each release.

What it costs and when it fits

The one number worth watching is R2 reads. Each first-time model download counts as one read or a few, and the browser caches it after that, so ten million reads a month is millions of new visitors. Cloudflare asks for a payment method to switch on R2, so set a small billing alert the day you create the bucket.

Not every idea suits this model. Before building, run it through a quick test:

• The input is something people won’t or can’t upload: contracts, bank statements, evidence photos, a child’s voice, unreleased code
• A hosted chatbot can’t do the job, or can’t without that upload
• It’s real processing, not glue between existing services
• The output is a file, a table or a marked-up result
• A model under about 500 MB does it, ideally under 100 MB
• The core function runs on WebAssembly on a mid-range laptop
• The model licence allows commercial use and redistribution
• You can build a test set from your own or openly licensed material
• It’s useful as a free tool with nothing to upsell

If an idea clears all nine, the hard part is the tool itself, not the infrastructure. The infrastructure is a static site and a bucket.

• AI
• architecture
• offline-first
• Cloudflare
• JavaScript