It is a common question about AI: should we run it on our own machine, so our data never leaves the office? The usual way to answer it is a spreadsheet, with a graphics card priced online on one side and a monthly AI bill on the other. In September we ran that costing, with the caveats below, to decide whether we should buy one ourselves, and the spreadsheet turns out to answer the wrong question. At working-hours use, owning loses on cost at every rental tier we priced, by a wide margin. The only argument for the box is compliance, and that changes what you are buying and what you need to check before you buy it.
The economics are not close.
The build we priced is the smallest one worth owning: one used graphics card with 24 GB of memory, a current desktop processor, 32 GB of system memory and 1 TB of storage. (Why 24 GB is the floor comes later.) Against it we priced renting a 24 GB card by the hour (a newer RTX 4090, so the rental is if anything the faster machine) at three tiers, from a peer-to-peer marketplace where private individuals rent out their own machines up to a secure datacenter tier. Divide the cost of the box by each hourly rate and you get the number of rented hours the box has to replace before owning comes out ahead.[1]
The dearest tier still needs about 4,700 hours, roughly 13 hours a day for a full year, before the box pays for itself. A small business does not run an AI model around the clock. It uses one in working hours, in bursts. At 40 hours a week, 4,700 hours is more than two years, and that is compute alone: the box also draws power and needs someone to update it and fix it when a driver breaks. Neither is in the bars, and both favour renting. One omission cuts the other way: the bars charge the full price of the box, as if it were worth nothing afterward, and a used card can be sold on.
The rates are second-hand, so test the conclusion against them instead of trusting them. Take only the dearest tier and halve the price of the box. That is the same arithmetic as doubling the hourly rate (in case the advertised floors understate what a rental really costs) or selling the box on later for half what you paid. The dearest tier then needs about 2,350 hours, still more than a year of 40-hour weeks, before power or upkeep. The conclusion does not depend on any one number being right.
Why owning costs more than expected.
Our research session first wrote down a price for the same parts list from an AI model's built-in knowledge, with training data running to about May 2026, and only then searched current listings.[1] The listed prices came back far higher. That is not a dated price series: it measures the September 2026 market against what a model trained to about May 2026 expected. The difference is the finding, so the chart keeps the first estimate as its baseline.
Three shortages hit at once: memory chips, flash storage and graphics cards, all for the same reason. AI datacenters buy from the same supply that consumer parts are made from, and they outbid consumers for it. System memory is the extreme case, at about five and a half times the first estimate, and the one trade outlet behind these figures, which we could not corroborate, forecast high prices into 2028.[1]
One consequence: the datacenter buying that makes rented and hosted AI cheap is the same buying that makes owning expensive. The two sides of the decision are not independent. One supply constraint pushes both of them, in the same direction, against the box.
One rule falls out for anyone who does buy: trim the system memory and the storage, not the graphics card. Running a model is limited by the card. Dropping to 32 GB of system memory and 1 TB of storage costs nothing for a model that fits on the card (the exception is a large mixture-of-experts model run partly from system memory) and took about a fifth off the build.[1] That is right-sizing, not waiting for a better price, since, if that forecast holds, the shortage outlasts any sensible wait.
The only reason to own.
If cost argues against owning at every tier we priced, the decision to own is not an economic one. The one thing the box buys that a rental cannot is control over where your data is processed. For a Quebec business, that is a Law 25 question.
In general terms (this is the posture we work from, not legal advice): Quebec's private-sector privacy law, as amended by Law 25, requires a privacy impact assessment before personal information is communicated outside Quebec, and a written agreement that reflects it.[2] It conditions the transfer; it does not forbid it. The line it draws is the Quebec border, not the Canadian one. A datacenter in Toronto is outside Quebec too. Confidential client material can also be bound by the confidentiality terms you signed with that client, whatever the privacy law says.
A local AI box is not an investment that pays back at volume. It is an insurance premium against a cross-border transfer, and the costing only tells you the size of the premium. Whether it is worth paying depends on what goes into the prompts, not on how many hours you would use the machine.
It also changes the order of the questions. Whether a local model is good enough for your work is something you can find out on rented hardware, with public or made-up prompts, for under one percent of what the box costs.[1] If the answer is no, the whole question closes: no purchase, no residency assessment.
Storage residency is not processing residency.
Between owning a box and sending everything to a US cloud sits the obvious middle option: a provider that offers Canadian hosting. It can be a real substitute for owning. Whether it is depends on a distinction that a "hosted in Canada" claim does not settle.
Storage residency is where your files are kept; processing residency is where the model runs when it reads your prompt. A provider can store your data in Canada and still run the AI model on hardware in another country.[1] Your files, the record, never leave. But every time you ask the model something, the request goes to wherever the model runs. With an AI model the request is the sensitive part: it carries the client's name, the clause you pasted, the figures from the file. The prompt carries the personal information, and it crosses the border on every call.
So "your data is hosted in Canada" answers the wrong question. The control that matters is processing residency, and it belongs in writing. The questions we would put to any provider before client material goes near it:
| ask | why it matters |
|---|---|
| where inference runs | Region by region, for the model you will actually use. Storage and processing are often in different places. |
| whether it can move | Can requests be sent to another region when the usual one is busy? That is a cross-border call you never see. |
| what gets logged | Are prompts and answers logged, and where are the logs kept? A log of prompts holds personal information too. |
| who else touches it | Is any of the processing subcontracted? A subprocessor in another country is the same transfer. |
| the contract | Will the provider commit to processing residency in the agreement? A sales page is not a commitment. |
This is also where the cost comparison comes back in. Our note did not price the Canadian-residency options, but it expected an owned box to be competitive again against them, on cost as well as control.[1] Once compliance is the reason, that is the comparison to run, not the one against the cheapest US rental.
There is a cruder trap on the cheapest tier. Marketplace floor rates come from unverified hosts, private individuals renting out their own machines. Assume the host can read the disk.[1] That tier is for public and made-up prompts only, a stricter rule than "not in the US": it applies even to material that only sits near client work. Our note treated only the secure datacenter tier as a defensible middle ground, so give the community tier the same caution until a provider's terms say otherwise.
Capacity, not speed.
If the work does justify the box, the most common mistake is the advice to "buy a better graphics card." Speed is not what decides whether a local model works. The memory on the card (VRAM) is.
A model needs roughly its parameter count times the bits stored per weight, divided by eight, in gigabytes, plus 15 to 25% for working memory and overhead. At the common 4-bit level (Q4_K_M, roughly 4.5 bits per weight in practice), that is about 0.56 GB per billion parameters before the overhead.[1] The ladder:
The property that matters is that this is a cliff, not a slope. A model that fits entirely in VRAM runs at roughly 15 to 60 tokens per second (a token is a fragment of a word). A model that spills into system memory drops to roughly 1 to 4, because every token now waits on the part of the model held in much slower system memory.[1] There is no usable middle. So the sizing question is always "what is the largest model we need to fit entirely," never "how fast is the card." A faster card one gigabyte short is the slower machine.
In our costing's judgement, 24 GB is where quality per dollar flattens. Below it, the output rarely justifies the effort against a hosted service. Above it, the cost climbs steeply while capability improves in small steps.[1]
The order to decide in.
- Test quality on rented hardware first.
Rent a GPU instance for an afternoon, run three model sizes (roughly 8B, 14B and 30B) against the same 15 to 20 real tasks rewritten with public or made-up data, grade the output blind where you can, and note each model's speed. If your work is in French, write the tasks in French: that is part of what you are testing. It costs under one percent of the box. If no model you could fit is good enough, stop there: you have saved the box and the assessment. Our six-line eval suite is a starting shape for the grading.
→ a yes or no on quality, before any purchase - If the prompts would carry personal information or client confidences, settle residency next.
List what would actually go into the prompts. If any of it is personal information or confidential client material, the hosting choice is a Law 25 question before it is a technical one. Get processing residency, not storage residency, in writing from every provider you consider, and check whether processing stays in Quebec or only in Canada. Some large clouds run a region inside Quebec; ask for one by name.
- Only then price the box, as a compliance cost.
Size it by the largest model that must fit entirely in VRAM, trim system memory and storage rather than the card, and price it from primary sources on the day you buy: the figures behind this note were second-hand when we gathered them, and perishable besides. Put power, updates and someone's time on the same line. Compare it with the residency-compliant rental, not with the cheapest one.
What this does not claim.
Second-hand prices. Every price reached our costing through search summaries: asking prices for used cards, advertised floor rates for rentals, one uncorroborated trade outlet for the shortage figures. That is why no dollar amount appears here, and why the argument was stress-tested by halving the box and keeping only the dearest tier.
One build, one usage pattern. The costing covers a single 24 GB card for small-business use in working hours. A team running a model around the clock reaches the dearest tier's break-even in under seven months, and that case, like a multi-card server, deserves its own costing. So does a business that already owns a desktop able to take the card. Our note rated buying only the card as by far the strongest move in this market, and the card alone is a little under half the build, which on our arithmetic brings the dearest tier's break-even close to one year of 40-hour weeks.[1]
Canadian-residency rentals not priced. The three tiers we priced are US infrastructure. Our note expected owning to be competitive against residency-compliant rentals, but put no number on them, and neither do we.
Compute only. The break-even leaves out power and the time it takes to keep a machine running, which favour renting, and resale value, which favours owning. Halving the box in the stress test is the same as selling it on for half its price, and the dearest tier still needs more than a year of 40-hour weeks.
Not legal advice. Law 25 is described here in general terms. Whether a given provider is acceptable for given data is an assessment specific to that data and that provider, and a question for your own counsel.
Not a vendor review. The rental providers named are the tiers we priced, not recommendations. Providers offering Canadian residency exist; we have evaluated none of them.
We do not run local AI hardware. The costing was a decision exercise. No JPL or client hardware, and no client engagement, is described.
Weighing whether to run AI in-house? Describe the work and what would go into the prompts through the contact form and you get a free written first read on whether it is a cost question or a residency question, and what to test first. Decisions like this fall under our automation audit.