The scaling wall is real. The fix might be in your pocket.

The dominant story in AI over the last five years has been scaling: more data and more compute produce better models, reliably and predictably. A position paper from Zhejiang University argues that both inputs to that strategy are running out, and that the way through is not a new training algorithm. It is a redistribution argument. The data and the compute already exist. They are just sitting in people's pockets.

Two walls. Both real.

Scaling laws have held well enough that "just train a bigger model on more data" has functioned as a credible research strategy. This paper argues that both inputs to that strategy are running out.

Wall one: data exhaustion. Current estimates place the total stock of high-quality human-generated text at roughly 4 × 10¹⁴ tokens. LLaMA 3.1's smallest model was trained on 15 trillion tokens. Projections suggest dataset sizes need to grow at around 2.4× per year to keep up with scaling demands. The math does not work past 2028, and possibly as early as 2026 if overtraining practices continue. Synthetic data is the obvious fallback, but it carries known pathologies: model collapse when models train iteratively on their own outputs, unverifiable quality in open-domain contexts, and linguistic homogenization that underrepresents cultural and low-frequency expression.

Wall two: compute monopolization. Training GPT-3 required 10,000 V100 GPUs. GPT-4 reportedly required over 25,000 A100s at an estimated cost north of $100M. Grok 3 pushed further still. This infrastructure is now concentrated in a handful of organizations, effectively locking academic institutions and smaller companies out of foundational AI research. Moore's Law is slowing, semiconductor fab capacity at 5nm and below is booked through 2026, and the compute growth rate required for continued scaling (12.8× per year since 2022) is outpacing the physical supply chain's ability to keep up.

The proposed solution is not a new training algorithm or a better synthetic data pipeline. It is a redistribution argument: the data and compute needed to break through both walls already exist, in the form of edge devices (smartphones, IoT sensors, laptops, wearables). They are just fragmented, private, and untapped.

What edge devices actually have.

The paper's most striking contribution is not a new technique. It is a quantification: how much data is being generated at the edge, and how much compute. The authors run these numbers seriously and the results are larger than most people would expect.

To put the compute number in context: training DeepSeek-v3 used 2,048 H100 GPUs, each delivering about 59.3 TFLOPS FP32. A flagship smartphone chip in 2024 (iPhone 16-class) delivers roughly 2 TFLOPS. If you could run them in parallel (a big "if"), you would need around 60,723 phones to match DeepSeek's compute configuration.

# DeepSeek-v3 training compute
2,048 GPUs × 59.3 TFLOPS = 121,446 TFLOPS

# iPhone 16-class chip
per_phone_tflops = ~2     # FP32

# Phones needed (ideal parallel)
phones = 121_446 / 2 ≈ 60,723 devices

# Context
global_shipments_2024     = ~1.24B units
deepseek_equiv_setups_yr  = ~20,000

The "ideally in parallel" qualifier is doing a lot of work in that number. Distributed training across heterogeneous devices with variable connectivity, hardware, and availability is an unsolved engineering problem. The paper is clear about this. But the raw resource argument is the point: the gap between what exists at the edge and what is being used is enormous.

Three layers of distributed AI.

The paper reviews the technical landscape across three increasingly ambitious deployment modes. Each is a real area of active research; the paper's contribution is framing them as a coherent stack rather than independent lines of work.

1. Small Language Models at the Edge (SLMs)

   The immediate, practical tier. Deploy compressed models directly onto devices for inference without cloud dependency. Architectures like Mamba (linear complexity, faster inference than Transformers), Hymba (combined attention plus state-space heads), and xLSTM (modernized LSTM with exponential gates) are purpose-built for memory-constrained environments. A 770M parameter model using distillation, quantization, and domain specialization has been shown to reach 95% of a 540B model's performance on specific tasks using under 0.15% of the compute. The iPhone 16 series can run real-time image enhancement and multilingual translation locally at 2 TFLOPS.

   status · deployed today
2. Collaborative Inference

   Distribute inference across multiple devices rather than a single model on a single device. When one device cannot run a full model, a network of devices can collectively handle it: splitting layers, passing activations, aggregating results. This trades latency for accessibility and enables larger models to run without centralized server infrastructure. The key engineering challenge is network topology and latency management when devices are heterogeneous and sporadically available.

   status · emerging
3. Federated Training

   The most ambitious tier, and the most technically unsolved. Each device trains on its local data and shares only model updates (gradients or weight deltas) with a central aggregator, not the raw data. This enables compliance with privacy regulations like GDPR while still using edge data for training. FedLLM and related frameworks have demonstrated federated fine-tuning at meaningful scale. On-device training is now feasible even on embedded systems: recent work achieves up to 95% sparsity for training operations, and the ElasticZO approach enables integer-arithmetic-only training with ~1.5× memory reduction. The paper's position: federated pre-training of large models across massive edge populations remains an open problem, but the technical trajectory makes it a realistic medium-term target.

   status · still an open problem

The two unsolved problems that matter most.

The paper identifies two fundamental technical challenges as critical blockers for the distributed edge training vision. They are not engineering difficulties to be incrementally engineered away. They are research problems that require conceptual breakthroughs.

01 · Heterogeneous Device Model Fusion

Edge devices differ wildly in memory, compute, architecture, and connectivity. A training protocol that works on an iPhone 16 may be incompatible with a mid-range Android or an IoT sensor. Federated learning typically assumes a shared model architecture; heterogeneous device populations violate this assumption. How do you aggregate model updates from devices running meaningfully different model variants? How do you prevent the most capable devices from dominating the update signal? This is unsolved at the scale the paper envisions.

02 · Heterogeneous Device Compute Sharing

Even if all devices could train the same model, coordinating intermittent, variable-bandwidth participation across millions of devices is a systems problem without a clean solution. Devices go offline, throttle under thermal pressure, vary in battery state, and have inconsistent network conditions. Training stability in the presence of stragglers and dropouts (a known problem in distributed deep learning even with reliable hardware) becomes dramatically harder when the participants are personal devices with no uptime guarantees and competing workloads.

These are worth naming clearly because they are the gap between the paper's inspiring quantifications and the actual deployed system. The data exists. The compute exists. The protocols to harness it reliably at scale do not yet. That is an honest framing, and the paper earns credibility by not overselling the readiness.

This is really about who gets to do AI.

The technical arguments in this paper are interesting. The political argument underneath them is more important. The current scaling paradigm concentrates AI development capability into roughly five organizations globally. Everyone else (every university, every startup, every national research institution without access to four-figure GPU clusters) is a consumer of models those organizations choose to release, on the terms those organizations choose to set.

The distributed edge argument is, at its core, an argument about participation. If the marginal data and compute needed for frontier-scale training can be contributed by ordinary devices running federated protocols, then the frontier is no longer gated by who can afford the cluster. The authors frame this explicitly as AI democratization, a term that often functions as marketing but here has a specific technical meaning: the training inputs become structurally decentralized.

There are obvious counterarguments. Coordination across millions of heterogeneous devices produces noise, introduces new attack surfaces (poisoning, free-riding), and requires incentive mechanisms that do not exist yet. The paper acknowledges all of this. The environmental argument cuts both ways: distributed edge training at massive scale has its own energy footprint, even if the per-device consumption is low.

For teams building production agent stacks today, the near-term relevance is in the SLM layer. The trajectory of on-device models is moving faster than most planning horizons account for. A 2 TFLOPS phone chip that can run 7B-parameter inference locally, with no API latency and no data leaving the device, changes the architecture of a meaningful slice of applications. That is not a prediction. It is already happening on the high end of the consumer hardware market, and the paper's compute growth projections suggest it will normalize within the current planning horizon.

If you would like a second read on how an SLM-first architecture would change your current agent stack, the contact form is the fastest way in. We do 30-minute reviews for production agent stacks, free.