A 180-Degree Turn
By late 2025, the compute market was still worried about oversupply.
By early 2026, the wind had completely shifted. Compute became scarce, prices started rising. Kimi's API revenue during just a few days of the Spring Festival matched its entire previous year.
The reason isn't complicated: the way we use AI has changed. From chat to agents, compute consumption multiplied a hundredfold in just two or three months. Data shows that in agent scenarios, token consumption per user is 10 to 50 times that of ordinary chat, and in some cases up by 370x. This is why AI is not just the next computer, but an industrial revolution—it is rewriting the basic formulas of how economies operate.
Cloud compute is being utilized quite fully—tech giants are scrambling for GPUs, startups are lining up, and resources aren't going to waste.
But edge compute is a different story.
What Is the Real Cost of Edge AI Inference?
AI PCs, edge inference servers, various AI accelerators—these devices are evolving rapidly, and prices are falling fast. NVIDIA's DGX Spark costs just over 30,000 RMB, while AMD's Ryzen AI Max+ 395 is even under 20,000. The compute is substantial, capable of running reasonably large models.
But are they being used well?
From what I've seen, the answer is no.
The problem lies in Total Cost of Ownership (TCO). The hardware itself has gotten cheaper, but the proportion of other cost items has been violently magnified.
Operations and Maintenance Costs
A 200,000 RMB server breaks down twice a year. Each time, it costs 1,000 RMB to get a technician to fix it—2,000 RMB per year. That's 1% of the device price, negligible.
Swap in a 20,000 RMB edge device. The failure rate won't be lower—it might even be higher, because edge environments are more complex. Still 2,000 RMB in O&M costs.
But now that's 10% of the device price. From 1% to 10%—a tenfold magnification.
Reality is worse. A single on-site service call for an edge device can hit 500 RMB per unit, and with devices scattered around, inspection efficiency is low.
Cold-Start Costs
A device fresh from the factory is just a lump of iron.
How do you turn it into real AI compute? How do you extract its full performance?
It's harder than you'd think. Hardware changes by the day, inference engines come in all flavors, models evolve too fast, and heterogeneous architectures are a headache. ARM, x86, and RISC-V coexist; firmware is fragmented. One company with 2,000 edge devices saw troubleshooting time triple simply because firmware versions were inconsistent.
To use these devices well, you need someone who understands hardware, inference engines, models, and applications. A monthly salary of 20,000 RMB for such a person isn't unreasonable.
But the device itself only costs 20,000 RMB.
That's a 100% premium just to make one device work. That doesn't make business sense.
Ongoing Usage Costs
Getting the device running isn't the end—it's the beginning.
In 2026, leading LLM vendors are releasing updates at a dizzying pace. Every month or two, another heavyweight model drops. Google iterated Gemini from 2.5 to 3 Flash within a year, Alibaba densely launched the Qwen3 series, and DeepSeek and Kimi took turns in the spotlight.
If your edge device is still running a year-old model, its value might have dropped to 10%.
Keeping up means continuous upgrading, adapting, testing, and deploying. These investments are headcount-based; they don't get cheaper just because the hardware did.
Do Edge AI Devices Need Human Management?
The traditional software approach is: design a system for humans to operate.
This approach carries a hidden assumption: that humans are willing to participate.
But in the edge AI scenario, does that assumption hold?
Think about electricity. People use it, but they don't participate in its production or management. Electricity is infrastructure; people just consume it. No one feels the need to "participate in the production of electricity."
Edge AI devices should be the same.
They are infrastructure, not tools. The ideal state is devices that self-run, self-heal, and self-upgrade, never breaking down. Such devices should be as boring as routers—tossed in a corner, running 24/7, completely out of mind.
If humans can stay out of it, all the better.
This means a shift in design philosophy: let the devices manage themselves, keep humans outside the loop, and make software not a tool for people but a "nervous system" for the devices. The goal isn't to reduce human burden; it's to eliminate human participation.
How Do We Get There?
I see several trends we can leverage.
First, AI agents themselves are rapidly improving. If your product doesn't let agents participate in core workflows, it's out of step with the times. Agents can handle O&M diagnostics, fault repair, performance tuning, and model upgrades without human intervention—automating away the work of that "20,000-RMB-a-month person." This isn't future tense; agent capabilities in 2026 are already sufficient.
Second, open-source community collaboration. Edge AI faces a four-dimensional optimization problem: hardware devices × inference engines × models × applications. Every dimension is changing rapidly; no single company can keep up. But as agents become ubiquitous, collaboration around code repositories becomes easier. Products need to think about how to bring more people and agents in to jointly tackle the growing complexity.
Finally, internet infrastructure. Edge devices are inherently scattered—AI PCs and edge servers are distributed across offices, factories, stores, and homes, unlike data centers where thousands of machines sit in one room. To extract value from these scattered devices, you must use the internet. Remote monitoring, OTA upgrades, cloud coordination—these aren't optional.
Closing Thoughts
The TCO problem in edge AI is, at its core, a management philosophy problem.
Continuing to design products with a "human-centric" mindset will only magnify O&M costs, keep cold-start costs high, and let ongoing usage costs continue to erode value.
The solution is to let AI participate. Use agents to automate operations, use the community to distribute complexity, and use the internet to connect scattered devices.
This is what we're working on. We open-sourced AIMA—infrastructure for managing AI inference with AI, with the goal of driving TCO down to roughly hardware plus electricity. And the significance of edge AI isn't just about cost—it's also a question of power.