Token Economics Has Shrunk the AI Tech-to-Profit Cycle to One Day
Our three weeks' tuning hit next-day P&L. Token economics cuts tech-to-profit: Zhipu 8x in 6mo; OpenAI Codex 5M/wk (GPT-5.5); MiniMax beat Baidu, fell.
6 min read
Tags:
Infrastructure8 posts
The Hidden Thread in Token Business: Cost Is Set by KV Cache Hits, Not Throughput
Token cost is often judged by TTFT, TPOT, and throughput, but a 10x bill swing comes from KV cache hits, not speed. Model, server, and user layers must align.
5 min read
A Token Is Not a Thing
GPT-5.5/Opus 4.7 demand is near-infinite, mid-tier models vanish, low-end compute idles; tokens resemble electricity but act like a mismatched gas station.
7 min read
When a Model Is 5× Faster, It’s No Longer the Same Model
Gemini 3.5 Flash and Zhipu GLM-5.1's 400 token/s mode show that crossing ~5× inference speed unlocks a different product category, not just faster answers.
6 min read
The Most Expensive Waste in the Agent Era: GPUs Waiting on CPUs
700+ AI Infra experiments cost me 35 hours on startup. I blamed GPT-5.5 fast mode, but GPUs waited on CPUs; Intel moves CPU:GPU from 1:8 to 1:1.
6 min read
DeepSeek V4 Day: It's About Infra, Not the Model
DeepSeek V4 matches Opus 4.6, but FP4, 1M-token context, and day-0 chip support stress inference infra. GPT-5.5, Vision Banana, and LPM 1.0 landed too.
7 min read
Why We Built AIMA: An Open Source Project for Managing AI Inference with AI
We open-sourced AIMA, a Go binary with 57 MCP tools and a YAML knowledge base for heterogeneous AI inference, as AI servers halve in value in three months.
7 min read
Edge AI Inference: A Goldmine of Compute, or a Management Black Hole?
As AI shifts from chat to agents, compute demand surges 100×. The real edge problem isn't buying hardware but using it—TCO traps eat cheap devices' value.
4 min read