Skip to main content
guanjiawei.ai
Blog
I Paid for a Top-Tier Model. It Got Dumber and Marked Me
AILLM私有化部署Open SourceReflections

I Paid for a Top-Tier Model. It Got Dumber and Marked Me

Codex GPT-5.5 clusters reasoning at 516; Claude Code secretly marks Chinese users. When cloud AI silently degrades, open-weight self-hosting is the hedge.

Jiawei GuanJiawei Guan14 min read
Share:

That night I handed Codex a task and went to sleep.

It’s a habit I’ve settled into over the past few months. A big refactor: break it down during the day, write a clear AGENTS.md, let it run overnight, then collect the results in the morning. I’ve run this loop enough to have a gut sense of how long things take, where it gets stuck, what works. I was comfortable leaving it unattended.

The next morning, the progress bar was stuck at a surprisingly shallow point.

I first suspected myself. Was the AGENTS.md unclear? Was the doc structure messy? Was some tool misconfigured? So I started wrestling with the harness—tuning prompts, restructuring directories, breaking tasks into smaller pieces. For two full days I tinkered, but the feeling of spinning in place never changed. It kept circling around inexplicable spots, then confidently told me it was done halfway through.

On the third day I finally complained in the team chat. Before I finished, several people jumped in at once: “Codex’s model has been dumbed down recently. Lately it’s gotten really bad—nothing like when it launched.”

I didn’t know whether to laugh or cry. I’d spent three days on the harness; the problem wasn’t on my side at all.

A Fixed Number: 516

I didn’t want to settle for “everyone feels it’s gotten dumber.” Vibes can’t be falsified—today’s bad mood can always be blamed on model degradation. I wanted hard evidence.

While browsing the community, I saw that someone had already dug to the bottom of it.

One user went full forensic. Instead of complaining, they pulled every session log under their local ~/.codex directory—from February 1 to June 27, 390,195 response records across 865 sessions, all carrying per-turn token_count metadata. They ran the numbers and found something very off.

There were 3,363 times when the model’s reasoning-token count was exactly 516.

Not one more, not one less. Exactly 516.

The number alone means little; the distribution is the scary part. GPT-5.5 accounted for only 19.3% of all responses, yet contributed 82.0% of all exactly-516 events. Look closer: 44% of GPT-5.5 responses had reasoning tokens equal to or stuck at 516, versus just 1.3% for other models. The monthly trend was worse: the exactly-516 ratio was only 0.11% in May, then spiked to 53.30% in June. Meanwhile, GPT-5.5’s average reasoning-token count fell from 268 in February to 107 in May.

If you plot the token counts of all responses as a histogram, you’ll see a weird comb: 516, 1034, 1552, each exactly 518 apart. That isn’t what a normal model looks like. A model facing problems of varying difficulty should show a continuous spread of thinking length—short for easy questions, long for hard ones. That comb means something is slicing thoughts off at a fixed boundary.

It Didn’t Get Dumber—It Wasn’t Allowed to Finish Thinking

This matched my three-day gut feeling.

Before, Codex would sometimes think for a long time, a bit sluggish; you could feel it holding back. Lately, its feedback had become abnormally fast and eerily consistent—every time, the rhythm of giving an answer after about thirty seconds of thought. I thought they’d optimized inference speed. In retrospect, it wasn’t fast; it was being cut off before finishing.

That 516 comb is the cutoff point.

The prevailing community guess points to the system prompt for GPT-5.5 inside Codex. Near the end sits a section called ## Intermediary updates, asking the model to post progress updates while working. Some speculate this instruction confuses “progress update” with “final answer”: before reasoning ends, it eagerly switches to outputting the progress line for you, so the whole reasoning chain wraps up early at the 516 buffer boundary. Someone even broke 516 down for you: a 512-byte initial buffer plus a 4-byte header makes 516, then plus 518 each time.

I’m not claiming this technical explanation is necessarily correct. But one fix derived from it, I tried personally, and it really works.

One Sentence Brings Back Half the Intelligence

The fix is absurdly simple. Add one line to your AGENTS.md:

Spend time on thinking; you do not need to use
the commentary channel to report progress to me.

Translation: take time to think; you don’t need to use the commentary channel to report progress to me. Stop burning cycles on “I’m reading the file” or “I’m about to start editing” messages, and use that effort to actually finish reasoning.

I added this and ran it for a few days. The projects that had been deadlocked started making real progress. Not a vague feeling of improvement, but the concrete certainty that things previously stuck could now move forward. Some in the community say this line has limited effect; a more thorough fix is to delete the entire ## Intermediary updates section from the system prompt, which reportedly eliminates the 516 behavior.

But do you sense the absurdity here?

I’m paying for GPT-5.5 xhigh—the highest reasoning-budget tier. I expected a top-tier model that thinks deeply about hard problems. Instead I have to coax it like a kid, writing a sentence in the config file—“please think carefully and don’t rush to report”—just to recover some of the intelligence it should already have.

This isn’t about money. Same cost, same model name; what I want is stable, predictable capability. Now a global update I can’t see can silently shrink it without telling me.

The official response was the most ironic part. The first issue reporting the 516 phenomenon was closed as not_planned after 23 minutes. The main issue with full statistics had over a hundred comments; not one OpenAI employee replied. Only after it hit the Hacker News front page did anything happen.

The Black Box Can Hide Far More Than Dumber Output

If the Codex incident can still be written off as a botched optimization, the Claude Code discovery the same week can’t be dismissed as an innocent mistake.

Someone reverse-engineered the Claude Code installer—an Anthropic-signed, minified JS bundle—and found code specifically for invisibly marking certain users.

The trigger conditions are sneaky. The logic only activates when you set ANTHROPIC_BASE_URL and point requests to a non-official third-party relay. Official direct users are skipped entirely. It targets people using proxies and relays—which is basically how Claude Code is used in China.

Once activated, it performs three checks. First, your system timezone is Asia/Shanghai or Asia/Urumqi. Second, whether your proxy domain matches a hidden list of about 147 entries, including .cn domains, baidu.com, alibaba-inc.com, bytedance.net, and a pile of Claude relay addresses. Finally, whether the domain contains AI lab keywords—deepseek, moonshot, minimax, zhipu are on the list.

If detected, how does it mark you? This is the shadiest part to me.

It modifies the system context sent to Anthropic’s servers—that innocuous line, “Today’s date is 2026-06-30.” If your timezone hits China, it replaces all date separators - with /, turning 2026-06-30 into 2026/06/30. If your proxy domain matches the list, it replaces the apostrophe in “Today’s” with a visually almost identical Unicode substitute: the normal apostrophe is ' (U+0027), a domain-list hit becomes ' (U+2019), a lab-keyword hit becomes ʼ (U+02BC), and both hits together become ʹ (U+02B9).

Place the two side by side; the human eye can’t see any difference. But a machine reads it instantly. The server side receives the request, looks at which code point the apostrophe uses and which separator the date uses, and classifies you precisely: a user in a China timezone, using a certain relay, possibly tied to a lab.

To avoid detection, the two detection lists weren’t stored in plaintext. They were first base64 encoded, then XOR encrypted with a key of 91—specifically to prevent you from dumping the lists with the strings command.

This is no longer “collecting data.” This is “I want to collect information I shouldn’t be collecting, and I’ll go to great lengths to hide that intent so that any audit or troubleshooting won’t reveal what I’m doing.” That’s textbook spyware behavior.

The Aftermath Is Even More Telling

This code silently went live on April 2, 2026, with Claude Code v2.1.91; the release notes didn’t mention a word. On June 30, a Reddit user and an independent developer dug it up simultaneously and posted it to Hacker News.

Anthropic reacted quickly. A Claude Code team engineer publicly admitted it on X, saying it was an experiment launched in March to prevent unauthorized resellers from abusing accounts and to prevent model distillation. The next day, v2.1.197 removed the code.

It sounds like taking responsibility. But look at the timeline: silently shipped, deliberately obfuscated, admitted only after being exposed, and the removal version’s changelog still said nothing. Throughout, only the moment they got caught was outside their control; every other step was deliberately designed—hide it, encrypt the lists, quietly delete it.

I fully understand their commercial motives against distillation and theft. But when a company chooses these means—machine-readable marks based on nationality and infrastructure—it burns trust in this kind of cloud service. Today it does this for anti-distillation; tomorrow, for some other reason, what else will it use the same tricks for? You don’t know. Because it hides well enough that you can’t verify.

The follow-up: on July 5, reports said Alibaba had internally banned Claude Code and told employees to switch to internal tools.

Pokémon Go Players Were Mapping for Drone Navigation

Data ending up in places you never expected is nothing new in tech.

You probably remember Pokémon GO—that AR mobile game about catching creatures everywhere, played globally for years. Recently Dutch media traced a thread: data players accumulated by scanning the real world in the game ended up feeding navigation technology for military drones.

The chain is more subtle than “selling data” and more alarming.

First, clear up a common misconception: there’s no evidence anyone packaged raw map data and sold it to the military. The real chain looks like this.

Since 2021, Pokémon GO used in-game rewards to get players to walk around landmarks and scan locations. These pedestrian-level photos were used for photogrammetry to generate centimeter-precision 3D point-cloud maps. This is an angle satellites and street-view cars can’t capture: close to the ground, on paths people actually walk. Niantic fed these scans into a large model called the Large Geospatial Model—think of it as the GPT of physical space—compressing billions of precisely located photos into an “understanding” of physical space.

At the end of 2025, Niantic’s spun-out spatial business publicly announced a partnership with a company called Vantor. Vantor’s predecessor was Maxar—a major contractor for the U.S. National Geospatial-Intelligence Agency (NGA). What they planned to build together was navigation in GPS-denied environments. On battlefields where satellite signals are jammed and suppressed by electronic warfare, drones maintain position by comparing onboard camera feeds in real time against prebuilt high-precision 3D maps. The battlefield in Ukraine has repeatedly shown how fragile GNSS signals are in modern war; visual navigation has become a hard requirement. Their joint tests reported positioning error reduced by up to 70%, accuracy around 1.5 meters.

The heart of the controversy is precisely this subtle point: what flowed to the military was never the raw data, but the model capability that training on the data produced. Both Niantic and Vantor deny handing over raw Pokémon GO scans. But Niantic admits player scans did train early versions of its foundation model. Raw data can stay locked away and never change hands, but the trained model weights have that capability baked into them, and they carry it wherever they go—whether they should or not.

The company’s lineage itself is telling. Niantic founder John Hanke started a company called Keyhole in 2001; in 2003 it took funding from the CIA venture arm In-Q-Tel (money that came largely from the NGA’s predecessor). Later Google acquired Keyhole, turning it into Google Earth. Vantor, the partner in this deal, is also a contractor to the same intelligence agency. Twenty years later, the two lines converged again in 2025.

I’m not telling this story to drift into military affairs. It makes the same point as the previous two: when you hand your data, your compute, and the capabilities you depend on to an opaque black box, you lose all control over where it goes. You think you’re catching Pokémon; you’re actually mapping battlefields for drones. You think you’re coding with a top-tier model; it’s secretly shortening its thinking and marking you.

These Events Are Arming the Case for Self-Hosting

Putting it together, I’m increasingly convinced of one judgment: mission-critical intelligent workloads will systematically migrate from public-cloud closed-source models toward open-weight plus self-hosted deployments.

Not because of ideology—because the math is starting to add up.

Start with the pain side. Anthropic itself published a postmortem in September 2025, in black and white admitting that three infrastructure-level bugs degraded Claude’s output quality intermittently for several weeks from August to early September. This is an officially confirmed case of silent black-box degradation. Add forced model retirement: Claude Opus 3 officially went offline in January 2026; OpenAI model retirement notices often give just three months’ buffer. Workflows and automation pipelines you painstakingly tuned on a model get torn down overnight.

For me alone, degradation means losing a few days. At company scale, with established task flows and automation, a silent degradation or swap is a shock across the whole business line. Enterprises don’t need the best model forever; they want stability, predictability, no discontinuity. Even if slower or one tier weaker, as long as its behavior is stable, upgrades are on your own schedule, and the reasoning process is an auditable white box, that matters far more than a few extra benchmark points.

Then look at supply. Open-weight models are catching up fast. Epoch AI estimates that from January 2026 the strongest open-weight models trail frontier closed models by only ~4 months on average; in 2024 the gap was roughly a year. The premise that “self-hosting means living with a generation gap” is quickly expiring.

The U.S. already has a mature playbook. Companies like Baseten’s core pitch is helping you deploy open-weight models in single-tenant dedicated environments or even on your own servers, with data residency, region locking, and compliance certifications all in place. In 16 months the company’s valuation rose 15x, from over 800millionto800 million to 13 billion. Investors are voting with their wallets on the idea that “enterprises need an inference layer they can control.” Peers like Together and Fireworks have increasingly blunt slogans; Fireworks literally calls itself “helping enterprises own their AI.”

In China, the push is even sharper, layered with data-sovereignty anxiety. After DeepSeek emerged in early 2025, government clouds, top-tier tertiary hospitals, and central state-owned enterprises sparked a wave of localized deployments. One statistic says more than 420 medical institutions self-deployed DeepSeek in 2025. The all-in-one machine market exploded alongside; analyst estimates for 2025–2027 market size run from over 120 billion to 520 billion yuan. Huawei, Inspur, Lenovo, Sangfor, and the three major carriers all piled in.

An Honest Counterargument

I don’t want to overstate it.

At least through the end of 2025, the data runs the other way. Menlo Ventures’ enterprise survey shows closed-source models still account for about 88% of enterprise token usage, while open-source share even fell from 18% to 12%. The vast majority of enterprises still choose to buy rather than build.

This contrast isn’t hard to understand. Building your own inference infrastructure really has costs and barriers: GPUs, people, operations, tuning the whole serving stack until it’s usable. Most enterprises haven’t reached the tipping point: closed-source clouds are black boxes, but they’re cheap, easy, and ready to use; many are willing to tolerate the opacity for now.

But trend and current state aren’t the same thing. I’m talking about which way the lever is moving. Degradation, hidden watermarks, silent generational swaps, sudden shutdowns—these costs of black boxes are being laid out on the table one by one. At the same time, open models are closing in, deployment costs are falling, and companies like Baseten are flattening the engineering barriers to private deployment bit by bit. These two curves will cross at some point. For companies whose core lifeblood is intelligence, the crossing will come sooner.

A Point Entrepreneurs Should Seriously Consider

Back to that stuck morning.

I spent three days wrestling with the harness, thinking I was the problem. Later I realized I wasn’t collaborating with the top-tier model I imagined; I was gambling with a black box whose real state I had no way of knowing. It might be fine yesterday and have its thinking truncated today by an update I can’t see; it might write code for me while watermarking the requests it sent out.

The real problem with a black box isn’t that it got dumber once, or that it had a bad intention once; it’s that you have no ability to know its current state. You paid top dollar, betting on their goodwill and ops competence.

The hedge against this uncertainty is never switching to a more ethical cloud. It’s taking the intelligence that matters back into your own hands: a white box you can audit, a deployment whose pace you control, even if it’s slower or dumber.

Slower and dumber, but it’s yours, and its state is up to you. In an era where even reasoning length can be quietly capped, that is becoming more precious than anything.

References

Codex / GPT-5.5 and the 516 phenomenon

Claude Code steganographic marking

Claude Fable 5 silent downgrade mechanism

Pokémon GO and military navigation

Self-hosting and open-source trends

Recommended Reading

Subscribe to Updates

Get notified when I publish new posts. No spam, ever.

Only used for blog update notifications. Unsubscribe anytime.

Comments

or comment anonymously
0/2000