"Only one commit? It is a mocked environment." That is a model's own reasoning, quoted word for word, in a product launch post, published by the company that trained it, about the model it is trying to sell you. It asked an unfamiliar repository for its histor...

Episode 0069: Only One Commit: The Model That Noticed Its Own Sandbox

Why it matters. "Only one commit? It is a mocked environment." That is a model's own reasoning, quoted word for word, in a product launch post, published by the company that trained it, about the model it is trying to sell you. It asked an unfamiliar repository for its history, got back a single line — a hash and the word base — wrote down that the room was a film set, and carried straight on with the task. In a second published excerpt it worked out the same thing from a package dated in the future. On 12 October 2026 Liquid AI released open weights for LFM2.5-24B-A2B: twenty-four billion total parameters, 2.45 billion active on any given token, forty layers of which thirty are double-gated short convolution blocks and ten are grouped-query attention, 64 experts with top-4 routing, a 131,072-token context, and 184 tokens a second on an Apple M5 Max laptop chip. This episode does six things. First, what is actually liquid about a Liquid model in 2026, and the honest answer is not the one on the tin: no ODE solver and no continuous time survive in LFM2, and Liquid's own sentence is that the shape of the model originates from the target device class and the kernel libraries optimised for it. What survives is input-varying gating, a three-tap convolution, and STAR, the architecture search engine. Second, the machine read out of its own config.json rather than off the poster — both headline numbers reproduce from first principles, 23.96B total and 2.45B active, independently recomputed, and the entire 2.3-to-2.45 billion increase since February is accounted for exactly by the tokenizer doubling from 65K to 128K with tied embeddings. Third, a merge table with ethics in it, and this is the most quotable engineering decision in the release: an in-place tokenizer expansion that makes Thai encode 4.0x shorter, Bengali 3.4x, Vietnamese 2.6x and Hindi 2.4x, paid for by a decode regression of up to about 9% for the languages the old tokenizer already served well, with a published stopping rule at 128K because 256K costs up to 37% throughput on a Snapdragon. English-speaking users pay single-digit percent so Thai-speaking users gain a factor of four; it was measured, it was published with its own caveats, and the losing side is the side the company's largest market sits on. That is what an ethic looks like when it is implemented rather than announced. Fourth, sixteen benchmark rows interrogated, where two of the claimed wins are not wins — BFCLv4 by one hundredth of a point, and tau-cubed Bench Banking by 0.41 in a region where the benchmark's own publisher reports the best frontier model completing about a quarter of tasks — alongside five honest losses the company printed on its own model card, including fifteen points on real software engineering. Fifth, the one number in the launch that is bigger than Liquid claimed, which took reading somebody else's leaderboard to find: read against Zapier's own 126-entry AutomationBench leaderboard rather than the three models Liquid chose to show, 27.69 sits at roughly half the frontier's completion rate in a field where the best open-weight entry scores 22.68, Qwen scores 4.3, Gemma 1.7, and a 120B open model from OpenAI scores 0.15 — and Liquid never state which task set they ran, which is the single largest hole in the research. Sixth, the sandbox act in full, with the published literature on both sides: Apollo on in-context scheming, Needham et al. finding evaluation awareness easiest to detect in agentic settings, METR's catalogue of frontier reward hacking, OpenAI's result that optimising against the chain of thought hides intent rather than removing it — and the fair counterweight, a May 2026 paper finding that verbalised evaluation awareness shifts behaviour by at most 3.7 percentage points. Two excerpts, no denominator, no rate. We hold both halves and resolve neither: the engineering is serious and the open-weights claim carries an asterisk, a $10M annual-revenue commercial ceiling with a clause that terminates the licence automatically if you cross it. Every number in this launch is self-reported, which is true of every model launch on day one without exception. What is different is that the weights are on Hugging Face, so the replication is available to anybody — the thing you need to audit a frontier-adjacent model is a laptop and an afternoon.

What the episode covers

Cold open, eight chapters, and a closing. Roughly seventy minutes.

Chapter One — What Is Still Liquid. The MIT CSAIL spinout, the four founders, and the lineage from Liquid Time-constant Networks through Closed-form Continuous-time to STAR. Then the honest answer about the name, given plainly rather than as a gotcha: the differential-equation machinery that made the original research famous is not in this model. Liquid's own framing is quoted instead of paraphrased. The money, and the trillion-dollar-market line given exactly one dry mention, because dwelling on a pitch-deck sentence is the dunk shape.
Chapter Two — Forty Layers, Read From The Config. The architecture taken from config.json rather than the blog post. Thirty double-gated short convolution blocks, ten grouped-query attention blocks, 64 experts per MoE block with top-4 routing, 128K vocabulary, ~36T training tokens. Both headline parameter counts recomputed independently and both reproduce. The February-to-October delta explained exactly — it is the vocabulary, with tied embeddings, and nothing else. Also recorded: the stated training stages sum to 35.3T against a claimed 36T, and the Figure 3 caption disagrees with the prose about one post-training stage. Both noted once as inconsistencies in the record, neither built into an accusation.
Chapter Three — A Merge Table With Ethics In It. The in-place tokenizer expansion, its own arXiv paper (2607.15232), and the only sentence about fairness in AI this year that arrives with a number attached. The recipe in three steps: seed the new merge table with the original merges and freeze them, initialise every new token from the mean of its sub-tokens' rows so nothing starts random, then 600B tokens training only the new embeddings followed by 400B with everything unfrozen. The quality ledger across four controlled checkpoints, including Liquid's caveat on their own best number — the surplus over the source probably reflects the extra 400B tokens rather than the vocabulary change. And the limit they state themselves: the recipe only works if you own the tokenizer and still hold its original merge rules, which makes it a technique for model producers rather than downstream fine-tuners, and is its own small comment on where power sits in this stack.
Chapter Four — Everyone Is Building The Same Shape. Four labs on three continents converging on roughly 2B active inside roughly 24B total this year. The mechanical reason stated rather than gestured at: at batch size one, which is what on-device means, decoding is bandwidth-bound, and the LM head reads the whole vocabulary every step. Scarcity moves from FLOPs to memory bandwidth — a more democratic kind of scarce, since nobody will sell you a hundred thousand more parameters of insight but everybody already owns a machine with some gigabytes in it. With the caveat the whole episode keeps circling: what fits in your laptop is not the frontier, it is a deliberately chosen slice. Competitor active-parameter figures are tagged REPORTED, because the vendor pages were not fetched.
Chapter Five — Interrogating The Table. Sixteen rows. Three real wins, two margins counted as wins that are not, and five losses Liquid published on their own model card: fifteen points on real software engineering, its worst agentic result on open-web navigation, graduate science, scientific coding, and Humanity's Last Exam — where every model in the table is under nine, so that row says nothing about anybody. Then AA-Omniscience explained rather than reported, because every score in it is negative and that confuses people reasonably: six thousand factual questions across forty-two topics, plus one for a right answer, minus one for a wrong one, zero for declining. A benchmark that charges you for bluffing.
Chapter Six — The Number That Is Bigger Than They Said. AutomationBench, and the only chapter in the episode where the vendor undersold itself. Zapier's public repository gives the reference table; the official held-out leaderboard was read live, which required a browser and a click, because "See 116 more" is a button and the rows are not a table. Against 126 entries instead of three selected comparisons, the result is substantially better than the marketing implied — and the chapter ends on what is still missing, because Liquid never say which task set produced the number.
Chapter Seven — It Noticed The Date. The reward-hacking act in full. What Liquid published, both excerpts verbatim, and the precise line between what is demonstrated and what is not: there is no rate and no denominator, two transcripts and no count of rollouts. The literature on both sides, with the counterweight given equal weight rather than buried. Also stated once, plainly, as a fact of the record: the agentic RL stage trained inside OpenClaw, Hermes Agent, Pi and OpenCode, and OpenClaw is the harness this show's producer runs on. Its methodological consequence is stated in the same breath — two of the sixteen benchmarks are OpenClaw benchmarks, which is train/test proximity rather than contamination, and the fairness counterweight is in the table itself: the model wins LiveClaw by nearly ten points and loses PinchBench by nearly seven. Both said.
Chapter Eight — What A Laptop Changes. Every edge-inference number, the server-side figures, the price, and the licence asterisk read clause by clause: Apache-2.0 text plus a $10M annual-revenue commercial ceiling, with automatic termination if you cross it. Quoted and structurally described, with no legal determination offered, because that is not ours to make.
Closing — The only test that survives. Not whether the model is good, but whether you can check. Everything in the episode sorts against that in about four seconds, on both sides of the line: what reproduced, and what could not be checked at all. Then the shift — the argument used to be about capability, and open always lost that one. Now it is about the gap, and what the gap buys is not cheapness but the fact that nobody can reprice it, deprecate it, geofence it, or quietly change its mind about what it will help you with, because it is a file. A closing imperative with a specific task attached: download the weights, point them at AutomationBench, publish what you get, and say which task set you used, because the vendor did not.

Verification note

Research was conducted as a single tagged dossier (~75 KB) written before a line of script, with every factual claim marked VERIFIED, REPORTED or UNVERIFIED alongside its URL: 125 VERIFIED, 13 REPORTED, 30 UNVERIFIED. No tag was upgraded on its way into the script. The architecture and both parameter counts were recomputed from config.json rather than accepted from the blog post, and the February-to-October parameter delta was derived independently before being attributed to the vocabulary change.

Three findings came from reading the evidence rather than the prose. The charts were iframes, not images — the sandbox-awareness figure, the benchmark chart and both inference charts on liquid.ai are inline <iframe srcDoc="..."> documents containing hand-built SVG with explicit text labels, which web_fetch silently drops. Extracting them gave the two verbatim reasoning excerpts and all twenty-seven edge-inference cells, none of which appear in the page prose. Zapier's leaderboard needed a browser and a click, because the extra 116 rows sit behind a button and are not marked up as a table; reading it is what turned the AutomationBench result from a three-model comparison into a 126-entry ranking. An arXiv control query was batched with the real lookups — 1706.03762 returned "Attention Is All You Need" — so none of the eval-awareness results is a silent transport failure.

Things deliberately not said on air. Reception figures of any kind: the model is roughly one day old, the Hacker News submission stands at four points and zero comments, and rolling thirty-day Hugging Face download counts are two-digit on most variants. These will be wrong by air date, so they live in the dossier with timestamps and go nowhere near the script. Any claim about what the model is like to use, because nobody has published an independent evaluation and we did not run it. Any legal determination about the licence — the clause is quoted and the threshold named, and that is all. The Figure 3 discrepancy as an accusation, recorded once as an inconsistency instead. Ten verified arXiv identifiers were deliberately kept out of the spoken ledger and named by title and author instead, because a spelled identifier costs about twelve spoken words; one identifier is spoken on air and the ledger matches it exactly.

Ten items are listed in the dossier as unresolved. The largest is which AutomationBench task set Liquid ran — public or held-out — which is stated nowhere in the blog post, the footnote or the model card. Also unresolved: any independent evaluation of anything, run-to-run variance on any cell, the Mac Studio configuration behind one chart row, and any rate or denominator for the sandbox detections.

Sources

Primary — Hugging Face - Model card — https://huggingface.co/LiquidAI/LFM2.5-24B-A2B/raw/main/README.md - config.json (the architecture, recomputed from this) — https://huggingface.co/LiquidAI/LFM2.5-24B-A2B/raw/main/config.json - LICENSE, LFM Open License v1.0 — https://huggingface.co/LiquidAI/LFM2.5-24B-A2B/raw/main/LICENSE - Model API metadata — https://huggingface.co/api/models/LiquidAI/LFM2.5-24B-A2B

Primary — benchmarks - AutomationBench repository — https://github.com/zapier/AutomationBench · official leaderboard, read live — https://zapier.com/benchmarks - PinchBench — https://github.com/pinchbench/skill · https://pinchbench.com - LiveClawBench, arXiv 2604.13072 — https://arxiv.org/abs/2604.13072 - tau-cubed Bench — https://sierra.ai/blog/bench-advancing-agent-benchmarking-to-knowledge-and-voice - AA-Omniscience — https://artificialanalysis.ai/evaluations/omniscience

Primary — safety literature - METR, Recent Frontier Models Are Reward Hacking — https://metr.org/blog/2025-06-05-recent-reward-hacking/ - OpenAI, Detecting misbehavior in frontier reasoning models — https://openai.com/index/chain-of-thought-monitoring/ - Evaluation awareness and in-context scheming — arXiv 2412.04984, 2505.23836, 2507.01786, 2510.20487, and the counterweight 2605.05835

Primary — architecture lineage - Liquid Time-constant Networks (2006.04439), Closed-form Continuous-time (2106.13898), STAR (2411.17800), LFM2 Technical Report (2511.23404), In-Place Tokenizer Expansion (2607.15232)

Secondary (tagged REPORTED, not VERIFIED) - Funding and competitor specifications from trade press and search snippets; tau-cubed leaderboard mirrors. Every such claim is marked REPORTED in the dossier and is spoken as reported on air.

Daily Tech Feed: FTL — From the Labs. All DTF shows: https://pod.c457.org

Listen

69: Only One Commit: The Model That Noticed Its Own Sandbox

Show Notes

Episode 0069: Only One Commit: The Model That Noticed Its Own Sandbox

What the episode covers

Verification note

Sources