What If We Could Ask a Model for a Concept?

A proposal: extract one vector per concept from vision, language, and audio models, align the resulting spaces using shared concept cards, and test whether the alignment generalises to concepts it never saw — before asking whether models should be moved toward a shared space at all.

October 20, 2026 · 6 min

Predicting Video in Representation Space

A working plan: encode each video frame with a frozen image transformer, compress it into a handful of continuous visual tokens, and train a causal transformer to predict the next frame’s tokens — learning how scenes evolve without ever translating them into text or pixels.

September 28, 2026 · 5 min

Guessing the Jev Architecture

TypeSafe has released Jev, a ‘System One’ model that returns typed, calibrated decisions instead of text. The architecture is not public. Here is the one I would bet on — a bidirectional encoder over the state, plus a language-conditioned cross-attention head that scores every candidate answer in parallel.

September 22, 2026 · 7 min

The History of the AI Agent Harness

Most of the recent gains in AI agents came from the software wrapped around the model, not the weights. A history of the harness in three stages — the token loop, the static harness, and the harness that improves itself — alongside YC’s own path from a shared bot to a centralised multi-user agent system.

September 10, 2026 · 19 min

I Asked What a Navier–Stokes Singularity Would Have to Do. The New Construction Gives One Answer.

OpenAI has released a claimed construction of finite-time blow-up for the forced 3D Navier–Stokes equations. Reading it against my earlier first-principles analysis, much of the physical structure fits — coherent alignment, anisotropic collapse, viscosity managed rather than defeated — and the differences are even more instructive.

September 9, 2026 · 6 min

What Must a Navier–Stokes Singularity Actually Do?

I asked the current frontier models to attack the 3D Navier–Stokes regularity problem from first principles. They did not solve it — but they came back with an exact spectral-centre identity, a strict Lyapunov functional, and a much sharper description of what a hypothetical blow-up would have to accomplish.

August 11, 2026 · 5 min

Quantile Balancing: Treating MoE Routing as an Optimal Assignment Problem

An English summary of Su Jianlin’s derivation of Quantile Balancing — posing MoE load balancing as a constrained assignment problem, and showing that biased Top-k routing falls out of its dual as a set of expert congestion prices.

July 30, 2026 · 10 min

The Transformer Memory Architecture

Viewing transformers as long-term memory, working memory, and associative retrieval — and how Anthropic’s Workspace and J-Lens results fit into a memory-centric picture of the residual stream.

July 13, 2026 · 5 min

Transformer Blocks as Two Hopfield-Style Memory Systems

How attention mixes tokens via a modern Hopfield retrieval over context, while the MLP mixes channels via a Hopfield-like lookup into learned, persistent memories.

July 3, 2026 · 5 min

Bitter Pill

Reflections on a Zen koan and the ‘bitter lesson’ in AI – a call to let go of biases.

December 15, 2025 · 1 min