I used to be a professional chess player. These days, I work on AI. For a long time, those two parts of my life seemed to have an obvious connection: computers had already become better at chess than humans, and the rest of AI was trying to catch up.

But the connection now feels less straightforward.

We have machines that play chess at a level I cannot approach. We also have language models that can talk about almost anything. Bringing those abilities together should give us something extraordinary: a patient, knowledgeable chess coach, available whenever we want to study.

In my free time, I have been building a chess AI agent with exactly that goal: a personal grandmaster coach I can study with whenever I want. It is also why a recent paper, Language Models that Play Chess and Explain Their Moves, caught my attention. Its model, Queen, attempts to bring strong chess play and language together.

I found the promise exciting. Working with it also made me less comfortable with a question I used to think was simple: what does it mean for an AI to understand what it is doing?

We solved the opponent problem long before the teacher problem

In 1997, Deep Blue defeated Garry Kasparov in a match under standard tournament conditions. IBM describes a system capable of evaluating around 200 million positions per second. That was an extraordinary achievement, built around specialized computation and chess knowledge. IBM’s history of Deep Blue gives a useful account of that moment.

Today, Stockfish gives an ordinary chess player access to an opponent far beyond human strength. For scale, the September 10, 2026 CCRL 40/15 list rated Stockfish 18 (64-bit, 4CPU configuration) at 3649—well above 3000. That is an engine-list rating under specified testing conditions, not a FIDE rating or a number directly comparable to the Lichess-based estimates discussed below. I use Stockfish in my own experiments with the agent. Finding a stronger move is usually much easier than turning that move into a useful lesson.

As a player, I care about what happens after the engine recommends a move.

Why does this move matter? What was wrong with my idea? Which reply did I underestimate? What feature of the position should I recognize next time?

An engine can supply candidate moves, evaluations, and a principal variation: a sequence representing its current best continuation. Those are valuable forms of evidence. A strong player can study that evidence and work out an explanation. But an evaluation and a line do not automatically perform that teaching work for the student.

Consider a hypothetical coaching conversation. A student wants to exchange a bishop for a knight. The coach recommends keeping the bishop. Saying that the engine evaluates one choice more highly settles very little for the student. The useful discussion concerns the pawn structure, the squares the remaining pieces can use, the opponent’s resources, and the conditions under which the exchange would become good.

The student needs to take something away that survives beyond this position.

This is why a language model with weaker play than Stockfish could still be a major advance. Its value would come from making strong chess knowledge accessible through a conversation: comparing ideas, answering follow-up questions, and helping a player change how they think.

Superhuman chess is still not solved chess

How can a machine become so strong? Modern Stockfish combines alpha-beta search with NNUE neural-network evaluation: search investigates continuations, while a learned evaluator assesses positions. Its strength comes from combining those abilities. The official Stockfish FAQ explains both components.

Yet even with engines this strong, chess has not been completely solved. There is no known proof establishing whether the initial position is a win for White, a draw, or a win for Black under perfect play. Beating every human and establishing the outcome against every possible defense are very different achievements. Computer-chess researchers distinguish these goals in On the Road to Perfection?.

The size of the search space helps explain the difference. Shannon’s classic estimate put the chess game tree on the order of 10¹²⁰. A commonly cited estimate for Go is around 10³⁶⁰. These are rough game-tree estimates, dependent on assumptions about branching and game length, rather than exact counts of legal board positions. See Shannon’s original paper and DeepMind’s comparison of game complexity.

An engine does not have to explore that entire tree to find a move that beats me. It searches ahead from the current position, spending more effort on promising or critical continuations, then evaluates positions without necessarily reaching the end of the game. Pruning, reductions, and extensions make that search selective. A reported depth of 20 counts half-moves, or plies—nominally ten moves by each side—but does not mean every branch was examined that far. There is no universal depth at which chess becomes solved or an engine becomes unbeatable. Stockfish’s explanation of search depth.

That is the distinction I care about: a bounded search can be extraordinarily useful without being a complete solution. Go’s much larger tree makes exhaustive search even less practical. Learned evaluation and search can work together in both games, as AlphaZero demonstrated across chess, Go, and shogi. DeepMind’s AlphaGo retrospective traces that progression.

These systems have already given us superhuman opponents. What I want as a student is access to the chess knowledge behind their decisions.

Fluent language, unreliable chess

Language models seem to offer the missing interface. They can hold a conversation, explain ideas in accessible language, and respond to a student’s questions. But ask a general-purpose chatbot to play an entire game without a chess engine, and fluent language can coexist with surprisingly weak play.

GothamChess’s chatbot matches made that mismatch hard to miss. His 2025 Chatbot Chess Championship put several chatbots into the same bracket. In the ChatGPT–Meta AI game, the commentary describes pieces reappearing and a bot attempting to move its opponent’s king. The available transcript captures how far the game wandered from legal chess. His ChatGPT vs Stockfish: ABSURD CHESS offers another example of the contrast between conversational AI and a dedicated engine.

These are entertaining demonstrations, not controlled estimates of every LLM’s strength. Model versions, prompts, board representations, and the handling of invalid moves matter; newer systems can perform much better. But the underlying gap is important: producing convincing chess language does not by itself establish reliable board tracking, calculation, or move selection.

A structured benchmark gives the concern more substance. The December 2025 paper LLM Chess: Benchmarking Reasoning and Instruction-Following in LLMs through Chess evaluated existing models with tools for inspecting the board, listing legal moves, and making moves. In its engine comparison, o3 (low) reached an estimated Elo of about 758, anchored to Dragon 1 skill levels associated with Chess.com ratings. This was an evaluation of existing models, not a chess fine-tuning method. The result is specific to that agentic protocol and cannot be directly compared with Queen’s later Lichess-anchored estimate.

The paper is also a historical snapshot. Its authors maintain a live LLM Chess leaderboard, linked from the paper’s repository. In the data I checked on November 12, 2026, leading estimates were around 1600: GPT-6 Astra (high) was approximately 1614 ±110 across 67 games. The current leaderboard combines results against Dragon with results against a random opponent calibrated against Dragon, and reports 95% confidence intervals. It also tracks game completion separately from playing strength. These are estimates within a particular testing setup, rather than official human ratings. Leaderboard methodology and underlying results.

That suggests an obvious combination. Let the engine find strong moves, and let the language model explain them. Could I give an LLM Stockfish’s evaluation and principal variation, then ask it to turn that analysis into a personal lesson?

It is an attractive starting point. The difficult part is whether the explanation faithfully captures why the move works. A line shows one continuation; a lesson needs to identify the important idea, compare the student’s alternative, and explain which replies matter. An LLM can describe the supplied moves while still inventing the reason they are good.

So there are two connected questions. Can a language model explain an engine’s decisions accurately? And can we give the language model deeper access to chess expertise, so its own analysis and language become better grounded?

One route is to train the language model directly on chess data. An earlier example is ChessGPT: Bridging Policy Learning and Language Modeling, published in 2023. Its authors fine-tuned RedPajama-3B on games, chess language, and mixed game–language data, then applied instruction tuning to build a chat model. This brought the learning of moves and language into the same training pipeline. But learning to produce both still leaves the question of whether an explanation faithfully supports a decision.

Queen takes a different architectural route: it connects the language model to representations from a chess expert. That is what makes it interesting to me.

What Queen actually demonstrates

Queen is a roughly 4B-parameter hybrid, rather than an ordinary text model fine-tuned on game records. Its released checkpoint combines an Lc0 BT5 chess encoder with a SmolLM3 language decoder. The board encoder supplies chess representations to the language component. The model card also warns that loading the decoder alone omits board conditioning.

The paper describes a cross-attention bridge, a board-understanding curriculum, and seven rounds of search distillation. Candidate continuations are analyzed, consolidated into explanations, filtered with Stockfish, and used for further training. Its reported ratings are:

Model Estimated rating
Queen 2697
Gemini-3.1-Pro 2201
GPT-5.6-Sol 2071
Median grandmaster reference 2730

These estimates use 32 games per model and a Lichess-anchored scale. They are neither FIDE ratings nor independent certification of a grandmaster title. Queen paper, Sections 3–4.

The LLM Chess leaderboard illustrates why the rating scale matters. It uses Dragon/Chess.com-based anchors, while Queen’s evaluation uses a Lichess-based scale. The opponents and interaction protocols also differ, and Queen was absent from the leaderboard’s November 12 snapshot. These figures describe performance in different testing environments; they cannot establish the size of Queen’s advantage across benchmarks. Queen’s 2697 is best understood as a promising result within its published setup, with broader playing strength still requiring independent evaluation.

That is still exciting. A compact model has learned to connect expert representations with move generation and language. It offers a plausible direction for building the kind of coach I wanted.

The same evaluation exposes the gap: Queen’s first move was mistake-free on 97.1% of general positions, but its full predicted continuation was mistake-free on 66.6%. Conceptual coherence scored 2.76/5 under an LLM judge, below the frontier baselines; the authors discuss persistent conceptual hallucinations. Queen paper, Sections 4.2–4.3.

Here, “mistake-free” uses the paper’s engine-based threshold; it does not mean perfect play. Those measures are not interchangeable. A good move does not certify the explanation attached to it. An LLM judge is also a limited proxy for what an experienced human teacher would accept.

The rating was what first caught my attention. Using Queen made the distance between a strong chess model and a dependable coach much more tangible. A recommendation could look promising, yet the explanation around it could lose contact with the position. I have not reproduced the paper’s Elo evaluation, and my own checks cannot establish a different rating. What they have shown me is that I still cannot treat the model as the grandmaster coach I hoped to have.

Trying to build my own grandmaster coach

In my free time, I have been trying to build an agent I can study with after a game. I want to bring it a position, explain what I was thinking, and work through where my plan went wrong. The attraction is the conversation: a strong player who is always available, patient enough to examine my alternatives, and able to explain an idea in a way I can use next time.

Much of the work has instead gone into checking the foundations of that conversation. I have had to make sure the agent is analyzing the actual board, that its variations are legal, and that the position it describes is the one those moves reach. Even with Queen supplying chess analysis, plausible explanations can contain errors. The surrounding code and review help catch them, but they do not turn every answer into reliable instruction.

The difficult failures go beyond an illegal move. A legal continuation can support a misleading explanation. A correct recommendation can arrive with a rationale that leaves out the opponent’s critical resource. An engine can help assess the moves, while checking the strategic prose remains a separate problem.

This has made me more cautious about what a playing score tells me. Grandmaster-level results in an evaluation would be impressive, but a personal coach also has to stay grounded through a conversation: follow a new variation, respond to an objection, and recognize when its earlier explanation no longer applies. In my experiments, that reliability is still missing often enough to matter.

Why I still believe in a grandmaster coach

Those difficulties have not made me pessimistic about the goal. I believe we will have chess language models that combine grandmaster-level play with dependable explanations. Richer expert data seems to me a major part of the path.

The data I would want goes well beyond a record of moves. Imagine a player such as Carlsen explaining which candidates he considered, why an attractive move failed, which reply changed his evaluation, and what he expected from the resulting position. That kind of material gives a model something a bare game score cannot: a connection between a decision, the alternatives, and the ideas used to compare them.

Expert annotations would still be accounts of thinking, rather than a complete recording of a player’s mind. They would need to be checked against the position, and successful training would need to show more than the ability to repeat familiar commentary. But I find it plausible that carefully grounded explanations, combined with targeted training and engine feedback, could produce a much stronger conversational chess model.

I suspect the limiting factor is partly where that effort goes. Engines already outperform people at chess, so another improvement in chess AI has to justify its value through something beyond winning games. Mathematics and science offer laboratories opportunities to produce results that existing tools cannot already supply. Public efforts such as DeepMind’s science program and AlphaEvolve make that direction visible. My impression is that a specialist chess coach competes with those ambitions for attention; it does not follow that the coaching problem has been solved or cannot be solved.

Even if we built that coach, I would still hesitate to treat its achievement as the whole of intelligence. A model trained on expert explanations might become excellent at selecting moves and describing them. For me, stronger evidence of understanding would come from what happens beyond the familiar examples: whether it recognizes when a principle stops applying, revises a mistaken assumption, and explains a changed position without inventing a reason.

Language complicates that judgment in an interesting way. It is one of the main ways people make their thinking available to each other. A model that reasons consistently through language may be doing something worth calling understanding. I would not dismiss that possibility simply because it learned from data. But fluency alone gives me little confidence; the explanation has to remain connected to the situation it describes.

This tension follows the discussion into mathematics. An AI can become powerful at finding solutions within a system that supplies feedback and checks its work. That is a real capability. The question I keep returning to is how much of the apparent understanding belongs to the model, how much depends on the surrounding tools, and what the system can explain when an accepted result is no longer enough.

When a correct answer is not enough

These experiments kept returning me to a word we use very freely in AI: verification.

In a chess application, it can mean checking whether a move is legal, searching whether a move loses material, or judging whether an explanation makes sense. Those checks have very different guarantees.

Check What it can establish What remains open
Replay a move under the rules The move is legal in that position Whether it is good
Run a finite engine search An evaluation under that search budget Whether deeper search changes the conclusion
Review a strategic explanation Reasons to accept or reject its claims Whether all relevant errors were caught
Check a formal proof The formal statement follows within the checked assumptions Whether it states the intended problem or gives useful insight

Verification does not require a unique answer. Several moves can be equally good; several programs can satisfy a specification; several proofs can establish a theorem. What matters is having a sufficiently precise claim and a checker suited to it.

Speculative decoding illustrates a useful asymmetry between generating a candidate and checking it. A small model proposes several tokens, and a larger target model scores the proposed sequence in a parallel forward pass. With the appropriate acceptance and correction procedure, this can produce several accepted tokens per target-model call while preserving the target model’s output distribution. The efficiency comes from reducing sequential decoding work. The check concerns agreement with that distribution, so it also preserves the possibility of hallucinations. Fast Inference from Transformers via Speculative Decoding.

Mathematics offers a more consequential use of the generate-and-check pattern. An LLM can propose a formal proof or intermediate steps, while a proof assistant such as Lean checks the resulting proof object. An unsuccessful attempt supplies feedback for further search; a successful proof can be checked mechanically. Lean’s small kernel performs that logical check. The language model contributes proposals and search guidance. Lean’s description of proof checking.

This division of labor is already central to formal theorem-proving research. For example, DeepSeek-Prover-V2 trains language models to construct Lean proofs using subgoal decomposition and reinforcement learning. It provides a concrete example of mathematical reasoning coupled to a formal proof system.

The advantage is having a precise claim and a reliable source of feedback. A problem having a definite answer does not automatically make finding or checking it easy. But where a useful checker exists, a system can explore candidates, reject failures, and build on successful steps without asking a human to inspect every attempt. This gives AI systems a powerful way to improve within a formal environment.

For an accepted formal proof, that can remove much of the need to check each logical step manually. The proof still has to be constructed, and the formal statement must express the intended claim under appropriate assumptions. Lean’s proof-validation documentation explains why checking statements, dependencies, and the verification setup matters.

That distinction changes the human workload. More of the search and checking can be automated, while deciding what to prove, explaining the result, and understanding where its ideas apply remain substantial tasks. A machine-checked result gives us real assurance about a formal claim. Turning it into knowledge that other people can use is another achievement.

Tao’s lighthouse, and the chess lesson it reminded me of

Terence Tao’s October interview with Science Friday helped me articulate this discomfort. Discussing OpenAI’s October 6 release of more than 700 AI-generated mathematical manuscripts, he described a growing backlog of results that people had not yet absorbed. He relayed a friend’s lighthouse analogy: a problem can guide exploration of the surrounding mathematical landscape, rather than serve only as a destination. His concern was about losing the understanding and exchange that normally accompany a solution. Interview and transcript, October 28, 2026.

Tao develops the institutional consequences in Mathematics in the Age of AI. Conditional on increasingly capable AI, he describes a transition from proof scarcity to proof abundance: proposed proofs can outpace verification, checked proofs can outpace readable exposition, and published results can outpace the community’s ability to absorb them. His argument gives explanation, review, and incorporation into mathematical theory a larger role as proof production becomes easier to automate.

This brings the argument back to the model’s capabilities. An AI system can become highly effective at producing proofs that a formal checker accepts. That establishes a real mathematical achievement. It does not, by itself, establish that the system can give a faithful explanation of the proof’s central idea, identify which assumptions matter, or show how the argument changes when those assumptions change.

Chess presents a similar distinction. A strong move, supported by engine analysis, gives evidence about the quality of a decision. A useful explanation must connect that decision to the position: identify the decisive feature, account for the opponent’s strongest reply, and explain why an attractive alternative falls short. As the position changes, the explanation should change for the right reason. In mathematics, the corresponding demand is to connect the proof to the structure of the problem and explain where its reasoning continues to apply.

Neither a persuasive explanation nor a checked result alone settles whether a model understands. But the gap between them matters. A system that repeatedly finds accepted solutions has demonstrated problem-solving competence; broader understanding requires further evidence that its explanations track the reasons those solutions work. This is why I remain skeptical of treating success under a verifier as a complete account of intelligence.

Tao’s concern adds a human consequence to that distinction. Even when correctness can be established, the work of making a result understandable and incorporating it into shared knowledge remains. A personal chess coach has to cross that same gap: its analysis needs to become an explanation the player can examine, challenge, and use. The value of the system depends on both the strength of its decisions and the reliability of that connection.

Trust does not disappear when the answer becomes probabilistic

Chess also helps separate several kinds of confidence.

“White has a forced mate” is a different claim from “this engine strongly prefers White.” Both differ from “White wins 80% of games against a particular population of opponents.” An engine’s principal variation is not proof that every move in it is forced. A percentage needs a defined interpretation and an appropriate basis for calibration.

Under perfect play, a fully specified chess position has a game-theoretic outcome. In practical play, our uncertainty concerns what that outcome is, what the players will find, and whether the analysis has missed something.

So if a system says I have a 50% chance of winning, my first concern is what the number means. Does it exclude draws? Is it an expected score, with a draw counting as half a point? Is it calibrated for players like me under this time control? A smaller percentage does not by itself make the system less trustworthy. An unjustified percentage does.

I can rely on Stockfish’s recommendation even when I cannot calculate the continuation myself. Its demonstrated playing strength gives me a reason to take its analysis seriously. But that confidence has a scope: evidence supporting a move does not automatically support a winning percentage or the explanation a language model attaches to it.

The same distinction applies to AI more broadly. Accepting a checked result, understanding why it works, and knowing when to apply it are separate judgments. Verification can justify the first while leaving the others unresolved. This becomes especially consequential when we move beyond a game whose rules and state can be specified exactly.

Beyond the board, the checker becomes part of the problem

Chess is unusually accommodating. Its rules are explicit. The state can be recorded exactly. A proposed move can be replayed without moving anything in the physical world.

Scientific and physical applications make those conveniences harder to assume.

Aircraft design makes the relationship between prediction and explanation concrete. A standard equation for lift is

\[L = C_L\,\frac{1}{2}\rho V^2 A.\]

Here, $L$ is the lift force, $\rho$ is air density, $V$ is the speed relative to the air, and $A$ is the reference wing area. The factor $\frac{1}{2}\rho V^2$ is the dynamic pressure. The dimensionless lift coefficient $C_L$ captures the influence of wing geometry, angle of attack, and flow conditions. NASA’s explanation of the lift equation.

The compact formula does not make all those dependencies simple. In a wind tunnel, engineers can measure lift and calculate

\[C_L = \frac{L}{\frac{1}{2}\rho V^2 A}.\]

That value summarizes the measured behavior. Its usefulness for prediction depends on applying it under appropriate conditions. NASA specifically explains why viscosity and compressibility effects must be matched when transferring a coefficient between cases. A value measured for one flow regime cannot simply be carried into another. NASA’s explanation of the lift coefficient.

A numerical example makes the changing coefficient easier to picture. Consider a hypothetical light airplane in steady, level flight, with weight $10{,}000\,\mathrm{N}$, wing area $16\,\mathrm{m^2}$, and air density $1.225\,\mathrm{kg/m^3}$. Lift must approximately balance weight. Substituting these values gives the following required lift coefficients:

Flight condition Airspeed Required $C_L$
Faster level flight 60 m/s (216 km/h) 0.28
Moderate-speed level flight 40 m/s (144 km/h) 0.64
Slow level flight 30 m/s (108 km/h) 1.13

These are calculated illustrations, not measured specifications for a particular airplane. At half the speed, the same airplane needs four times the lift coefficient to support the same weight. It must change its aerodynamic state to achieve that coefficient; the number is not a freely adjustable software setting. Whether the wing can supply the required value depends on its lift curve and operating limits. Increasing angle of attack raises lift only up to the stall region, while deploying flaps changes the wing’s lift characteristics and can raise its maximum lift coefficient. NASA on angle of attack and stall and the FAA’s discussion of flap effects.

We understand the physical basis of lift, and coefficients can be derived mathematically for some simplified cases. For complex flows, however, determining their values generally requires experiments, often alongside numerical modeling. A coefficient can be reliable within a tested range even when there is no simple analytical account of all the interactions determining its precise value. That is a meaningful gap between having a useful prediction and having a complete explanation.

Those measured relationships can be encoded in aerodynamic models and lookup tables. During flight, sensor readings change, and software can evaluate the stored model at the current conditions. Conventional gain-scheduled controllers also select or interpolate precomputed control gains—the settings that determine how strongly the controller responds. The numerical values used during flight therefore change even though the underlying model and scheduling rules remain fixed. This is feedback control, without necessarily involving machine learning. NASA’s discussion of gain scheduling, Section 4.2.

Adaptive control goes further by estimating model parameters or adjusting control laws from measured behavior. Neural-network flight-control research has explored learning during flight as well. Those are different architectures; changing a value through a lookup table is not the same as learning a new relationship from experience. NASA’s review of adaptive systems, Sections 4.3 and A.1.3.

In each case, the software helps control the aircraft’s physical state. The wing and airflow produce the forces. The coefficient describes that behavior; it does not create lift by being programmed into a computer.

The same distinction matters when AI proposes a coefficient or a design. Passing a simulation establishes agreement with that simulation’s assumptions. Physical tests can supply further evidence. Neither automatically establishes that the value will remain valid in a new regime, or that the system can explain which assumptions make it work. Trust depends on the scope of the evidence, especially when the proposed result will be used outside the conditions in which it was checked.

The same concern shapes my view of robotics. I see value in a system imagining possible futures before acting. But if its imagined world omits a relevant constraint, repeated success inside that world can reinforce the omission. Physical interaction introduces consequences that a resettable chess position does not have.

This is why I remain cautious about transferring impressive chat or simulation performance into expectations of dependable physical autonomy. The difficulty includes deciding what to measure, noticing when the environment has changed, and recovering when the system’s assumptions fail. Some mistakes leave little room for another attempt.

Intelligence, understanding, and trust

Chess makes a gap visible that is easy to overlook when a system’s results are impressive. Choosing a strong move, giving a faithful explanation, and helping another person understand it are related achievements, but progress in one does not guarantee progress in the others. The same distinction matters when the output is a mathematical proof, a scientific prediction, or an action in the physical world.

I remain optimistic about the capabilities we can develop. Better expert data and training may give us dependable grandmaster-level chess models. Systems that combine language models with reliable checkers can also automate work that once demanded substantial human effort. These are meaningful advances, even while their limitations remain visible.

What I am not ready to equate with a complete account of intelligence is the repeated production of answers that pass a test. For me, understanding also involves recognizing the assumptions behind a result, knowing when they stop applying, and revising an explanation when the situation changes. Success at verification gives us evidence about a claim; it does not settle all of those questions about the system that produced it.

Language keeps the question open. Much of human knowledge becomes available to others through explanations, arguments, and conversation. A model that learns to reason through language may develop abilities that deserve to be called understanding. Equally, convincing language can conceal a weak connection to the world it describes. The difference has to emerge in how the system handles objections, unfamiliar cases, and its own mistakes.

As AI produces more answers, the work of deciding what those answers establish becomes more consequential. We will need both reliable ways to check results and explanations that make their assumptions and limits available for scrutiny. The challenge is to turn increasingly capable problem solving into knowledge that people can understand, question, and use.