TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games

We ran 16 LLMs, covering leading frontier and open-weight models. Every model attempted all 47 puzzles in the game, plus eight community tournament puzzles, in one scored attempt each (pass@1), with 10 to 45 minutes per puzzle, depending on difficulty, to find a working program and keep improving it.

Every result here is a program we can run and test on fresh inputs, which makes a programming game a useful benchmark. Models get what a player can inspect, through text tools instead of a mouse and screenshots: the manual, visible test inputs, registers, and game feedback. They don’t get community code or exact records, only the game’s coarse score histograms.

Undecimal Twin RounderSPEC44149538, one of the eight tournament puzzles, rounds an input to the nearest multiple of N on one output and of 11−N on the other. When we froze the leaderboard, the best community program took 2,694 cycles. Opus 5.5 passed its first program after 6.5 minutes, at 5,669 cycles. It went under the record 16 minutes in and finished at 1,918, which is 29% faster. That speed cost one more node and 28 more instructions than the old record.

The core of the program is a two-speed search. Instead of subtracting N one step at a time, nodes 5 and 6 subtract 4N until they overshoot, then add N back until the value turns positive. Each drives a small coprocessor, node 2 or node 7, by sending it jump offsets through a port, so the coprocessor works as a lookup table for 4N and N.

Five models got under the old record in-game. The other two cycle records are GPT-6.1 Sol’s 3,460 on Intersection IdentifierSPEC64160260, against 3,872 (11% faster), and Gemini 4 Argon’s 188 on Error: 8SPEC79324720, against 198 (5% faster). All three are on tournament puzzles, none in the main game. Of the 133 new frontier points, 18 beat a community record outright. The other 115 are new tradeoffs: better balances between the three scores than any community solution.

GPT-6.1 Sol reaches 75.6 record proximity at an estimated $0.75 per puzzle. Opus 5.5 reaches 77.2 at $5.08, 7 times the cost for 2 more points. Fable 5.1 is the most expensive at $7.19 and scores 70.2.

We grouped all 55 puzzles by their main demand, then read the programs. Fifteen of the sixteen models solved all five routing puzzles, and none found a new frontier point there. The examples below show how different designs spend the same resources.

The main benchmark hides the community’s scores. In a separate study on six puzzles, five models got:

• The target: exact community scores, showing which tradeoffs remain unbeaten.
• Diagnostics: cycle-by-cycle traces and profiles of where each node stalls.
• Edge cases: a fast simulator that runs 10,000 random tests and returns failures.
• More time: 30 minutes, plus extensions for progress, up to 60.

Every model improved: GPT-6 Astra from 82.8 to 91.8, GPT-6.1 Sol from 82.9 to 90.6, Opus 5.5 from 81.4 to 89.1, Grok 4.7 from 67.5 to 75.1, and Gemini 4 Argon from 83.5 to 87.9 in mean record proximity. Scores rose in 28 of the 30 model–puzzle pairs.

The diagnostics changed what models did next. On Signal Multiplexer22280, GPT-6.1 Sol’s first passing program took 294 cycles. Its profile showed node 2 working 86% of the time while nodes 1 and 3 spent 84% blocked waiting to write to it. GPT-6.1 Sol moved the branching out of node 2 and into those two neighbors. The next run took 204 cycles, 31% faster, at a cost of two more nodes and five more instructions.

Information, tools, and time all changed together, so this study can’t say which addition helped most. These runs are outside the main leaderboard and the headline count.

We submitted selected programs to the leaderboard, and the community improved on them almost immediately. On Undecimal Twin RounderSPEC44149538, @LongingForRest took Opus 5.5’s 1,918-cycle program and cut it to 1,744 by the next day. The record is now credited to both: LongingForRest/opus-5-5.

Neither got there alone. The models found new tradeoffs quickly, and players read those programs, worked out why they worked, and took them further.

Mathematics is seeing the same shift. Terence Tao and colleagues used LLM-guided search across 67 problems and improved on the best known results in several. In “After Math,” Silvia De Toffoli and Eamon Duede argue that a result only becomes a solution once mathematicians can understand it, and that AI fits better as an assistant than a competitor. TIS-100 is a small, measurable version of that future: a model finds a better program, and people turn it into something they understand.

ISA-Bench studies reasoning with constrained instruction sets, including TIS-100. Opus Magnum Bench tests agents’ ability to build and optimize machines in another Zachtronics game. We focus on optimization across all TIS-100 puzzles and eight community puzzles, keeping tradeoffs across cycles, nodes, and instructions. The tasks and protocols differ, so scores aren’t directly comparable.

Thanks to Zachtronics, the TIS-100 players, and the maintainers of the community leaderboard, TIS-100-CXX, and leaderboard bot. Their solutions and tools make this comparison possible. Thanks also to @12345ieee, @LongingForRest, @Hersmunch, @easone, @18111398, and everyone on the Zachtronics Discord who discussed the submissions and kept improving them.

TIS-100, SpaceChem, and Opus Magnum are trademarks of Zachtronics. Model names and logos are trademarks of their respective owners and appear only to identify the models tested. Wincon Research is not affiliated with or endorsed by Zachtronics or any model provider.