Tree-Ising Lifts Qwen Math 29pp at 350x Lower Cost

Two researchers have published a method that repairs a small language model's math reasoning for ninety cents and 31 minutes of compute time. The Tree-Ising technique, released November 11, 2026 by Hadi Salloum at QDeep in Riyadh and Ammar Ali at ITMO University's LEAD Lab, recasts weight repair as a sparse discrete optimization problem solved with Ising and QUBO solvers instead of agent-driven edits. On Qwen3.5-0.8B, the method lifts MATH-500@4 accuracy from 7.9% to 55.1% and GSM8K@1 from 13.8% to 70.8%, outperforming the prior GPT-6 Astra agent baseline by 2.9 percentage points while cutting cost by 350x and runtime by 30x.

Release date and availability

The paper was officially released on November 11, 2026 via Hugging Face. It compares Tree-Ising directly against an earlier repair of the same model driven by the GPT-6 Astra agent, which required 158 agent turns, 89.7 million tokens, 50 forward-pass requests, 15.6 hours, and $312 in compute cost. Tree-Ising completed the same repair task in 31 minutes using four forward passes and $0.90 in compute, no LLM token generation required.

How Tree-Ising works

The method decomposes two targeted MLP layers into row-level QUBO and Ising optimization problems, solving each row independently. Moves are restricted to sign flips and exponent shifts inside BF16 weight groups, preserving shared exponents and mantissas. The authors frame this as sparse discrete optimization over BF16 weights rather than continuous gradient descent or agent-proposed edits. Two solver variants were tested: simulated annealing reached 54.6% on MATH-500@4, while simulated bifurcation hit 55.1%, both exceeding the agent's 51.7%. The authors report the improvement is statistically significant at p≈0.003.

Transfer to larger models

Transferred to Qwen3.5-2B without any reference edit or retraining, the same recipe automatically selected layers 15 and 19 and lifted GSM8K@1 accuracy from 21.6% to 69.4%. The repair modified 292,845 weights, representing 1.2% of the targeted matrices. Of 954 test problems, 669 were solved only by the edited model, 38 only by the base model, and 247 by both. This zero-shot transfer demonstrates that the optimization procedure generalizes across model scales without additional tuning.

Benchmarks and validation

Qwen3.5-0.8B's baseline MATH-500@4 score was 7.9%. The GPT-6 Astra agent pushed it to 51.7%. Tree-Ising's simulated-bifurcation variant reached 55.1%, a gain of 29 percentage points over baseline and 2.9 points over the agent. On GSM8K@1, the base model scored 13.8%, and Tree-Ising lifted it to 70.8%. The paper acknowledges that 100 of the 500 MATH-500 problems were seen during development, the recipe was formulated retrospectively from agent logs, and the quantum demonstration was limited to a single 8-variable subproblem on IBM Heron hardware with no downstream accuracy measurement.

What this means

Tree-Ising positions weight-space repair as a discrete optimization task that sidesteps token-intensive agent loops. The 350x cost reduction and 30x speedup over agent-driven methods suggest that combinatorial solvers may offer a practical path to low-budget post-training fixes for small models. The zero-shot transfer to Qwen3.5-2B hints at broader applicability, though the method's reliance on retrospective recipe design and partial test-set visibility during development will require independent replication. For labs targeting cost-efficient model repair at small scale, the approach opens a non-gradient, non-agent alternative worth testing.