VerdictOpen the solver

Verified math and code

Right at math and code.
Proven, not promised.

Verdict adapts a model to be correct at math and code, then checks every answer with sympy and real unit tests before you see it.

Run the solver →See the method
verdict · live proof
mathcalculus
solve∫₀¹ 3x² dx

The problem

Sounding right is not being right.

Generic models answer math and code in a confident, fluent voice. On hard problems that voice often hides a wrong answer. The fix is not a better tone. It is a check.

generic model✗ unchecked

Sum the divisors of 28.

The divisors of 28 are 1, 2, 4, 7, 14, 28, which sum to 54. (stated with full confidence, and quietly off by 2)

Fluent, formatted, and wrong. Nothing ran.

verdict✓ verified

Sum the divisors of 28.

divisors 1, 2, 4, 7, 14, 28 answer 56 sympy divisor_sigma(28) == 56 → True ✓

Same question. Answer matched by sympy, then kept.

The thesis

Math and code are checkable. So the improvement is provable.

Every answer is run through a check before it counts. Two checks carry the whole idea.

sympy checks the math

✓ verified

The proposed answer is compared to the reference with a symbolic engine, so an algebraically equal form still counts as correct.

from sympy import simplify, Eq # model answer vs reference simplify(answer - reference) == 0 # → True ✓ kept

real tests run the code

✓ tests pass

The proposed program is executed against a real unit test suite. If a test fails, the solution is dropped. Only passers are kept.

$ pytest solution_test.py -q .. 2 passed in 0.02s # → all green ✓ kept

Verified, not vouched for

13,260 rows passed a real check.

The cards below are illustrative. The counts are not. I checked every math answer against an independent gold standard three ways, ran every code snippet against real unit tests, and labelled the rows I did not check instead of counting them in.

0rows published
0independently verified, 78.0 percent
0named sources, on every row
  • calculus✓ verified

    d/dx [ x sin x ]

    sin x + x cos x

    sympy diff == diff → True
  • number theory✓ verified

    gcd(252, 198)

    18

    sympy gcd(252,198) == 18 → True
  • arrays✓ verified

    max_subarray(nums)

    Kadane, O(n)

    pytest 14 / 14 passed
  • strings✓ verified

    group_anagrams(words)

    sorted key buckets

    pytest 11 / 11 passed
  • algebra✓ verified

    Σ k for k = 1..100

    5050

    sympy summation == 5050 → True
  • graphs✓ verified

    shortest_path(g, s, t)

    BFS on unweighted

    pytest 8 / 8 passed

How it works

From verified data to a number you can check.

  1. 01

    Build the verified data

    17,000 rows from 7 named public corpora. 9,104 math rows checked against an independent gold standard, 4,156 code rows run against unit tests, and 3,740 general rows I did not check and did not pretend to.

  2. 02

    Adapt on Adaption

    Adaptive Data by Adaption ingested 17,586 rows from the corpus and ran the adaptation. The platform scored the data 8.0 before the pass and 8.2 after, grade B either side.

  3. 03

    Measure it head to head

    AutoScientist scored the adapted model against the base, head to head. Adapted 46. Base 54. It went backwards, and the run state says exactly why.

  4. 04

    Ship all of it open

    Dataset, build scripts, weights, and the result that lost. All of it on Hugging Face and Kaggle. Four commands on the open page settle every number here.

Read the full method →

Open

The dataset, the scripts and the weights are all published.

17,000 rows on Hugging Face and Kaggle. The nine scripts that built them. The trained adapter. And the result, which went backwards, with the reason.

CC-BY-SA-4.0Hugging FaceKaggle
See the open release →

Try it

Stop trusting the confident voice. Get the checked verdict.

Open the solver →