Other claims
What else a certificate can prove
A certificate records one plain fact: this program, on questions fixed before anyone saw it, gave these answers at this cost. A speedup is just one thing you can read off it. These are others.
Exact cost, not seconds
Instead of timing a program, count the Python instructions it runs. For a program that behaves the same way every time, anyone with the same Python version gets exactly the same number, so a re-run proves the recorded cost without trusting anyone's computer, clock or sealed box.
What it does not show: It is not a time. Work done inside one built-in function (sorting, big-number arithmetic) counts as a single instruction, so it only means something for plain Python on both sides. The solver must be written in a restricted, pure-computation subset of Python.
guldo practice my_solver.py --family factor --count --dims 38 40 42Rounds that count
A contest can announce, before anyone submits, that every solver will be counted on the same question sizes. The sizes, the cap, the Python version and the counting program are written into the round's first log entry. Every source is public, so anyone can re-run any count.
What it does not show: It is called COUNTED, not official: official means timed inside the verified locked box. Today it is for the factoring problem only, and the operator's machine still runs the submitted code, so it needs a throwaway user or machine.
guldo ops round-new --count --count-dims 38 40 42 ...A benchmark score
How many of the questions fixed in advance did a program answer correctly, with an uncertainty range, for both the submitted solver and the baseline. The questions come from secret numbers fingerprinted before the program was seen, so nobody could have trained on them.
What it does not show: It is a rate on this question set, under the time limit, for this kind of puzzle. It says nothing about other questions. With seconds as the cost, 'within the limit' depends on the measuring computer; an exact-count certificate removes that.
guldo claims correctness x.cert.jsonA verified negative result
'Nobody in this contest reached a speedup of T bits.' The claim holds only if there is a certificate for every submission the round's log accepted and the log shows the round was closed, so nobody can hide the attempt that worked.
What it does not show: It covers the programs actually submitted, not every possible approach. The credit is deliberately cautious, so 'did not reach T' means 'T was not shown', not 'T is impossible'. A submission whose evaluation failed makes the claim incomplete.
guldo claims negative a.cert.json b.cert.json --threshold 2The worst case seen
For an exact-count certificate: the largest number of instructions seen at each question size, and the chance that one more question from the same source would need more (at most 1 in n+1).
What it does not show: It is the largest value seen, not a guarantee for every possible input of that size. A size where any question timed out or failed has no bound at all.
guldo claims worst-case x.cert.jsonHow these stay honest
- Each claim is recomputed from a certificate that has first been checked: nothing is read off a file that does not verify.
- A claim file is not trusted either. guldo claims check CLAIM.json CERT.json recomputes it and demands the same answer.
- Every claim prints what it does not show, every time.
The guide on this site explains how to check a certificate; the Limits page lists what checking cannot prove.