Guldo

Results

What has been measured

Every number here comes from saved timings you can download and recompute. Each run below shows the data behind its number: the difficulty groups, every question, and a table of the same figures.

Leaderboard

SubmissionFamilyCredit (bits)StatusPer seed
factor_trial_npfactor4.1Provisional
PROVISIONAL (0/3 official seeds)
4.1
Not official because
  • not run in the sandbox
tuned_bkzgm0.1Provisional
PROVISIONAL (0/3 official seeds)
0.1, 2.6
Not official because
  • baseline file hash not recorded
  • budget 60.0 < 120s
  • n=12 < 24 instances/dim
  • no CPU model recorded
  • not run in the sandbox
  • seed commitment not checked against a published value
  • solver file hash not recorded

The credit on the board is the lowest over a submission's seeds, so one weak or small run pulls it down. A row that reads lower than the pre-registered result below it usually includes an early design run (12 questions per size, 60 s limit, the data the credit rule was built from). That is why each row lists its per-seed values and the reasons it is not official.

Pre-registered test, seed 755a57fd

The pass criteria were written before any data from this seed existed and are applied here without interpretation.

Primary claim
2.58 bits: outcome (a) credit >= 1.0 bit, which SUPPORTS the claim that this ladder size can certify a pruning-sized speedup.
Exploratory hard-pool claim
lower bound 2.94 bits: first out-of-sample support for promoting it (lower bound >= 1.0 bit).
Concurrency check
not recorded on this page; see the remeasure output
The registration, verbatim
PRE-REGISTRATION, seed D (supersedes the VOID seed-B and seed-C registrations). Written BEFORE any seed-D data exists.
date: 2026-11-18T20:34:47Z
seed commitment (sha256): 755a57fd4dd7d65fec1c77318096fc5bbc233fafe1336745d535615634b34b47
config: family gm, dims [64, 66, 68, 70, 72], n=24/dim, cap 120 s, --jobs 6 (safe max on this machine: 8), local mode (own solvers only), baseline_bkz vs tuned_bkz, this machine.

PRIMARY estimator (as in the code hashes below): difficulty-stratified credit, JOINT bootstrap re-selecting bins every resample;
bins = octaves of baseline cost from 0.5 s; usable = >=5 instances and submission median cost >=0.5 s; top 3 hardest usable bins;
baseline-timeout instances excluded; one-sided 95% lower bound.
UNCONDITIONAL criterion for the primary estimator:
  (a) credit >= 1.0 bit                   -> supports the claim that this ladder size can certify a pruning-sized speedup
  (b) 0 <= credit < 1.0, conclusive       -> FAILS
  (c) inconclusive (< 2 usable bins)      -> FAILS (insufficient power for gm 24/dim at a 120 s cap)

EXPLORATORY estimator (reported beside, never instead): hard-pool claim, baseline cost >= 4 s, n >= 12, submission median >= 0.5 s.
  Its 4 s threshold was chosen AFTER the author saw seed A's bin table (in-sample there). A hard-pool lower bound >= 1.0 bit on seed D is
  the first out-of-sample support for promoting it; below 1.0 or inconclusive means it is NOT promoted.

CONCURRENCY check (after the ladders finish): remeasure.py re-runs 4 instances per cost octave of the BASELINE ladder sequentially.
  Criterion: overall median |shift| <= 0.10 bits AND no octave with n>=3 beyond +-0.20 bits; otherwise the concurrency bias is flagged
  and the seed-D credit must not be called clean.

Known before the run (so they are not later presented as findings): on seed A (12/dim, 60 s cap) the primary estimator gave 0.1 bits
(median speedup 2.2) with ~0% simulated power; the exploratory claim gave 2.3 (in-sample). The primary rule's design (bins, top-3, floor)
was influenced by seed A; the joint-bootstrap correction came from simulation. Production-code simulations at n=120: lower bound exceeds truth
2.5-5% of the time; 63% power at a true 1.5 bits, 100% at 2.5. Real runs may have fewer eligible instances (timeouts, floor).
Bare local mode: a solver can escape cleanup (setsid) and macOS does not kill SIGXCPU-ignorers at the CPU limit; both solvers here are mine and
well-behaved, so neither applies. No Docker is used (the sandbox meter is unverified).

code hashes (sha256):
2feb07a369fcfdb7c08e8fc416f5a846f29c7fa644096d4b12be140c8a82123b  extrapolate.py
5a8393b185fbfaeed37c0208ab2f6fb2f468d262ced14bbd88db1cda56fc6e4b  families.py
ea2a61628f280fa37262b86a9f61fbfebec8164fab43ac6a4f4edc3876929067  instances.py
6da9b5b0b3fc232b3d162ff4d03248964dca1902722c2715aaae7a10c60f1ed6  harness.py
fde3e74fe6a0442d7d63246c2b29052fb9b3fe40bccf2fcb6f707dc79a92068f  runner.py
8d33943015f2c2089f98c738f868451a63fddf1b8663d80293e3b6a731f20a1d  solvers/baseline_bkz.py
4d011945d3f33adf3bc856f977784b5c572fe3cb0d831bb4db2307ef853247ee  solvers/tuned_bkz.py

Positive control, seed 88ecb627

The criteria were written before any data existed and are applied here without interpretation. The true effect was calibrated in-process at about 4.4 bits.

Primary claim
4.12 bits (median 4.24): outcome (a) credit at least 2.5 and median within [3.5, 5.5], which means the pipeline RECOVERS a known large speedup.
Concurrency check
not recorded on this page yet

Does the compiler's prediction match real solvers? (dlog)

Two real solvers were run through the full pipeline on three fresh seeds. What would count as success was written down before any data existed (the registration), and each criterion below is applied exactly as written, with no reinterpretation. Local mode, one computer, no locked box: even a full pass is provisional.

CriterionOutcomeThe numbers
code unchangedNot metchanged since registration: harness.py. Results measured with different code are not the registered experiment
A1: every seed conclusiveMetclaim 1.54 bits (per seed ['1.61', '1.67', '1.54']); pooled paired median 1.61 bits over 201 eligible pairs (all-pairs median 1.60)
A2: claim in [0.7, 1.7] and not above the paired median + 0.3Metclaim 1.54 bits (per seed ['1.61', '1.67', '1.54']); pooled paired median 1.61 bits over 201 eligible pairs (all-pairs median 1.60)
A3: claim >= 1.0 bitMetclaim 1.54 bits (per seed ['1.61', '1.67', '1.54']); pooled paired median 1.61 bits over 201 eligible pairs (all-pairs median 1.60)
B1: not credited at 5 bits or more (the ceiling is real)Metclaim inconclusive bits (per seed ['inconclusive', 'inconclusive', 'inconclusive']); pooled paired median 12.82 bits over 201 eligible pairs (all-pairs median 12.61)
B2: paired median reported beside the claimMetclaim inconclusive bits (per seed ['inconclusive', 'inconclusive', 'inconclusive']); pooled paired median 12.82 bits over 201 eligible pairs (all-pairs median 12.61). The gap between them is the ceiling's size, not a failure

Overall: FAIL.

Runs

dlog_stride4 on dlog, seed 725ff238

Inconclusive

Fewer than two difficulty bins had at least 5 instances and a submission cost of at least 0.5 s, so no credit can be certified from this run.

Instances
9 generated, 4 where the baseline hit the 12 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
inconclusive (needs 12 instances at 4 s or more and a submission median cost of at least 0.5 s)
Speedup by how hard the baseline found each instance
-10+1+2+30.5–1n=12–4n=28–16n=1baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+228< floor29capped30cappeddimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-10+1+2+30.528baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–11+2.110.138not usable
2–42+1.660.931not usable
8–161+1.613.726not usable

dlog_bsgs on dlog, seed d00100a7

Inconclusive

Fewer than two difficulty bins had at least 5 instances and a submission cost of at least 0.5 s, so no credit can be certified from this run.

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
inconclusive (needs 12 instances at 4 s or more and a submission median cost of at least 0.5 s)
Speedup by how hard the baseline found each instance
-20+2+4+6+8+10+12+14+16+180.5–1n=41–2n=92–4n=74–8n=128–16n=916–32n=1232–64n=964–128n=6baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-20+2+4+6+8+10+12+14+1628< floor29< floor30< floordimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-20+2+4+6+8+10+12+14+16+180.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–14+9.680.001not usable
1–29+10.600.001not usable
2–47+11.650.001not usable
4–812+12.400.001not usable
8–169+13.630.001not usable
16–3212+14.370.001not usable
32–649+15.130.001not usable
64–1286+16.010.001not usable

dlog_stride4 on dlog, seed d00100a7

1.7 bits certified (median speedup 1.7; the credit is the one-sided 95% lower bound)

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
at least 1.6 bits (median 1.7, 48 instances)
Speedup by how hard the baseline found each instance
-10+1+2+30.5–1n=41–2n=92–4n=74–8n=128–16n=916–32n=1232–64n=964–128n=6credit 1.7baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+2+328n=2429n=2430n=24dimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-10+1+2+30.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–14+1.580.281not usable
1–29+1.380.567usable
2–47+1.400.987usable
4–812+1.342.241usable
8–169+1.734.058usable
16–3212+1.676.517in the credit
32–649+1.8012.954in the credit
64–1286+1.9719.227in the credit

dlog_bsgs on dlog, seed e5dab2c8

Inconclusive

Fewer than two difficulty bins had at least 5 instances and a submission cost of at least 0.5 s, so no credit can be certified from this run.

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
inconclusive (needs 12 instances at 4 s or more and a submission median cost of at least 0.5 s)
Speedup by how hard the baseline found each instance
-20+2+4+6+8+10+12+14+160.5–1n=61–2n=82–4n=94–8n=128–16n=516–32n=1232–64n=664–128n=6baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-20+2+4+6+8+10+12+14+1628< floor29< floor30< floordimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-20+2+4+6+8+10+12+14+16+180.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–16+9.560.001not usable
1–28+10.440.001not usable
2–49+11.680.001not usable
4–812+12.290.001not usable
8–165+13.500.001not usable
16–3212+14.570.001not usable
32–646+15.110.001not usable
64–1286+14.160.004not usable

dlog_stride4 on dlog, seed e5dab2c8

1.6 bits certified (median speedup 1.8; the credit is the one-sided 95% lower bound)

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
at least 1.6 bits (median 1.7, 41 instances)
Speedup by how hard the baseline found each instance
-10+1+2+30.5–1n=61–2n=82–4n=94–8n=128–16n=516–32n=1232–64n=664–128n=6credit 1.6baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+2+328n=2429n=2430n=24dimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-10+1+2+30.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–16+1.550.25not usable
1–28+1.670.473not usable
2–49+1.411.156usable
4–812+1.431.841usable
8–165+1.752.878usable
16–3212+1.787.004in the credit
32–646+1.8710.745in the credit
64–1286+1.7523.737in the credit

dlog_bsgs on dlog, seed e70be0d8

Inconclusive

Fewer than two difficulty bins had at least 5 instances and a submission cost of at least 0.5 s, so no credit can be certified from this run.

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
inconclusive (needs 12 instances at 4 s or more and a submission median cost of at least 0.5 s)
Speedup by how hard the baseline found each instance
-20+2+4+6+8+10+12+14+16+180.5–1n=31–2n=42–4n=134–8n=158–16n=1116–32n=1032–64n=964–128n=4baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-20+2+4+6+8+10+12+14+1628< floor29< floor30< floordimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-20+2+4+6+8+10+12+14+16+180.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–13+9.740.001not usable
1–24+10.440.001not usable
2–413+11.600.001not usable
4–815+12.390.001not usable
8–1611+13.520.001not usable
16–3210+14.470.001not usable
32–649+15.590.001not usable
64–1284+16.190.001not usable

dlog_stride4 on dlog, seed e70be0d8

1.5 bits certified (median speedup 1.6; the credit is the one-sided 95% lower bound)

Instances
72 generated, 0 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 1 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
at least 1.5 bits (median 1.6, 49 instances)
Speedup by how hard the baseline found each instance
-10+1+2+30.5–1n=31–2n=42–4n=134–8n=158–16n=1116–32n=1032–64n=964–128n=4credit 1.5baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+2+328n=2429n=2430n=24dimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-10+1+2+30.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–13+1.350.288not usable
1–24+1.320.556not usable
2–413+1.331.246usable
4–815+1.501.986usable
8–1611+1.623.367in the credit
16–3210+1.607.336in the credit
32–649+1.5818.072in the credit
64–1284+1.7324.966not usable

factor_trial_np on factor, seed 88ecb627

4.1 bits certified (median speedup 4.2; the credit is the one-sided 95% lower bound)

Instances
96 generated, 7 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 6 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
at least 3.8 bits (median 3.8, 89 instances)
Speedup by how hard the baseline found each instance
-10+1+2+3+4+54–8n=198–16n=2716–32n=1232–64n=1364–128n=18credit 4.1baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+2+3+4+5+658n=2460n=2462n=24647 cappeddimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-10+1+2+3+4+5+60.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
4–819+3.780.472not usable
8–1627+3.690.912usable
16–3212+3.841.812in the credit
32–6413+4.162.399in the credit
64–12818+4.504.013in the credit

tuned_bkz on gm, seed 755a57fd

2.6 bits certified (median speedup 4.1; the credit is the one-sided 95% lower bound)

Instances
120 generated, 41 where the baseline hit the 120 s cap (excluded)
Run
bare local process, 6 concurrent job(s), Apple M4 Pro
Hard-pool claim (exploratory)
at least 2.9 bits (median 4.1, 30 instances)
Speedup by how hard the baseline found each instance
-20+2+4+6+80.5–1n=151–2n=112–4n=104–8n=1116–32n=1332–64n=564–128n=1credit 2.6baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-10+1+2+3+4+5+664n=24664 capped687 capped70capped72cappeddimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-4-20+2+4+6+80.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–115-0.860.951usable
1–211+0.650.698usable
2–410+2.280.663usable
4–811+2.560.819in the credit
16–3213+4.910.815in the credit
32–645+5.041.015in the credit
64–1281+6.461.144not usable

tuned_bkz on gm, seed e78e4a8f

0.1 bits certified (median speedup 2.2; the credit is the one-sided 95% lower bound)

Instances
108 generated, 19 where the baseline hit the 60 s cap (excluded)
Run
bare local process, 1 concurrent job(s), unknown CPU
Hard-pool claim (exploratory)
at least 2.3 bits (median 4.6, 13 instances)
Speedup by how hard the baseline found each instance
-10+1+2+3+4+5+60.5–1n=151–2n=52–4n=84–8n=48–16n=116–32n=732–64n=1credit 0.1baseline cost of the instance (s, octaves)median speedup (bits)

in the credit usable not usablecredit (lower bound)

Speedup by dimension
-2-10+1+2+3+4+5+656< floor58< floor60< floor62< floor642 capped661 capped68capped702 capped72cappeddimensionmedian speedup (bits)

Median over all paired instances, with a 90% bootstrap interval; where the baseline hit its cap its cost is taken as the cap, so those instances can only understate the speedup. Hollow points are dimensions the estimator does not use ("capped": more than half the baseline runs hit the cap; "< floor": costs under 0.5 s): they are not measured, shown only as lower bounds or for reference. Intervals of neighbouring dimensions often overlap, so a rising median is a tendency, not a measured rate. Growth beyond the last measured dimension is not measured either: no line is drawn across a gap, and nothing here is extrapolated.

Every instance the baseline finished
-3-2-10+1+2+3+4+5+60.52832baseline cost of the instance (s, log scale)speedup (bits)
Table view
Baseline cost (s)InstancesMedian bitsSubmission median (s)Role
0.5–115+0.170.53usable
1–25+0.540.997in the credit
2–48+2.210.674in the credit
4–84+1.911.524not usable
8–161+4.570.381not usable
16–327+5.020.744in the credit
32–641+4.871.28not usable

Reading these charts

  • Each bar is the typical (median) speedup in bits for questions grouped by how hard the baseline found them. Each bit is a factor of two.
  • Solid bars are the three hardest groups that count, and they make up the credit. Light bars count but are not among the hardest. Dashed bars have too few questions, or sit below the 0.5 s floor, so they do not count.
  • The dashed line is the credit. It sits below the bars on purpose: it is the cautious number, not the typical one.
  • The "speedup by problem size" chart asks a different question: does the speedup grow as the problem gets harder? It shows the typical speedup at each size with a 90% range. Where the baseline ran out of time, its cost is taken as the time limit, so those questions can only understate the speedup. Hollow points are sizes the credit does not use (too many baseline time-outs, or costs under the 0.5 s floor). They are lower bounds or reference points, not measurements, and nothing is guessed beyond them or joined by a line.
  • A group can show a negative speedup: a tuned solver with a fixed setup cost is slower on trivial questions. That is why the floor exists, and it is shown, not hidden.