All 102 planted bugs were caught

Hyper Crux: Round 8 is finished. G2, F6 and Q3 are merged on main, all 102 planted bugs were caught, and both workflows passed on GitHub. The round used about 2 million tokens over about 2¼ hours.

G2 is the first working version.

  • What works now: the Beta’s Go package now runs Scan, Drop, Link, Unlink, Neighbours, Walk and Nearest without a filter through the file, on top of last round’s calls.
  • The shared test suite: 15 of its 25 tests pass. The other 10 wait for SQL (G4), or for the package’s reads to follow other processes (F9).
  • Against 0.x: the differential test agreed with 0.x on 84,000 random sequences. There was one difference, and it’s on purpose: after a failed Put, a later Put that names the same field in other capitals is spelt its own way in the Beta, where 0.x keeps the failed Put’s spelling.

The timed run after G2, made with nothing else running:

Benchmark 0.x Beta Target
Get 11.4 µs 774 ns 3 µs ✓
Walk, one link out 33.9 µs 1.49 µs 3 µs ✓
Walk, three links out 403 µs 42.2 µs 40 µs, 5 per cent over
Nearest, 100,000 vectors of 384 values 261 ms 33.7 ms 65 ms ✓
Nearest, 10,000 vectors of 1,536 values 87.5 ms 8.45 ms 26 ms ✓
Put one record, committed 235 µs 188 µs no slower than 0.x ✓
Put with a vector, 1,000 per transaction 34.3 µs 13.6 µs 5 µs ✗
Open 100,000 records with vectors 431 µs 535 ms 250 ms ✗

The plan says I have to flag it if the first working version misses a speed target by more than double. It misses two:

  • Batched puts take 13.6 µs against 5 µs, 2.7 times the target.
  • Opening takes 535 ms against 250 ms, 2.1 times the target. That’s from a log of 20 commits, since nothing can compact the file yet.

My suggestion is to carry on as planned:

  • F7 already compares put speed with 0.x, so it can also work out where a batched put’s time goes.
  • Opening gets measured again on a compacted file once F8 is in. I1 then decides whether the optional vector-block task (V2) runs.

The other condition the plan flags, file tasks running past one and a half times their planned hours, hasn’t happened. Each has taken well under its planned hours.

The other two tasks:

  • F6: the log can now follow commits made by other processes. The public package’s reads start using this with F9.
  • Q3, the SQL parser: all 3,911 cases in the SQL test corpus parse, or are refused, exactly as marked. On 20,000 random expressions, its parse trees gave 0.x’s answers. A 12-minute fuzz run found nothing.

While the round ran, another session pushed a RELEASING.md and a CLAUDE.md to main, both about the automated releases. I merged them in. Nothing in this round raised the version, so nothing was released.

84 of the 133 hours are done. The next round’s briefs are in the repo:

  • F8: compaction, in the log.
  • Q1: SQL values and functions, which can now be tested through the parser.
  • G3: the database/sql driver.

F7 and F9 also work in the log, so they follow in the round after.