slovo/.llm/reviews/BETA_14_RELEASE_REVIEW.md

1.0.0-beta.14 Release Review

Status: ready for publication after controller release gate.

Verdict

No blocking findings found for the benchmark suite catalog and metadata gate.

Findings

No blocking findings.

Non-blocking note: compiler/tests/benchmark_suite_catalog_beta14.rs verifies byte-stable JSON, the current 10 benchmark names/directories, top-level counts, required scaffold-file status, missing-file lists, checksum metadata presence, timing mode strings, and implementation slot names. The current runner output also includes run_args, source_stem, and per-implementation source paths. If those fields become contractual in a later release, the focused gate should parse the JSON and assert them per benchmark.

Scope Checked

• Public release docs in scope: README.md, benchmarks/README.md, docs/POST_BETA_ROADMAP.md, docs/language/SPEC-v1.md, docs/language/ROADMAP.md, docs/language/RELEASE_NOTES.md, docs/compiler/ROADMAP.md, docs/compiler/RELEASE_NOTES.md, and .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md.
• Tooling and tests in scope: benchmarks/runner.py, compiler/tests/benchmark_suite_catalog_beta14.rs, compiler/tests/benchmark_math_loop_scaffold.rs, scripts/release-gate.sh, compiler/Cargo.toml, and compiler/Cargo.lock.
• Release-facing current-stage wording points at 1.0.0-beta.14. Remaining beta13 references are historical beta13 diagnostics/release notes, the existing diagnostics_schema_beta13 gate, or older review/contract files.
• Benchmark docs and runner output consistently keep timing local-machine-only, do not publish timing numbers, do not define performance thresholds, and do not claim a stable benchmark JSON schema.
• The top-level catalog documents the current 10 suites and all suite-list commands. Relative links to benchmarks/README.md resolve from the files that introduce them.
• The runner preserves per-suite run.py --list --json behavior and adds root benchmarks/runner.py --suite-list and --suite-list --json metadata.
• scripts/release-gate.sh now runs cargo test --test benchmark_suite_catalog_beta14 before the full compiler test suite.
• Cargo package version and lockfile version both read 1.0.0-beta.14.
• Focused private/local publication text scan found no machine-local paths, private checkout/user names, private remotes, or local IP text in the reviewed release surface.

Verification Commands And Results

• python3 benchmarks/runner.py --suite-list --json: passed. Output reported benchmark_count: 10, implementation_slots: 60, status ok, cold/hot timing modes, required scaffold-file status with no missing files, checksum metadata, runtime args, implementation source paths, and the local-only timing disclaimer.
• python3 benchmarks/runner.py --suite-list: passed.
• python3 benchmarks/math-loop/run.py --list --json: passed, confirming the existing per-benchmark list mode still works through the shared runner.
• python3 benchmarks/math-loop/run.py --suite-list --json: passed, confirming suite listing also resolves correctly through a per-benchmark wrapper.
• cargo test --test benchmark_suite_catalog_beta14: passed, 1 test.
• cargo test --test benchmark_math_loop_scaffold: passed, 1 test.
• git diff --check: passed.
• git diff --check -- README.md benchmarks/README.md docs/POST_BETA_ROADMAP.md docs/language/SPEC-v1.md docs/language/ROADMAP.md docs/language/RELEASE_NOTES.md .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md: passed.
• bash -n scripts/release-gate.sh: passed.
• cargo fmt --check: passed.
• Focused rg stale-current-stage scan for beta13 current release/stage wording in the touched release docs produced no matches.
• Focused rg private/local publication text scan over README, docs, benchmarks, .llm, compiler tests, and scripts produced no matches.

The full ./scripts/release-gate.sh was not run during this review; the focused beta14 gate coverage and requested lightweight checks passed.