77 lines
4.0 KiB
Markdown
77 lines
4.0 KiB
Markdown
# 1.0.0-beta.14 Release Review
|
|
|
|
Status: ready for publication after controller release gate.
|
|
|
|
## Verdict
|
|
|
|
No blocking findings found for the benchmark suite catalog and metadata gate.
|
|
|
|
## Findings
|
|
|
|
No blocking findings.
|
|
|
|
Non-blocking note: `compiler/tests/benchmark_suite_catalog_beta14.rs` verifies
|
|
byte-stable JSON, the current 10 benchmark names/directories, top-level counts,
|
|
required scaffold-file status, missing-file lists, checksum metadata presence,
|
|
timing mode strings, and implementation slot names. The current runner output
|
|
also includes `run_args`, `source_stem`, and per-implementation source paths.
|
|
If those fields become contractual in a later release, the focused gate should
|
|
parse the JSON and assert them per benchmark.
|
|
|
|
## Scope Checked
|
|
|
|
- Public release docs in scope: `README.md`, `benchmarks/README.md`,
|
|
`docs/POST_BETA_ROADMAP.md`, `docs/language/SPEC-v1.md`,
|
|
`docs/language/ROADMAP.md`, `docs/language/RELEASE_NOTES.md`,
|
|
`docs/compiler/ROADMAP.md`, `docs/compiler/RELEASE_NOTES.md`, and
|
|
`.llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md`.
|
|
- Tooling and tests in scope: `benchmarks/runner.py`,
|
|
`compiler/tests/benchmark_suite_catalog_beta14.rs`,
|
|
`compiler/tests/benchmark_math_loop_scaffold.rs`,
|
|
`scripts/release-gate.sh`, `compiler/Cargo.toml`, and
|
|
`compiler/Cargo.lock`.
|
|
- Release-facing current-stage wording points at `1.0.0-beta.14`. Remaining
|
|
beta13 references are historical beta13 diagnostics/release notes, the
|
|
existing `diagnostics_schema_beta13` gate, or older review/contract files.
|
|
- Benchmark docs and runner output consistently keep timing local-machine-only,
|
|
do not publish timing numbers, do not define performance thresholds, and do
|
|
not claim a stable benchmark JSON schema.
|
|
- The top-level catalog documents the current 10 suites and all suite-list
|
|
commands. Relative links to `benchmarks/README.md` resolve from the files
|
|
that introduce them.
|
|
- The runner preserves per-suite `run.py --list --json` behavior and adds root
|
|
`benchmarks/runner.py --suite-list` and `--suite-list --json` metadata.
|
|
- `scripts/release-gate.sh` now runs `cargo test --test
|
|
benchmark_suite_catalog_beta14` before the full compiler test suite.
|
|
- Cargo package version and lockfile version both read `1.0.0-beta.14`.
|
|
- Focused private/local publication text scan found no machine-local paths,
|
|
private checkout/user names, private remotes, or local IP text in the reviewed
|
|
release surface.
|
|
|
|
## Verification Commands And Results
|
|
|
|
- `python3 benchmarks/runner.py --suite-list --json`: passed. Output reported
|
|
`benchmark_count: 10`, `implementation_slots: 60`, status `ok`, cold/hot
|
|
timing modes, required scaffold-file status with no missing files, checksum
|
|
metadata, runtime args, implementation source paths, and the local-only
|
|
timing disclaimer.
|
|
- `python3 benchmarks/runner.py --suite-list`: passed.
|
|
- `python3 benchmarks/math-loop/run.py --list --json`: passed, confirming the
|
|
existing per-benchmark list mode still works through the shared runner.
|
|
- `python3 benchmarks/math-loop/run.py --suite-list --json`: passed, confirming
|
|
suite listing also resolves correctly through a per-benchmark wrapper.
|
|
- `cargo test --test benchmark_suite_catalog_beta14`: passed, 1 test.
|
|
- `cargo test --test benchmark_math_loop_scaffold`: passed, 1 test.
|
|
- `git diff --check`: passed.
|
|
- `git diff --check -- README.md benchmarks/README.md docs/POST_BETA_ROADMAP.md docs/language/SPEC-v1.md docs/language/ROADMAP.md docs/language/RELEASE_NOTES.md .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md`:
|
|
passed.
|
|
- `bash -n scripts/release-gate.sh`: passed.
|
|
- `cargo fmt --check`: passed.
|
|
- Focused `rg` stale-current-stage scan for beta13 current release/stage wording
|
|
in the touched release docs produced no matches.
|
|
- Focused `rg` private/local publication text scan over README, docs,
|
|
benchmarks, `.llm`, compiler tests, and scripts produced no matches.
|
|
|
|
The full `./scripts/release-gate.sh` was not run during this review; the
|
|
focused beta14 gate coverage and requested lightweight checks passed.
|