From d3e628553f169e29366e8972ccb5340a59e57b29 Mon Sep 17 00:00:00 2001 From: sanjin Date: Fri, 22 May 2026 21:55:36 +0200 Subject: [PATCH] Release 1.0.0-beta.14 benchmark suite catalog --- ...NCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md | 69 +++++++ .llm/reviews/BETA_14_RELEASE_REVIEW.md | 76 ++++++++ README.md | 58 ++++-- benchmarks/README.md | 83 +++++++++ benchmarks/runner.py | 174 ++++++++++++++++++ compiler/Cargo.lock | 2 +- compiler/Cargo.toml | 2 +- .../tests/benchmark_suite_catalog_beta14.rs | 134 ++++++++++++++ docs/POST_BETA_ROADMAP.md | 33 ++++ docs/compiler/RELEASE_NOTES.md | 36 +++- docs/compiler/ROADMAP.md | 13 +- docs/language/RELEASE_NOTES.md | 41 ++++- docs/language/ROADMAP.md | 83 +++++---- docs/language/SPEC-v1.md | 46 ++++- docs/language/STDLIB_API.md | 2 +- scripts/release-gate.sh | 1 + 16 files changed, 790 insertions(+), 63 deletions(-) create mode 100644 .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md create mode 100644 .llm/reviews/BETA_14_RELEASE_REVIEW.md create mode 100644 benchmarks/README.md create mode 100644 compiler/tests/benchmark_suite_catalog_beta14.rs diff --git a/.llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md b/.llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md new file mode 100644 index 0000000..32e6e48 --- /dev/null +++ b/.llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md @@ -0,0 +1,69 @@ +# 1.0.0-beta.14 Benchmark Suite Catalog And Metadata Gate + +Status: release scope for `1.0.0-beta.14`. + +`1.0.0-beta.14` is a docs/tooling metadata slice for the existing benchmark +suite. It keeps the `1.0.0-beta` source language, typed core, runtime, +standard library, API surface, compiler diagnostics, diagnostic output shape, +ABI/layout behavior, and compiler-known runtime names unchanged. + +## Scope + +- Add `benchmarks/README.md` as the top-level benchmark suite catalog. +- Document `python3 benchmarks/runner.py --suite-list` as the non-JSON suite + inventory command. +- Document `python3 benchmarks/runner.py --suite-list --json` as the beta + tooling metadata form for the same inventory. +- Record the current suite inventory: + `math-loop`, `branch-loop`, `parse-loop`, `array-index-loop`, + `string-eq-loop`, `array-struct-field-loop`, + `enum-struct-payload-loop`, `vec-i32-index-loop`, + `vec-string-eq-loop`, and `json-quote-loop`. +- Document that benchmark timings are local-machine evidence only. +- Update README, language roadmap, language spec, release notes, post-beta + roadmap, and this release contract to introduce beta14. + +## Acceptance + +- Public docs name current release/stage `1.0.0-beta.14`. +- `benchmarks/README.md` explains both root suite-list commands: + `python3 benchmarks/runner.py --suite-list` and + `python3 benchmarks/runner.py --suite-list --json`. +- The suite catalog lists the current benchmark suite inventory without adding + or removing benchmark kernels. +- The suite catalog verifies required scaffold files for each suite: + `benchmark.json`, `run.py`, `slovo.toml`, and `src/main.slo`. +- The docs say timing output is local-machine evidence only and do not publish + timing numbers. +- The JSON suite listing is described as beta tooling metadata, not a stable + public schema. +- Explicit exclusions include no new benchmark kernels, no timing publication, + no performance thresholds, no stable JSON schema, no source-language change, + no runtime change, no stdlib/API change, no diagnostic-output change, and no + ABI/layout change. + +## Explicit Non-Scope + +- no new benchmark kernels or implementation language slots +- no benchmark timing publication +- no release performance threshold +- no stable JSON schema or stable benchmark metadata compatibility promise +- no source-language syntax, typed-core, lowering, runtime, stdlib, or API + change +- no compiler diagnostic or diagnostic-output shape change +- no ABI/layout or optimizer guarantee +- no worker-owned release publication work before controller review and gates + +## Expected Controller Verification + +- Run lightweight docs checks: + - `git diff --check -- README.md benchmarks/README.md docs/POST_BETA_ROADMAP.md docs/language/SPEC-v1.md docs/language/ROADMAP.md docs/language/RELEASE_NOTES.md .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md` + - run an `rg` check for stale beta13-only current-stage phrasing across the + same touched docs +- Smoke the metadata commands: + - `python3 benchmarks/runner.py --suite-list` + - `python3 benchmarks/runner.py --suite-list --json` +- Run focused compiler/tooling checks: + - `cargo test --test benchmark_suite_catalog_beta14` + - `cargo test --test benchmark_math_loop_scaffold` +- Do not commit, tag, push, or run release publication from this worker scope. diff --git a/.llm/reviews/BETA_14_RELEASE_REVIEW.md b/.llm/reviews/BETA_14_RELEASE_REVIEW.md new file mode 100644 index 0000000..3850336 --- /dev/null +++ b/.llm/reviews/BETA_14_RELEASE_REVIEW.md @@ -0,0 +1,76 @@ +# 1.0.0-beta.14 Release Review + +Status: ready for publication after controller release gate. + +## Verdict + +No blocking findings found for the benchmark suite catalog and metadata gate. + +## Findings + +No blocking findings. + +Non-blocking note: `compiler/tests/benchmark_suite_catalog_beta14.rs` verifies +byte-stable JSON, the current 10 benchmark names/directories, top-level counts, +required scaffold-file status, missing-file lists, checksum metadata presence, +timing mode strings, and implementation slot names. The current runner output +also includes `run_args`, `source_stem`, and per-implementation source paths. +If those fields become contractual in a later release, the focused gate should +parse the JSON and assert them per benchmark. + +## Scope Checked + +- Public release docs in scope: `README.md`, `benchmarks/README.md`, + `docs/POST_BETA_ROADMAP.md`, `docs/language/SPEC-v1.md`, + `docs/language/ROADMAP.md`, `docs/language/RELEASE_NOTES.md`, + `docs/compiler/ROADMAP.md`, `docs/compiler/RELEASE_NOTES.md`, and + `.llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md`. +- Tooling and tests in scope: `benchmarks/runner.py`, + `compiler/tests/benchmark_suite_catalog_beta14.rs`, + `compiler/tests/benchmark_math_loop_scaffold.rs`, + `scripts/release-gate.sh`, `compiler/Cargo.toml`, and + `compiler/Cargo.lock`. +- Release-facing current-stage wording points at `1.0.0-beta.14`. Remaining + beta13 references are historical beta13 diagnostics/release notes, the + existing `diagnostics_schema_beta13` gate, or older review/contract files. +- Benchmark docs and runner output consistently keep timing local-machine-only, + do not publish timing numbers, do not define performance thresholds, and do + not claim a stable benchmark JSON schema. +- The top-level catalog documents the current 10 suites and all suite-list + commands. Relative links to `benchmarks/README.md` resolve from the files + that introduce them. +- The runner preserves per-suite `run.py --list --json` behavior and adds root + `benchmarks/runner.py --suite-list` and `--suite-list --json` metadata. +- `scripts/release-gate.sh` now runs `cargo test --test + benchmark_suite_catalog_beta14` before the full compiler test suite. +- Cargo package version and lockfile version both read `1.0.0-beta.14`. +- Focused private/local publication text scan found no machine-local paths, + private checkout/user names, private remotes, or local IP text in the reviewed + release surface. + +## Verification Commands And Results + +- `python3 benchmarks/runner.py --suite-list --json`: passed. Output reported + `benchmark_count: 10`, `implementation_slots: 60`, status `ok`, cold/hot + timing modes, required scaffold-file status with no missing files, checksum + metadata, runtime args, implementation source paths, and the local-only + timing disclaimer. +- `python3 benchmarks/runner.py --suite-list`: passed. +- `python3 benchmarks/math-loop/run.py --list --json`: passed, confirming the + existing per-benchmark list mode still works through the shared runner. +- `python3 benchmarks/math-loop/run.py --suite-list --json`: passed, confirming + suite listing also resolves correctly through a per-benchmark wrapper. +- `cargo test --test benchmark_suite_catalog_beta14`: passed, 1 test. +- `cargo test --test benchmark_math_loop_scaffold`: passed, 1 test. +- `git diff --check`: passed. +- `git diff --check -- README.md benchmarks/README.md docs/POST_BETA_ROADMAP.md docs/language/SPEC-v1.md docs/language/ROADMAP.md docs/language/RELEASE_NOTES.md .llm/BETA_14_BENCHMARK_SUITE_CATALOG_AND_METADATA_GATE.md`: + passed. +- `bash -n scripts/release-gate.sh`: passed. +- `cargo fmt --check`: passed. +- Focused `rg` stale-current-stage scan for beta13 current release/stage wording + in the touched release docs produced no matches. +- Focused `rg` private/local publication text scan over README, docs, + benchmarks, `.llm`, compiler tests, and scripts produced no matches. + +The full `./scripts/release-gate.sh` was not run during this review; the +focused beta14 gate coverage and requested lightweight checks passed. diff --git a/README.md b/README.md index f936ac0..a974e4b 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This repository is the canonical public monorepo for the language design, standard library source, compiler, runtime, examples, benchmarks, and technical documents. -Current release: `1.0.0-beta.13`. +Current release: `1.0.0-beta.14`. ## Repository Layout @@ -24,7 +24,7 @@ scripts/ local release and document tooling ## Beta Scope -`1.0.0-beta.13` keeps the `1.0.0-beta` language baseline, includes the +`1.0.0-beta.14` keeps the `1.0.0-beta` language baseline, includes the `1.0.0-beta.1` tooling/install hardening slice, the `1.0.0-beta.2` runtime/resource foundation bundle, the `1.0.0-beta.3` standard-library stabilization bundle, the `1.0.0-beta.4` language-usability diagnostics @@ -35,7 +35,8 @@ alias foundation, the `1.0.0-beta.9` collection alias unification and generic reservation slice, the `1.0.0-beta.10` developer-experience API discovery slice, and the `1.0.0-beta.11` local package API documentation slice, plus the `1.0.0-beta.12` concrete vector query and prefix parity -slice, and the `1.0.0-beta.13` diagnostic catalog and schema policy slice. +slice, the `1.0.0-beta.13` diagnostic catalog and schema policy slice, and +the `1.0.0-beta.14` benchmark suite catalog and metadata gate. The language baseline supports practical local command-line, file, and loopback-network programs with: @@ -73,23 +74,31 @@ S-expression/JSON relationship, required and optional fields, JSON-line discipline, source-less diagnostics, manifest diagnostic metadata, compatibility and migration classes, and the current golden diagnostic code catalog. +The `1.0.0-beta.14` benchmark metadata slice documents the existing benchmark +suite catalog in [`benchmarks/README.md`](benchmarks/README.md). It explains +`python3 benchmarks/runner.py --suite-list` for the human-readable suite +inventory and `python3 benchmarks/runner.py --suite-list --json` for beta +tooling metadata, with required scaffold-file verification for each current +suite. Benchmark timings remain local-machine evidence only; the JSON field +set is not a stable public schema. Still deferred before stable: executable generics, maps/sets, broad package registry semantics, stable Markdown schema, stable stdlib/API compatibility freeze, DNS/TLS/async networking, LSP/watch guarantees, SARIF and daemon protocols, stable `1.0.0` diagnostics freeze, re-exports/globs/hierarchical modules, mutable vectors, slice/view APIs, iterators, new compiler-known -runtime names, stable ABI and layout, performance claims, and runtime changes -for generic collections. +runtime names, stable ABI and layout, performance claims, stable benchmark +JSON metadata schema, and runtime changes for generic collections. -The next likely language slice after `1.0.0-beta.13` should continue from the -developer-experience, package, and reserved generic/collection lanes without -claiming executable generics, maps, sets, traits, inference, monomorphization, -iterators, ABI stability, runtime changes, LSP/watch protocols, SARIF/daemon -protocols, registry semantics, stable Markdown schema, a stable `1.0.0` -diagnostics freeze, standard-library/API compatibility freeze, mutable -vectors, slice/view APIs, new runtime names, or performance claims until the -contract and gates are explicit. +The next likely language slice after `1.0.0-beta.14` should continue from the +developer-experience, package, benchmark metadata, and reserved +generic/collection lanes without claiming executable generics, maps, sets, +traits, inference, monomorphization, iterators, ABI stability, runtime changes, +LSP/watch protocols, SARIF/daemon protocols, registry semantics, stable +Markdown schema, stable benchmark JSON schema, a stable `1.0.0` diagnostics +freeze, standard-library/API compatibility freeze, mutable vectors, slice/view +APIs, new runtime names, or performance claims until the contract and gates +are explicit. ## Build And Test @@ -332,6 +341,28 @@ This release does not change the source language, runtime, stdlib API, diagnostic output shape, compiler CLI, LSP/watch behavior, SARIF/daemon protocols, stable Markdown schema, or stable `1.0.0` diagnostics freeze. +## 1.0.0-beta.14 Benchmark Suite Catalog And Metadata Gate + +The `1.0.0-beta.14` release documents the existing benchmark suite catalog as +beta-scoped metadata tooling. It adds +[`benchmarks/README.md`](benchmarks/README.md) with the current ten-suite +inventory, local evidence policy, suite-list commands, and exclusions. + +The root suite catalog commands are: + +```bash +python3 benchmarks/runner.py --suite-list +python3 benchmarks/runner.py --suite-list --json +``` + +The non-JSON listing is for local review. The JSON listing is beta tooling +metadata for local gates and adapters, not a stable public schema. + +This release does not add benchmark kernels, publish timing numbers, define +performance thresholds, change the source language, runtime, stdlib/API +surface, diagnostic output, compiler ABI/layout behavior, or make cross-machine +performance claims. + ## Documentation - [Language Manifest](docs/language/MANIFEST.md) @@ -339,6 +370,7 @@ protocols, stable Markdown schema, or stable `1.0.0` diagnostics freeze. - [Diagnostics Policy](docs/language/DIAGNOSTICS.md) - [Local Package And Workspace Guide](docs/language/PACKAGES.md) - [Standard Library API Catalog](docs/language/STDLIB_API.md) +- [Benchmark Suite Catalog](benchmarks/README.md) - [Compiler Manifest](docs/compiler/GLAGOL_COMPILER_MANIFEST.md) - [Post-Beta Roadmap](docs/POST_BETA_ROADMAP.md) - [Slovo Whitepaper](docs/papers/SLOVO_WHITEPAPER.md) diff --git a/benchmarks/README.md b/benchmarks/README.md new file mode 100644 index 0000000..874d546 --- /dev/null +++ b/benchmarks/README.md @@ -0,0 +1,83 @@ +# Slovo Benchmark Suite Catalog + +Release stage: `1.0.0-beta.14`. + +The benchmark suite is beta-scoped local tooling. It catalogs deterministic +same-machine comparison scaffolds for Slovo and sibling implementations. It +does not publish benchmark results, set performance thresholds, define optimizer +claims, or create cross-machine comparisons. + +## Suite Listing + +From the repository root, list the suite catalog for humans: + +```bash +python3 benchmarks/runner.py --suite-list +``` + +List the same catalog as beta tooling metadata: + +```bash +python3 benchmarks/runner.py --suite-list --json +``` + +The non-JSON listing is for local review. The JSON listing is for local gates +and tooling adapters that need the current suite inventory. The JSON field set +is not a stable public schema; it may change during beta releases. +Both forms verify the required scaffold files for each suite: +`benchmark.json`, `run.py`, `slovo.toml`, and `src/main.slo`. + +Each suite still owns its local metadata and run commands: + +```bash +python3 benchmarks//run.py --list +python3 benchmarks//run.py --list --json +python3 benchmarks//run.py --dry-run +``` + +## Current Suites + +All current suites provide Slovo, C, Rust, Python, Clojure, and Common +Lisp/SBCL source slots. Missing local toolchains are skipped by the per-suite +runner where possible. + +| Suite | Focus | Base checksum | Hot-loop checksum | Runtime args | +| --- | --- | --- | --- | --- | +| `math-loop` | arithmetic and scalar accumulation | `5000001` | `50000001` | none | +| `branch-loop` | deterministic branch-heavy integer loop | `1185071` | `220775` | none | +| `parse-loop` | repeated signed decimal `i32` parsing | `345000001` | `450000001` | `12345` | +| `array-index-loop` | immutable fixed-array indexing | `3875007` | `38750007` | none | +| `string-eq-loop` | fixed string lookup and equality | `4600001` | `46000001` | `omega` | +| `array-struct-field-loop` | fixed-array access through an immutable struct field | `3875011` | `38750011` | none | +| `enum-struct-payload-loop` | enum payload matching with struct and array access | `3500013` | `35000013` | none | +| `vec-i32-index-loop` | runtime-owned `(vec i32)` indexing | `3875007` | `38750007` | none | +| `vec-string-eq-loop` | runtime-owned `(vec string)` lookup and equality | `4600001` | `46000001` | `omega` | +| `json-quote-loop` | compact JSON string quoting | `15000001` | `150000001` | `slo"vo\path` | + +The base loop count is `1000000` for every current suite. The hot-loop count is +`10000000` for every current suite. The runner supplies loop counts and runtime +arguments at execution time so native compilers cannot fold the work into a +constant answer. + +## Local Evidence Only + +Benchmark output is local-machine evidence only: + +- cold-process mode measures execution after each implementation has been built + once; it does not include compile time +- hot-loop mode is startup-amortized local evidence and reports total time plus + normalized timing for the base loop count +- Clojure timings include JVM and Clojure startup +- Common Lisp timings include SBCL script startup +- reported timings depend on the local CPU, OS, compiler versions, toolchain + availability, thermal/load state, and runner configuration + +This catalog intentionally publishes no timing numbers. + +## Exclusions + +`1.0.0-beta.14` does not add benchmark kernels, publish timings, define +performance thresholds, define a stable JSON schema, change the Slovo source +language, change runtime behavior, change standard-library or API contracts, +change diagnostic output, change ABI/layout behavior, or make cross-machine +performance claims. diff --git a/benchmarks/runner.py b/benchmarks/runner.py index afa74ba..be79c38 100644 --- a/benchmarks/runner.py +++ b/benchmarks/runner.py @@ -18,6 +18,12 @@ from typing import Callable TIMING_SCOPE = "local-machine comparison only" TIMING_MODES = ["cold-process", "hot-loop"] +SUITE_NAME = "glagol-local-benchmark-suite" +LOCAL_TIMING_DISCLAIMER = ( + "Local timing comparison only; not a published benchmark result and not a cross-machine performance claim." +) +REQUIRED_BENCHMARK_FILES = ["benchmark.json", "run.py", "slovo.toml", "src/main.slo"] +EXPECTED_IMPLEMENTATION_NAMES = ["slovo", "c", "rust", "python", "clojure", "common_lisp"] @dataclass(frozen=True) @@ -52,6 +58,14 @@ class Implementation: def main(root: Path, argv: list[str]) -> int: + if any(arg == "--suite-list" for arg in argv): + return main_suite(root, argv) + if not (root / "benchmark.json").is_file(): + parser = argparse.ArgumentParser(description="Shared local Glagol benchmark runner.") + parser.add_argument("--suite-list", action="store_true", help="list suite metadata and exit") + parser.add_argument("--json", action="store_true", help="emit JSON for --suite-list") + parser.error("run from a benchmark run.py, or pass --suite-list at the benchmark suite root") + spec = read_spec(root) implementations = available_implementations(root, spec) parser = argparse.ArgumentParser(description=f"Run local {spec.name} timing comparisons.") @@ -89,6 +103,25 @@ def main(root: Path, argv: list[str]) -> int: return 1 if any(result["status"] == "failed" for result in results) else 0 +def main_suite(root: Path, argv: list[str]) -> int: + parser = argparse.ArgumentParser(description="List local Glagol benchmark suite metadata.") + parser.add_argument("--suite-list", action="store_true", help="list suite metadata and exit") + parser.add_argument("--json", action="store_true", help="emit JSON for suite metadata") + args = parser.parse_args(argv) + if not args.suite_list: + parser.error("pass --suite-list to list benchmark suite metadata") + + suite_root = resolve_suite_root(root) + emit_suite_list(build_suite_catalog(suite_root), args.json) + return 0 + + +def resolve_suite_root(root: Path) -> Path: + if (root / "benchmark.json").is_file(): + return root.parent + return root + + def read_spec(root: Path) -> BenchmarkSpec: data = json.loads((root / "benchmark.json").read_text(encoding="utf-8")) loop_count = int(data["loop_count"]) @@ -278,6 +311,143 @@ def select_implementations(implementations: list[Implementation], names: list[st return [impl for impl in implementations if impl.name in selected_names] +def build_suite_catalog(suite_root: Path) -> dict[str, object]: + benchmarks: list[dict[str, object]] = [] + implementation_slot_count = 0 + missing_required_files: list[str] = [] + missing_implementation_slots: list[str] = [] + + for root in suite_benchmark_roots(suite_root): + spec = read_spec(root) + implementations = available_implementations(root, spec) + implementation_slot_count += len(implementations) + benchmark = suite_benchmark_metadata(suite_root, root, spec, implementations) + benchmarks.append(benchmark) + + directory = str(benchmark["directory"]) + for required_file in benchmark["required_files"]: + assert isinstance(required_file, dict) + if required_file["status"] != "present": + missing_required_files.append(f"{directory}/{required_file['path']}") + + present_implementations = { + str(implementation["name"]) + for implementation in benchmark["implementation_slots"] + if isinstance(implementation, dict) + } + for expected in EXPECTED_IMPLEMENTATION_NAMES: + if expected not in present_implementations: + missing_implementation_slots.append(f"{directory}:{expected}") + + return { + "suite": SUITE_NAME, + "timing_scope": TIMING_SCOPE, + "timing_modes": TIMING_MODES, + "timing_disclaimer": LOCAL_TIMING_DISCLAIMER, + "benchmark_count": len(benchmarks), + "benchmarks": benchmarks, + "verification": { + "status": "ok" if not missing_required_files and not missing_implementation_slots else "incomplete", + "benchmark_metadata_files": len(benchmarks), + "required_files": len(benchmarks) * len(REQUIRED_BENCHMARK_FILES), + "missing_required_files": missing_required_files, + "implementation_slots": implementation_slot_count, + "expected_implementation_slots": len(benchmarks) * len(EXPECTED_IMPLEMENTATION_NAMES), + "missing_implementation_slots": missing_implementation_slots, + }, + } + + +def suite_benchmark_roots(suite_root: Path) -> list[Path]: + return sorted( + [path for path in suite_root.iterdir() if path.is_dir() and (path / "benchmark.json").is_file()], + key=lambda path: path.name, + ) + + +def suite_benchmark_metadata( + suite_root: Path, + root: Path, + spec: BenchmarkSpec, + implementations: list[Implementation], +) -> dict[str, object]: + return { + "name": spec.name, + "directory": str(root.relative_to(suite_root)), + "source_stem": spec.source_stem, + "timing_modes": TIMING_MODES, + "loop_count_source": "stdin", + "loop_count": spec.loop_count, + "hot_loop_count": spec.hot_loop_count, + "expected_checksum": spec.expected_checksum, + "hot_expected_checksum": spec.hot_expected_checksum, + "required_files": [ + { + "path": relative, + "status": "present" if (root / relative).is_file() else "missing", + } + for relative in REQUIRED_BENCHMARK_FILES + ], + "checksum_metadata": { + "cold_process": { + "expected_checksum": spec.expected_checksum, + "stdin": spec.stdin_text, + }, + "hot_loop": { + "expected_checksum": spec.hot_expected_checksum, + "stdin": spec.hot_stdin_text, + }, + }, + "run_args": spec.run_args, + "implementation_slots": [ + { + "name": impl.name, + "language": impl.language, + "source": str(impl.source.relative_to(suite_root)), + } + for impl in implementations + ], + } + + +def emit_suite_list(metadata: dict[str, object], as_json: bool) -> None: + if as_json: + print(json.dumps(metadata, indent=2, sort_keys=True)) + return + + print(f"{metadata['suite']}: {metadata['timing_scope']}") + print(str(metadata["timing_disclaimer"])) + print(f"benchmark_count={metadata['benchmark_count']}") + print(f"timing_modes={','.join(TIMING_MODES)}") + verification = metadata["verification"] + assert isinstance(verification, dict) + print(f"verification_status={verification['status']}") + print(f"required_files={verification['required_files']}") + print(f"implementation_slots={verification['implementation_slots']}") + print("benchmarks:") + for benchmark in metadata["benchmarks"]: + assert isinstance(benchmark, dict) + print( + " {name} ({directory}): loop_count={loop_count} hot_loop_count={hot_loop_count} " + "expected_checksum={expected_checksum} hot_expected_checksum={hot_expected_checksum}".format( + name=benchmark["name"], + directory=benchmark["directory"], + loop_count=benchmark["loop_count"], + hot_loop_count=benchmark["hot_loop_count"], + expected_checksum=benchmark["expected_checksum"], + hot_expected_checksum=benchmark["hot_expected_checksum"], + ) + ) + print(" required_files:") + for required_file in benchmark["required_files"]: + assert isinstance(required_file, dict) + print(f" {required_file['path']}: {required_file['status']}") + print(" implementations:") + for implementation in benchmark["implementation_slots"]: + assert isinstance(implementation, dict) + print(f" {implementation['name']}: {implementation['language']} ({implementation['source']})") + + def emit_list(root: Path, spec: BenchmarkSpec, implementations: list[Implementation], as_json: bool) -> None: metadata = { "benchmark": spec.name, @@ -520,3 +690,7 @@ def shlex_quote(value: str) -> str: if value and all(char.isalnum() or char in "/._:-" for char in value): return value return "'" + value.replace("'", "'\"'\"'") + "'" + + +if __name__ == "__main__": + raise SystemExit(main(Path(__file__).resolve().parent, sys.argv[1:])) diff --git a/compiler/Cargo.lock b/compiler/Cargo.lock index 7117418..15044d4 100644 --- a/compiler/Cargo.lock +++ b/compiler/Cargo.lock @@ -4,4 +4,4 @@ version = 3 [[package]] name = "glagol" -version = "1.0.0-beta.13" +version = "1.0.0-beta.14" diff --git a/compiler/Cargo.toml b/compiler/Cargo.toml index d91c758..3578a33 100644 --- a/compiler/Cargo.toml +++ b/compiler/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "glagol" -version = "1.0.0-beta.13" +version = "1.0.0-beta.14" edition = "2021" description = "Glagol, the first compiler for the Slovo language" license = "MIT OR Apache-2.0" diff --git a/compiler/tests/benchmark_suite_catalog_beta14.rs b/compiler/tests/benchmark_suite_catalog_beta14.rs new file mode 100644 index 0000000..efedbd8 --- /dev/null +++ b/compiler/tests/benchmark_suite_catalog_beta14.rs @@ -0,0 +1,134 @@ +use std::{ + env, + path::Path, + process::{Command, Output}, +}; + +const BENCHMARKS: &[&str] = &[ + "array-index-loop", + "array-struct-field-loop", + "branch-loop", + "enum-struct-payload-loop", + "json-quote-loop", + "math-loop", + "parse-loop", + "string-eq-loop", + "vec-i32-index-loop", + "vec-string-eq-loop", +]; + +const IMPLEMENTATIONS: &[&str] = &["slovo", "c", "rust", "python", "clojure", "common_lisp"]; + +#[test] +fn suite_catalog_is_byte_stable_and_lists_current_benchmarks() { + let repo = Path::new(env!("CARGO_MANIFEST_DIR")).join(".."); + let python = python_command(); + + let first = run_suite_catalog(&repo, &python); + let second = run_suite_catalog(&repo, &python); + + assert_success("first suite catalog run", &first); + assert_success("second suite catalog run", &second); + assert_eq!( + first.stdout, second.stdout, + "suite catalog JSON must be byte-stable across runs" + ); + + let stdout = String::from_utf8_lossy(&first.stdout); + for needle in [ + r#""suite": "glagol-local-benchmark-suite""#, + r#""benchmark_count": 10"#, + r#""benchmark_metadata_files": 10"#, + r#""required_files": 40"#, + r#""missing_required_files": []"#, + r#""implementation_slots": 60"#, + r#""expected_implementation_slots": 60"#, + r#""missing_implementation_slots": []"#, + r#""status": "ok""#, + r#""timing_scope": "local-machine comparison only""#, + r#""timing_disclaimer": "Local timing comparison only; not a published benchmark result and not a cross-machine performance claim.""#, + r#""cold-process""#, + r#""hot-loop""#, + r#""loop_count": 1000000"#, + r#""hot_loop_count": 10000000"#, + r#""checksum_metadata""#, + r#""expected_checksum""#, + r#""hot_expected_checksum""#, + r#""required_files""#, + r#""path": "benchmark.json""#, + r#""path": "run.py""#, + r#""path": "slovo.toml""#, + r#""path": "src/main.slo""#, + r#""status": "present""#, + r#""implementation_slots""#, + r#""loop_count_source": "stdin""#, + ] { + assert_contains(&stdout, needle); + } + + for benchmark in BENCHMARKS { + assert_contains(&stdout, &format!(r#""name": "{}""#, benchmark)); + assert_contains(&stdout, &format!(r#""directory": "{}""#, benchmark)); + } + + for implementation in IMPLEMENTATIONS { + assert_contains(&stdout, &format!(r#""name": "{}""#, implementation)); + } +} + +fn run_suite_catalog(repo: &Path, python: &str) -> Output { + Command::new(python) + .arg("benchmarks/runner.py") + .arg("--suite-list") + .arg("--json") + .current_dir(repo) + .output() + .unwrap_or_else(|err| { + panic!( + "run `{}` benchmarks/runner.py --suite-list --json: {}", + python, err + ) + }) +} + +fn python_command() -> String { + if let Some(python) = env::var_os("PYTHON") { + return python.to_string_lossy().into_owned(); + } + + for candidate in ["python3", "python"] { + if Command::new(candidate) + .arg("--version") + .output() + .map(|output| output.status.success()) + .unwrap_or(false) + { + return candidate.to_string(); + } + } + + panic!("benchmark suite catalog test requires python3 or python") +} + +fn assert_success(context: &str, output: &Output) { + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + + assert!( + output.status.success(), + "{} failed\nstdout:\n{}\nstderr:\n{}", + context, + stdout, + stderr + ); + assert!(stderr.is_empty(), "{} wrote stderr:\n{}", context, stderr); +} + +fn assert_contains(haystack: &str, needle: &str) { + assert!( + haystack.contains(needle), + "suite catalog output missing `{}`\nstdout:\n{}", + needle, + haystack + ); +} diff --git a/docs/POST_BETA_ROADMAP.md b/docs/POST_BETA_ROADMAP.md index cfa2b01..3b4a448 100644 --- a/docs/POST_BETA_ROADMAP.md +++ b/docs/POST_BETA_ROADMAP.md @@ -379,6 +379,38 @@ Why thirteenth: diagnostics already have a machine schema and broad golden coverage, but the compatibility policy and current code inventory need one central reference before future tooling or migration work builds on them. +### 14. Benchmark Suite Catalog And Metadata Gate + +Goal: document the existing benchmark suite inventory and metadata listing +commands before benchmark tooling grows additional consumers. + +Work: + +- add [`benchmarks/README.md`](../benchmarks/README.md) as the top-level + benchmark suite catalog +- document `python3 benchmarks/runner.py --suite-list` for the non-JSON suite + inventory +- document `python3 benchmarks/runner.py --suite-list --json` for beta tooling + metadata +- list the current suite inventory without adding new benchmark kernels +- state that benchmark timings are local-machine evidence only +- keep suite-list JSON beta-scoped rather than a stable public schema +- keep timing publication, performance thresholds, source-language/runtime/ + stdlib/API/diagnostic-output changes, and ABI/layout changes out of scope + +Released in `1.0.0-beta.14`: +[`benchmarks/README.md`](../benchmarks/README.md) now catalogs the current +benchmark suite, documents the root suite-list commands, records local-machine +evidence policy, and names the explicit exclusions. The release is +documentation/tooling metadata only; it does not add kernels, publish timing +numbers, define performance thresholds, define a stable JSON schema, or change +the source language, runtime, stdlib/API surface, diagnostics, or ABI/layout +behavior. + +Why fourteenth: the benchmark suite is already part of the public monorepo, but +its suite-level inventory and metadata boundary need one central reference +before future tooling can rely on it. + ## Stable `1.0.0` Gate Slovo should not become stable until all of these are true: @@ -390,6 +422,7 @@ Slovo should not become stable until all of these are true: - release gates are reproducible on a clean checkout - diagnostics and formatter output are stable for promoted features - performance publications are repeatable and labeled as local-machine evidence +- benchmark metadata promoted to stable, if any, has an explicit schema policy - the language can build useful local CLI tools, libraries, file-processing programs, and basic host-interaction programs without undocumented behavior diff --git a/docs/compiler/RELEASE_NOTES.md b/docs/compiler/RELEASE_NOTES.md index bc18023..3912f1e 100644 --- a/docs/compiler/RELEASE_NOTES.md +++ b/docs/compiler/RELEASE_NOTES.md @@ -10,11 +10,43 @@ integration/readiness release, not the first real beta. ## Unreleased -Next scoped Glagol work is expected to continue after the `1.0.0-beta.13` -diagnostic catalog and schema policy hardening update. +Next scoped Glagol work is expected to continue after the `1.0.0-beta.14` +benchmark suite catalog and metadata gate. No unreleased compiler scope is committed here yet. +## 1.0.0-beta.14 + +Release label: `1.0.0-beta.14` + +Release date: 2026-05-22 + +Release state: benchmark suite catalog and metadata gate + +### Summary + +The beta.14 compiler-side contract is tooling-only benchmark catalog hardening. +It keeps benchmark execution local and unchanged while adding deterministic +suite-level metadata for release verification. + +- Bump the `glagol` compiler package version to `1.0.0-beta.14`. +- Add `python3 benchmarks/runner.py --suite-list --json` for deterministic + suite-level benchmark listing and verification while preserving normal + per-benchmark `run.py` execution through the shared runner. +- Emit all 10 current benchmark names and directories, timing modes, cold/hot + loop counts, checksum metadata, required scaffold-file status, + implementation slots, and the local-only timing disclaimer. +- Add focused `benchmark_suite_catalog_beta14` coverage requiring byte-stable + suite catalog output across two runs and the current 10-benchmark inventory. +- Run the focused beta.14 suite catalog test in `scripts/release-gate.sh` + before the full compiler test suite. + +### Explicit Deferrals + +This release does not implement `glagol bench`, timing-result publication, a +new benchmark kernel, runtime changes, source-language changes, a stable JSON +schema claim, benchmark thresholds, or cross-machine performance claims. + ## 1.0.0-beta.13 Release label: `1.0.0-beta.13` diff --git a/docs/compiler/ROADMAP.md b/docs/compiler/ROADMAP.md index 5662f32..125e993 100644 --- a/docs/compiler/ROADMAP.md +++ b/docs/compiler/ROADMAP.md @@ -22,8 +22,8 @@ general-purpose beta release. A Glagol feature is done only when it has parser/lowerer support, checker behavior, diagnostics for invalid forms, backend behavior or explicit unsupported diagnostics, and tests. -Current stage: `1.0.0-beta.13`, released on 2026-05-22 as a diagnostic -catalog and schema policy hardening update. It keeps the `1.0.0-beta` +Current stage: `1.0.0-beta.14`, released on 2026-05-22 as a benchmark suite +catalog and metadata gate update. It keeps the `1.0.0-beta` language/compiler support baseline and includes the `1.0.0-beta.1` tooling hardening release, the `1.0.0-beta.2` runtime/resource foundation release, the `1.0.0-beta.3` standard-library stabilization release, @@ -55,9 +55,14 @@ source-language runtime behavior. The beta.13 tooling/docs-only diagnostics slice centralizes the `slovo.diagnostic` schema name/version constants and adds structural gates for S-expression diagnostics, `--json-diagnostics`, and artifact-manifest diagnostics metadata without changing emitted machine -diagnostic shape. +diagnostic shape. The beta.14 tooling-only benchmark slice adds deterministic +suite-level local benchmark listing and release-gate coverage for +`python3 benchmarks/runner.py --suite-list --json`, including all current +benchmark directories, timing modes, cold/hot loop counts, checksum metadata, +required scaffold-file status, implementation slots, and a local-only timing +disclaimer without publishing timing results or claiming a stable JSON schema. -Next stage target: post-`1.0.0-beta.13` developer-experience, package, and +Next stage target: post-`1.0.0-beta.14` developer-experience, package, and collection/generic planning. Generic vectors, generic collections, maps, sets, generic stdlib dispatch, runtime collection changes, collection unification, stable human diagnostic text, stable Markdown schema, LSP/watch protocols, diff --git a/docs/language/RELEASE_NOTES.md b/docs/language/RELEASE_NOTES.md index 3ba1267..f8279be 100644 --- a/docs/language/RELEASE_NOTES.md +++ b/docs/language/RELEASE_NOTES.md @@ -8,7 +8,7 @@ Historical `exp-*` releases listed here are experimental maturity milestones. The pushed tag `v2.0.0-beta.1` is historical. It is now documented as an experimental integration/readiness release, not as a beta maturity claim. -The current release is `1.0.0-beta.13`, published on 2026-05-22. It keeps the +The current release is `1.0.0-beta.14`, published on 2026-05-22. It keeps the `1.0.0-beta` language surface, includes the first post-beta tooling/install hardening bundle from `1.0.0-beta.1`, and adds the first runtime/resource foundation bundle from `1.0.0-beta.2` plus the first standard-library @@ -23,12 +23,49 @@ collection alias unification and generic reservation slice from `1.0.0-beta.10`, and the local package API documentation extension from `1.0.0-beta.11`, plus the concrete vector query and prefix parity slice from `1.0.0-beta.12`, and the diagnostic catalog and schema policy slice from -`1.0.0-beta.13`. +`1.0.0-beta.13`, plus the benchmark suite catalog and metadata gate from +`1.0.0-beta.14`. ## Unreleased No unreleased language scope is committed here yet. +## 1.0.0-beta.14 + +Release label: `1.0.0-beta.14` + +Release name: Benchmark Suite Catalog And Metadata Gate + +Release date: 2026-05-22 + +Status: released beta documentation/tooling metadata update on the +`1.0.0-beta` language baseline. + +`1.0.0-beta.14` documents the existing benchmark suite catalog and metadata +gate without changing the source language, runtime, standard library, API +surface, diagnostic output, compiler-known runtime names, or ABI/layout +behavior: + +- Adds [`benchmarks/README.md`](../../benchmarks/README.md) as the top-level + catalog for the current ten benchmark suites: + `math-loop`, `branch-loop`, `parse-loop`, `array-index-loop`, + `string-eq-loop`, `array-struct-field-loop`, + `enum-struct-payload-loop`, `vec-i32-index-loop`, + `vec-string-eq-loop`, and `json-quote-loop`. +- Documents `python3 benchmarks/runner.py --suite-list` as the non-JSON suite + inventory command. +- Documents `python3 benchmarks/runner.py --suite-list --json` as the beta + tooling metadata form for local gates and adapters. +- States that benchmark timings are local-machine evidence only and publishes + no timing numbers. +- Records that suite-list JSON is beta tooling metadata, not a stable public + schema. + +This release does not add benchmark kernels, publish timings, define +performance thresholds, define a stable JSON schema, change source-language, +runtime, stdlib/API, diagnostic-output, compiler ABI/layout, or optimizer +behavior, or make cross-machine performance claims. + ## 1.0.0-beta.13 Release label: `1.0.0-beta.13` diff --git a/docs/language/ROADMAP.md b/docs/language/ROADMAP.md index 10c9a99..e9bf9db 100644 --- a/docs/language/ROADMAP.md +++ b/docs/language/ROADMAP.md @@ -10,46 +10,53 @@ Long-horizon planning lives in release train from the historical `v2.0.0-beta.1` tag toward and beyond the first real general-purpose beta Slovo contract. -Current stage: `1.0.0-beta.13`, released on 2026-05-22 as a post-beta -diagnostic catalog and schema policy update. It keeps the `1.0.0-beta` -language contract and includes the `1.0.0-beta.1` tooling hardening release, -the `1.0.0-beta.2` -runtime/resource foundation release, the `1.0.0-beta.3` standard-library -stabilization release, the `1.0.0-beta.4` language-usability diagnostics -release, the `1.0.0-beta.5` package/workspace discipline release, and a narrow -`std.net` source facade for blocking loopback TCP client/server primitives over -opaque `i32` handles and concrete `result` families, plus a narrow `std.json` -source facade for compact JSON text construction, plus top-level module-local -transparent aliases for supported concrete target types, plus local alias use -inside the current concrete vector, option, and result facades, plus a -generated standard-library API catalog that lists exact exported helper -signatures with module-local aliases normalized to concrete public types, plus -`glagol symbols` deterministic source metadata for files, projects, and -workspaces, plus `glagol doc -o ` public API -sections for local packages/modules with exact exported function signatures, -exported struct fields, exported enum variants/payload types, non-export -filtering, and module-local alias normalization, plus source-authored concrete -vector helper parity: `std.vec_i64` gains `count_of`, `starts_with`, -`without_prefix`, `ends_with`, and `without_suffix`, and `std.vec_f64` gains -`count_of`, plus [`docs/language/DIAGNOSTICS.md`](DIAGNOSTICS.md) as the beta +Current stage: `1.0.0-beta.14`, released on 2026-05-22 as a post-beta +benchmark suite catalog and metadata gate. It keeps the `1.0.0-beta` language +contract and includes the `1.0.0-beta.1` tooling hardening release, the +`1.0.0-beta.2` runtime/resource foundation release, the `1.0.0-beta.3` +standard-library stabilization release, the `1.0.0-beta.4` +language-usability diagnostics release, the `1.0.0-beta.5` package/workspace +discipline release, and a narrow `std.net` source facade for blocking loopback +TCP client/server primitives over opaque `i32` handles and concrete `result` +families, plus a narrow `std.json` source facade for compact JSON text +construction, plus top-level module-local transparent aliases for supported +concrete target types, plus local alias use inside the current concrete vector, +option, and result facades, plus a generated standard-library API catalog that +lists exact exported helper signatures with module-local aliases normalized to +concrete public types, plus `glagol symbols` deterministic source metadata for +files, projects, and workspaces, plus +`glagol doc -o ` public API sections for local +packages/modules with exact exported function signatures, exported struct +fields, exported enum variants/payload types, non-export filtering, and +module-local alias normalization, plus source-authored concrete vector helper +parity: `std.vec_i64` gains `count_of`, `starts_with`, `without_prefix`, +`ends_with`, and `without_suffix`, and `std.vec_f64` gains `count_of`, plus +[`docs/language/DIAGNOSTICS.md`](DIAGNOSTICS.md) as the beta `slovo.diagnostic` version `1` policy and current golden diagnostic code -catalog. JSON parsing, recursive JSON values, executable generics, generic -aliases, parameterized aliases, cross-module alias visibility, maps/sets, -traits, inference, monomorphization, iterators, runtime changes for generic -collections, DNS, TLS, UDP, async IO, non-loopback binding, HTTP frameworks, -rich host-error ADTs, stable ABI/layout, stable Markdown schema, stable -stdlib/API compatibility freeze, LSP/watch, SARIF/daemon protocols, -stable `1.0.0` diagnostics freeze, re-exports/globs/hierarchical modules, -mutable vectors, slice/view APIs, new runtime names, performance claims, and -package registry semantics remain deferred. +catalog, plus [`../../benchmarks/README.md`](../../benchmarks/README.md) as +the beta-scoped benchmark suite catalog documenting the current suite +inventory, `python3 benchmarks/runner.py --suite-list`, and +`python3 benchmarks/runner.py --suite-list --json`. Benchmark timings remain +local-machine evidence only, and suite-list JSON is beta tooling metadata +rather than a stable public schema. JSON parsing, recursive JSON values, +executable generics, generic aliases, parameterized aliases, cross-module alias +visibility, maps/sets, traits, inference, monomorphization, iterators, runtime +changes for generic collections, DNS, TLS, UDP, async IO, non-loopback binding, +HTTP frameworks, rich host-error ADTs, stable ABI/layout, stable Markdown +schema, stable stdlib/API compatibility freeze, LSP/watch, SARIF/daemon +protocols, stable `1.0.0` diagnostics freeze, re-exports/globs/hierarchical +modules, mutable vectors, slice/view APIs, new runtime names, timing +publication, performance claims, stable benchmark JSON schema, and package +registry semantics remain deferred. -Next stage target: continue after `1.0.0-beta.13` from developer-experience, -package, and reserved generic/map/set planning without claiming executable -generics, an LSP/watch protocol, SARIF/daemon protocol, stable Markdown -schema, registry semantics, stable `1.0.0` diagnostics freeze, or stable -standard-library/API compatibility freeze, mutable vectors, slice/view APIs, -new runtime names, performance claims, maps/sets, or iterators until the exact -scope is frozen from the manifest and roadmap. +Next stage target: continue after `1.0.0-beta.14` from developer-experience, +package, benchmark metadata, and reserved generic/map/set planning without +claiming executable generics, an LSP/watch protocol, SARIF/daemon protocol, +stable Markdown schema, registry semantics, stable benchmark JSON schema, +stable `1.0.0` diagnostics freeze, or stable standard-library/API +compatibility freeze, mutable vectors, slice/view APIs, new runtime names, +performance claims, maps/sets, or iterators until the exact scope is frozen +from the manifest and roadmap. The final experimental precursor scope is `exp-125`, defined in `.llm/EXP_125_UNSIGNED_U32_U64_NUMERIC_AND_STDLIB_BREADTH_ALPHA.md`. Its diff --git a/docs/language/SPEC-v1.md b/docs/language/SPEC-v1.md index b8726bf..e39ed5e 100644 --- a/docs/language/SPEC-v1.md +++ b/docs/language/SPEC-v1.md @@ -5,7 +5,8 @@ Status: living beta contract for `1.0.0-beta`, with the post-beta foundation update, `1.0.0-beta.10` developer-experience API discovery update, `1.0.0-beta.11` local package API documentation update, and `1.0.0-beta.12` concrete vector query and prefix parity update, and `1.0.0-beta.13` -diagnostic catalog and schema policy update. The language contract integrates +diagnostic catalog and schema policy update, and `1.0.0-beta.14` benchmark +suite catalog and metadata gate. The language contract integrates promoted language slices through `exp-125` and the historical publication baseline through `exp-123`. `1.0.0-beta` is the first real general-purpose beta release. `exp-125` completed the unsigned numeric and stdlib breadth @@ -157,6 +158,16 @@ Current v1 release surface and explicit experimental targets: current golden diagnostic code catalog; this is docs/tooling policy only, not a source-language/runtime change, LSP/watch contract, SARIF/daemon protocol, stable Markdown schema, or stable `1.0.0` diagnostics freeze +- `1.0.0-beta.14` benchmark suite catalog and metadata target: + [`benchmarks/README.md`](../../benchmarks/README.md) documents the existing + benchmark suite inventory and root suite-list commands: + `python3 benchmarks/runner.py --suite-list` and + `python3 benchmarks/runner.py --suite-list --json`; benchmark timing remains + local-machine evidence only, and suite-list JSON is beta tooling metadata + only, not a stable public schema; this is docs/tooling metadata only, not new + benchmark kernels, timing publication, performance thresholds, + source-language/runtime/stdlib/API changes, diagnostic-output changes, or + ABI/layout changes - `exp-1` owned runtime strings: compiler-known `std.string.concat` accepts two `string` values and returns an immutable runtime-owned `string`; existing string equality, length, printing, locals, parameters, returns, and calls work @@ -1379,6 +1390,39 @@ protocols, stable Markdown schema, stable `1.0.0` diagnostics freeze, source-map/debug-metadata contracts, localized diagnostic text, automated machine fix-its, or a compiler-emitted diagnostic catalog artifact. +### 4.4.11 Post-Beta Benchmark Suite Catalog And Metadata Gate + +Status: released in `1.0.0-beta.14` as a docs/tooling metadata update for the +existing benchmark suite. + +`1.0.0-beta.14` adds +[`benchmarks/README.md`](../../benchmarks/README.md) as the top-level +benchmark suite catalog. The catalog records the current suite inventory, +base/hot-loop checksum metadata, local evidence policy, and explicit +exclusions. + +The root suite-list commands are documented as: + +```bash +python3 benchmarks/runner.py --suite-list +python3 benchmarks/runner.py --suite-list --json +``` + +The non-JSON listing is local review output. The JSON listing is beta tooling +metadata for local gates and adapters. The field set is intentionally not a +stable public schema in this release. + +The source language, typed core, runtime, standard library, public API surface, +compiler diagnostics, diagnostic output shape, compiler-known runtime names, +ABI/layout behavior, and optimizer contract are unchanged by this target. +Benchmark timings remain local-machine evidence only and no timing numbers are +published by the release contract. + +This target explicitly does not add benchmark kernels, implementation language +slots, timing publication, performance thresholds, cross-machine performance +claims, a stable benchmark JSON schema, source-language/runtime/stdlib/API +changes, diagnostic-output changes, or ABI/layout guarantees. + ## 4.5 v2.0.0-beta.1 Experimental Integration Readiness Status: current experimental Slovo-side release contract, released 2026-05-17. diff --git a/docs/language/STDLIB_API.md b/docs/language/STDLIB_API.md index ccfb8e5..a829356 100644 --- a/docs/language/STDLIB_API.md +++ b/docs/language/STDLIB_API.md @@ -6,7 +6,7 @@ Do not edit this file by hand. ## Stability Tiers - `beta-supported`: exported from `lib/std` and covered by source-search, promotion, or facade gates in the current beta line. -- `experimental`: not used for exported `lib/std` helpers in `1.0.0-beta.13`; future releases may mark new helpers this way before they graduate. +- `experimental`: not used for exported `lib/std` helpers in `1.0.0-beta.14`; future releases may mark new helpers this way before they graduate. - `internal`: helper names that are not exported from their module; they are intentionally omitted from this catalog. The catalog is a beta API discovery aid, not a stable `1.0.0` standard-library freeze. diff --git a/scripts/release-gate.sh b/scripts/release-gate.sh index 668c129..cde4909 100755 --- a/scripts/release-gate.sh +++ b/scripts/release-gate.sh @@ -64,6 +64,7 @@ fi cd "${compiler_dir}" cargo fmt --check cargo test --test diagnostics_schema_beta13 +cargo test --test benchmark_suite_catalog_beta14 # Full cargo test includes unignored integration gates such as dx_v1_7, # beta_v2_0_0_beta_1, and beta_1_0_0. cargo test