From 98f81d2d59e1f4e23df0da2d7650c5472129489b Mon Sep 17 00:00:00 2001 From: sanjin Date: Sat, 23 May 2026 01:01:44 +0200 Subject: [PATCH] Release 1.0.0-beta.19 test discovery foundation --- .../BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md | 87 ++++ .llm/reviews/BETA_19_RELEASE_REVIEW.md | 55 +++ README.md | 31 +- compiler/Cargo.lock | 2 +- compiler/Cargo.toml | 2 +- compiler/src/driver.rs | 10 + compiler/src/main.rs | 42 +- compiler/src/project.rs | 20 + compiler/src/test_runner.rs | 35 ++ compiler/tests/test_discovery_beta19.rs | 384 ++++++++++++++++++ docs/POST_BETA_ROADMAP.md | 15 + docs/compiler/RELEASE_NOTES.md | 37 +- docs/compiler/ROADMAP.md | 30 +- docs/language/RELEASE_NOTES.md | 36 +- docs/language/ROADMAP.md | 35 +- docs/language/SPEC-v1.md | 15 +- docs/language/STDLIB_API.md | 2 +- scripts/release-gate.sh | 1 + 18 files changed, 787 insertions(+), 52 deletions(-) create mode 100644 .llm/BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md create mode 100644 .llm/reviews/BETA_19_RELEASE_REVIEW.md create mode 100644 compiler/tests/test_discovery_beta19.rs diff --git a/.llm/BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md b/.llm/BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md new file mode 100644 index 0000000..33729cd --- /dev/null +++ b/.llm/BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md @@ -0,0 +1,87 @@ +# 1.0.0-beta.19 Test Discovery And User-Project Conformance Foundation + +## Scope + +`1.0.0-beta.19` is a compiler/tooling and conformance slice. It does +not change the Slovo source language or standard library surface. + +Add deterministic list-only test discovery for: + +- `glagol test --list ` +- `glagol --run-tests --list ` for the legacy single-file path + +## Contract + +List mode must reuse the same checked front-end path as normal test execution: +parse, lower, type-check, resolve project/workspace inputs, discover tests, and +apply `--filter `. + +The command then lists discovered/selected tests without evaluating test bodies. +It must not execute runtime calls from test bodies, mutate files through test +logic, open sockets through test logic, or otherwise trigger user test +side-effects. + +Ordering must remain deterministic and match current test execution discovery: + +- single-file tests keep source order +- project tests keep existing module/package discovery order +- workspace tests keep existing workspace/package discovery order + +Normal `glagol test` behavior and output remain unchanged unless `--list` is +present. Invalid files, projects, and workspaces still fail through the +existing diagnostic path. + +## Output Shape + +The initial output format is beta tooling. It should be stable enough for local +release-gate tests, but it is not a frozen public schema. + +The output should make selected, skipped, total discovered, and filter state +visible. A concise text shape is enough; a stable JSON/event stream is out of +scope for this slice. + +## Non-Scope + +This scope does not add: + +- source-language syntax +- runtime helper names +- JSON expansion +- parallel test execution +- retries +- tags or groups +- coverage reports +- event streams +- stable artifact-manifest schema freeze +- stable Markdown schema freeze +- LSP or watch behavior +- SARIF or daemon protocols +- package registries +- semver solving +- performance claims + +## Acceptance Criteria + +- `glagol test --list ` lists checked/discovered tests without + executing bodies. +- `glagol test --list ` and workspace inputs preserve current + project/workspace ordering. +- `glagol --run-tests --list ` works for the legacy single-file path. +- `--filter ` marks/selects the same tests as normal filtered + execution while avoiding body evaluation. +- Normal `glagol test` output stays byte-stable for existing covered cases. +- Invalid inputs still emit existing diagnostics. +- Docs describe beta19 as a released tooling/conformance slice. +- Release-gate coverage includes the focused beta19 test-discovery suite. + +## Suggested Gates + +```bash +cargo fmt --check +cargo test --test test_discovery_beta19 +cargo test --test project_mode +cargo test --test cli_v1_1 +cargo test --test diagnostics_schema_beta13 +cargo test +./scripts/release-gate.sh +``` diff --git a/.llm/reviews/BETA_19_RELEASE_REVIEW.md b/.llm/reviews/BETA_19_RELEASE_REVIEW.md new file mode 100644 index 0000000..53c65cd --- /dev/null +++ b/.llm/reviews/BETA_19_RELEASE_REVIEW.md @@ -0,0 +1,55 @@ +# 1.0.0-beta.19 Release Review + +Scope: Test Discovery And User-Project Conformance Foundation + +## Findings + +No blocking findings. + +The implementation matches the beta19 contract at the release-blocking level: +`glagol test --list ` and legacy +`glagol --run-tests --list ` route through checked discovery, avoid test +body evaluation, preserve the existing discovery ordering, honor +`--filter `, keep ordinary test output unchanged, and are wired into +the release gate through `cargo test --test test_discovery_beta19`. + +## Non-Blocking Notes + +- Resolved during controller integration: unfiltered list output now prints the + same summary suffix as filtered list output, including `total_discovered`, + `selected`, `passed`, `failed`, `skipped`, and `filter none`. +- Resolved during controller integration: the grammar typo in + `docs/POST_BETA_ROADMAP.md` was corrected. + +## Verification Notes + +Inspected: + +- Working tree status and beta19 diff across CLI parsing/dispatch, project test + mode, test-runner listing, focused tests, docs, version files, and + `scripts/release-gate.sh`. +- Contract drift against + `.llm/BETA_19_TEST_DISCOVERY_AND_CONFORMANCE.md`, + `docs/language/SPEC-v1.md`, release notes, roadmaps, and README beta scope. +- Cached diff status; no cached beta19 changes were present. + +Read-only checks run: + +- `git diff --check` - passed. +- `git diff --cached --check` - passed. +- Stale-version scan for beta18/beta19 references - no blocking stale current + release references found. +- Conflict-marker and trailing-whitespace scans for the untracked beta19 + contract/test files - passed. + +Not run: + +- `cargo fmt --check`, focused cargo tests, full `cargo test`, and + `./scripts/release-gate.sh`; those commands write build artifacts, and this + review was constrained to read-only commands except for the review file. + +## Verdict + +Release-ready from this review. No blocking beta19 issues remain in the current +working tree diff. The controller should still run the focused beta19 test suite +and full release gate before tagging. diff --git a/README.md b/README.md index cdd1c09..ce9383c 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This repository is the canonical public monorepo for the language design, standard library source, compiler, runtime, examples, benchmarks, and technical documents. -Current release: `1.0.0-beta.18`. +Current release: `1.0.0-beta.19`. ## Repository Layout @@ -24,7 +24,7 @@ scripts/ local release and document tooling ## Beta Scope -`1.0.0-beta.18` keeps the `1.0.0-beta` language baseline, includes the +`1.0.0-beta.19` keeps the `1.0.0-beta` language baseline, includes the `1.0.0-beta.1` tooling/install hardening slice, the `1.0.0-beta.2` runtime/resource foundation bundle, the `1.0.0-beta.3` standard-library stabilization bundle, the `1.0.0-beta.4` language-usability diagnostics @@ -39,8 +39,9 @@ slice, the `1.0.0-beta.13` diagnostic catalog and schema policy slice, the `1.0.0-beta.14` benchmark suite catalog and metadata gate, and the `1.0.0-beta.15` reserved generic collection boundary hardening and collection ledger, the `1.0.0-beta.16` string scanning and token boundary foundation, -the `1.0.0-beta.17` JSON primitive scalar parsing foundation, and the -`1.0.0-beta.18` JSON string token parsing foundation. +the `1.0.0-beta.17` JSON primitive scalar parsing foundation, the +`1.0.0-beta.18` JSON string token parsing foundation, and the +`1.0.0-beta.19` test discovery and user-project conformance foundation. The language baseline supports practical local command-line, file, and loopback-network programs with: @@ -131,6 +132,13 @@ escapes `\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, and `\t`, and returns parsing, tokenizer APIs, Unicode escape decoding/normalization, embedded NUL policy, stable ABI/layout, and a stable stdlib/API freeze remain deferred. +The `1.0.0-beta.19` test discovery and user-project conformance foundation +adds `glagol test --list ` plus legacy +`glagol --run-tests --list ` support for listing checked and discovered +tests without executing test bodies. The list mode preserves existing file, +project, and workspace test ordering, honors `--filter `, and +remains beta tooling rather than a stable output schema. + Still deferred before stable: executable generics, generic aliases, maps/sets, broad package registry semantics, stable Markdown schema, stable stdlib/API compatibility freeze, DNS/TLS/async networking, LSP/watch guarantees, SARIF @@ -140,16 +148,11 @@ iterators, additional compiler-known runtime names, stable ABI and layout, performance claims, stable benchmark JSON metadata schema, and runtime changes for generic collections. -The next likely language slice after `1.0.0-beta.18` should continue from the -developer-experience, package, benchmark metadata, collection, or string -processing lanes without claiming executable generics, maps, sets, traits, -inference, monomorphization, iterators, ABI stability, broad runtime changes, -LSP/watch protocols, SARIF/daemon protocols, registry semantics, stable -Markdown schema, stable benchmark JSON schema, a stable `1.0.0` diagnostics -freeze, standard-library/API compatibility freeze, mutable vectors, language -slice/view APIs, additional runtime names, Unicode/grapheme semantics, broader -JSON object/array/full-value parsing, or performance claims until the contract -and gates are explicit. +The beta19 tooling scope is deliberately tooling-only. It does not add parallel +test execution, retries, tags/groups, coverage reports, event streams, stable +manifest or Markdown schema guarantees, LSP/watch behavior, SARIF/daemon +protocols, JSON expansion, runtime helper names, source-language syntax, +remote package registries, semver solving, or performance claims. ## Build And Test diff --git a/compiler/Cargo.lock b/compiler/Cargo.lock index bfda16e..96f73d4 100644 --- a/compiler/Cargo.lock +++ b/compiler/Cargo.lock @@ -4,4 +4,4 @@ version = 3 [[package]] name = "glagol" -version = "1.0.0-beta.18" +version = "1.0.0-beta.19" diff --git a/compiler/Cargo.toml b/compiler/Cargo.toml index e0493bb..43df5b7 100644 --- a/compiler/Cargo.toml +++ b/compiler/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "glagol" -version = "1.0.0-beta.18" +version = "1.0.0-beta.19" edition = "2021" description = "Glagol, the first compiler for the Slovo language" license = "MIT OR Apache-2.0" diff --git a/compiler/src/driver.rs b/compiler/src/driver.rs index fa7874e..d4a0b71 100644 --- a/compiler/src/driver.rs +++ b/compiler/src/driver.rs @@ -52,6 +52,16 @@ pub fn run_tests( test_runner::run(file, &checked, filter) } +pub fn list_tests( + file: &str, + source: &str, + filter: Option<&str>, +) -> Result { + let checked = + checked_program(file, source).map_err(test_runner::TestRunFailure::before_execution)?; + Ok(test_runner::list(&checked, filter)) +} + fn checked_program(file: &str, source: &str) -> Result> { let tokens = lexer::lex(file, source)?; let forms = sexpr::parse(file, &tokens)?; diff --git a/compiler/src/main.rs b/compiler/src/main.rs index 3ed743b..749831f 100644 --- a/compiler/src/main.rs +++ b/compiler/src/main.rs @@ -184,7 +184,13 @@ fn run_project_check(invocation: Invocation) -> ! { } fn run_project_test(invocation: Invocation) -> ! { - match project::run_tests(&invocation.path, invocation.test_filter.as_deref()) { + let result = if invocation.test_list { + project::list_tests(&invocation.path, invocation.test_filter.as_deref()) + } else { + project::run_tests(&invocation.path, invocation.test_filter.as_deref()) + }; + + match result { Ok(success) => { let output = success.output; let primary_output = if let Some(output_path) = invocation.output_path.as_deref() { @@ -679,7 +685,13 @@ fn finish_formatted_source( fn run_test_mode(invocation: Invocation, source: &str) -> ! { let foreign_imports = c_imports_for_manifest(&invocation.path, source); - match driver::run_tests(&invocation.path, source, invocation.test_filter.as_deref()) { + let result = if invocation.test_list { + driver::list_tests(&invocation.path, source, invocation.test_filter.as_deref()) + } else { + driver::run_tests(&invocation.path, source, invocation.test_filter.as_deref()) + }; + + match result { Ok(result) => { let output = result.output; let primary_output = if let Some(output_path) = invocation.output_path.as_deref() { @@ -1101,6 +1113,7 @@ struct Invocation { project_template: scaffold::ProjectTemplate, link_c_paths: Vec, test_filter: Option, + test_list: bool, run_args: Vec, } @@ -1137,6 +1150,7 @@ fn parse_args(raw_args: &[String]) -> Result { let mut project_template = scaffold::ProjectTemplate::Binary; let mut link_c_paths = Vec::new(); let mut test_filter = None; + let mut test_list = false; let mut run_args = Vec::new(); let mut no_color = false; let command_line = raw_args.join(" "); @@ -1345,6 +1359,17 @@ fn parse_args(raw_args: &[String]) -> Result { command_line: command_line.clone(), })?); } + "--list" => { + if test_list { + return parse_error( + "`--list` was provided more than once", + manifest_path, + diagnostics, + command_line, + ); + } + test_list = true; + } "check" | "fmt" | "test" | "build" | "run" | "clean" | "new" | "doc" | "symbols" if path.is_none() => { @@ -1464,6 +1489,15 @@ fn parse_args(raw_args: &[String]) -> Result { ); } + if test_list && mode != Mode::RunTests { + return parse_error( + "`--list` is only supported with `test` and `--run-tests`", + manifest_path, + diagnostics, + command_line, + ); + } + if !run_args.is_empty() && mode != Mode::Run { return parse_error( "`--` program arguments are only supported with `run`", @@ -1515,6 +1549,7 @@ fn parse_args(raw_args: &[String]) -> Result { project_template, link_c_paths, test_filter, + test_list, run_args, })) } @@ -1588,6 +1623,7 @@ fn exit_parse_error(err: ParseError, command_line: &str) -> ! { project_template: scaffold::ProjectTemplate::Binary, link_c_paths: Vec::new(), test_filter: None, + test_list: false, run_args: Vec::new(), }; write_manifest_or_exit( @@ -2567,6 +2603,6 @@ fn normalized_output_path(path: &str) -> Option { fn print_usage() { eprintln!( - "usage: glagol [check|fmt|test|build|run|clean|symbols] [--json-diagnostics] [--no-color] [--manifest ] [--link-c ] [-o ] [--filter ] [-- ...]\n glagol fmt [--check|--write] \n glagol new [--name ] [--template binary|library|workspace]\n glagol doc -o \n glagol symbols \n glagol [--emit=llvm|--format|--print-tree|--inspect-lowering=surface|--inspect-lowering=checked|--check-tests|--run-tests] [--json-diagnostics] [--no-color] [-o ] [--manifest ] [--filter ] \n glagol --version" + "usage: glagol [check|fmt|test|build|run|clean|symbols] [--json-diagnostics] [--no-color] [--manifest ] [--link-c ] [-o ] [--filter ] [--list] [-- ...]\n glagol fmt [--check|--write] \n glagol new [--name ] [--template binary|library|workspace]\n glagol doc -o \n glagol symbols \n glagol [--emit=llvm|--format|--print-tree|--inspect-lowering=surface|--inspect-lowering=checked|--check-tests|--run-tests] [--json-diagnostics] [--no-color] [-o ] [--manifest ] [--filter ] [--list] \n glagol --version" ); } diff --git a/compiler/src/project.rs b/compiler/src/project.rs index 6fa1c84..008d62e 100644 --- a/compiler/src/project.rs +++ b/compiler/src/project.rs @@ -452,6 +452,26 @@ pub fn run_tests( } } +pub fn list_tests( + input: &str, + filter: Option<&str>, +) -> Result { + let checked = load_checked_project(input, false).map_err(|failure| ProjectTestFailure { + diagnostics: failure.diagnostics, + report: None, + sources: failure.sources, + artifact: failure.artifact, + })?; + + let success = test_runner::list(&checked.program, filter); + Ok(ProjectTestSuccess { + output: success.output, + report: success.report, + sources: checked.sources, + artifact: checked.artifact, + }) +} + struct CheckedProject { program: CheckedProgram, sources: Vec, diff --git a/compiler/src/test_runner.rs b/compiler/src/test_runner.rs index 46ca463..2476bd5 100644 --- a/compiler/src/test_runner.rs +++ b/compiler/src/test_runner.rs @@ -173,6 +173,39 @@ pub fn run( } } +pub fn list(program: &CheckedProgram, filter: Option<&str>) -> TestRunSuccess { + let mut output = String::new(); + let mut report = TestReport { + total_discovered: program.tests.len(), + selected: 0, + passed: 0, + failed: 0, + skipped: 0, + filter: filter.map(str::to_string), + }; + + for test in &program.tests { + output.push_str("test "); + write_test_name(&test.name, &mut output); + if let Some(filter) = filter { + if !test.name.contains(filter) { + report.skipped += 1; + output.push_str(" ... skipped\n"); + continue; + } + } + + report.selected += 1; + output.push_str(" ... selected\n"); + } + + output.push_str(&format!("{} test(s) selected", report.selected)); + write_report_suffix(&report, &mut output); + output.push('\n'); + + TestRunSuccess { output, report } +} + fn write_report_suffix(report: &TestReport, output: &mut String) { output.push_str(&format!( " (total_discovered {}, selected {}, passed {}, failed {}, skipped {}", @@ -181,6 +214,8 @@ fn write_report_suffix(report: &TestReport, output: &mut String) { if let Some(filter) = report.filter.as_deref() { output.push_str(", filter "); write_test_name(filter, output); + } else { + output.push_str(", filter none"); } output.push(')'); } diff --git a/compiler/tests/test_discovery_beta19.rs b/compiler/tests/test_discovery_beta19.rs new file mode 100644 index 0000000..6250cf6 --- /dev/null +++ b/compiler/tests/test_discovery_beta19.rs @@ -0,0 +1,384 @@ +use std::{ + fs, + path::PathBuf, + process::{Command, Output}, + sync::atomic::{AtomicUsize, Ordering}, + time::{SystemTime, UNIX_EPOCH}, +}; + +static NEXT_FIXTURE_ID: AtomicUsize = AtomicUsize::new(0); + +#[test] +fn file_list_preserves_order_and_does_not_execute_bodies() { + let fixture = write_fixture( + "file-list", + r#" +(module main) + +(test "alpha first" + true) + +(test "beta would fail" + false) +"#, + ); + + let output = run_glagol(["test".as_ref(), "--list".as_ref(), fixture.as_os_str()]); + + assert_success_stdout( + "file test list", + output, + "test \"alpha first\" ... selected\n\ +test \"beta would fail\" ... selected\n\ +2 test(s) selected (total_discovered 2, selected 2, passed 0, failed 0, skipped 0, filter none)\n", + ); +} + +#[test] +fn file_list_filter_marks_selected_and_skipped_in_order() { + let fixture = write_fixture( + "file-list-filter", + r#" +(module main) + +(test "alpha first" + false) + +(test "beta second" + false) +"#, + ); + + let output = run_glagol([ + "test".as_ref(), + "--list".as_ref(), + fixture.as_os_str(), + "--filter".as_ref(), + "beta".as_ref(), + ]); + + assert_success_stdout( + "file filtered test list", + output, + "test \"alpha first\" ... skipped\n\ +test \"beta second\" ... selected\n\ +1 test(s) selected (total_discovered 2, selected 1, passed 0, failed 0, skipped 1, filter \"beta\")\n", + ); +} + +#[test] +fn legacy_run_tests_list_matches_test_subcommand_list() { + let fixture = write_fixture( + "legacy-list", + r#" +(module main) + +(test "legacy first" + false) + +(test "legacy second" + false) +"#, + ); + + let subcommand = run_glagol(["test".as_ref(), "--list".as_ref(), fixture.as_os_str()]); + let legacy = run_glagol([ + "--run-tests".as_ref(), + "--list".as_ref(), + fixture.as_os_str(), + ]); + + assert_success("test --list", &subcommand); + assert_success("legacy --run-tests --list", &legacy); + assert_eq!( + legacy.stdout, subcommand.stdout, + "legacy list output differed from test subcommand" + ); + assert!(legacy.stderr.is_empty(), "legacy list wrote stderr"); +} + +#[test] +fn project_list_preserves_topological_order_and_filter_counts() { + let project = write_project( + "project-list", + &[( + "provider", + "(module provider (export value))\n\n\ +(fn value () -> i32\n 1)\n\n\ +(test \"provider first\"\n false)\n", + )], + "(module main)\n\n\ +(import provider (value))\n\n\ +(fn main () -> i32\n (value))\n\n\ +(test \"consumer second\"\n false)\n", + ); + + let output = run_glagol([ + "test".as_ref(), + "--list".as_ref(), + project.as_os_str(), + "--filter".as_ref(), + "consumer".as_ref(), + ]); + + assert_success_stdout( + "project filtered list", + output, + "test \"provider first\" ... skipped\n\ +test \"consumer second\" ... selected\n\ +1 test(s) selected (total_discovered 2, selected 1, passed 0, failed 0, skipped 1, filter \"consumer\")\n", + ); +} + +#[test] +fn workspace_list_preserves_package_order_without_running_tests() { + let workspace = write_workspace( + "workspace-list", + "[workspace]\nmembers = [\"packages/app\", \"packages/util\"]\n", + &[ + WorkspacePackageSpec { + member: "packages/util", + manifest: "[package]\nname = \"util\"\nversion = \"0.1.0\"\n", + modules: &[( + "util", + "(module util (export answer))\n\n\ +(fn answer () -> i32\n 41)\n\n\ +(test \"util provider first\"\n false)\n", + )], + }, + WorkspacePackageSpec { + member: "packages/app", + manifest: "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n[dependencies]\nutil = { path = \"../util\" }\n", + modules: &[( + "main", + "(module main)\n\n\ +(import util.util (answer))\n\n\ +(fn main () -> i32\n (answer))\n\n\ +(test \"app consumer second\"\n false)\n", + )], + }, + ], + ); + + let output = run_glagol(["test".as_ref(), "--list".as_ref(), workspace.as_os_str()]); + + assert_success_stdout( + "workspace list", + output, + "test \"util provider first\" ... selected\n\ +test \"app consumer second\" ... selected\n\ +2 test(s) selected (total_discovered 2, selected 2, passed 0, failed 0, skipped 0, filter none)\n", + ); +} + +#[test] +fn ordinary_test_output_stays_byte_stable() { + let fixture = write_fixture( + "ordinary-test", + r#" +(module main) + +(test "alpha first" + true) + +(test "beta second" + true) +"#, + ); + + let subcommand = run_glagol(["test".as_ref(), fixture.as_os_str()]); + assert_success_stdout( + "ordinary test", + subcommand, + "test \"alpha first\" ... ok\n\ +test \"beta second\" ... ok\n\ +2 test(s) passed\n", + ); + + let legacy = run_glagol(["--run-tests".as_ref(), fixture.as_os_str()]); + assert_success_stdout( + "ordinary legacy run-tests", + legacy, + "test \"alpha first\" ... ok\n\ +test \"beta second\" ... ok\n\ +2 test(s) passed\n", + ); +} + +#[test] +fn list_mode_reuses_checked_diagnostics_for_invalid_tests() { + let fixture = write_fixture( + "invalid-list", + r#" +(module main) + +(test "not bool" + 1) +"#, + ); + + let output = run_glagol(["test".as_ref(), "--list".as_ref(), fixture.as_os_str()]); + + assert_failure_stderr_contains( + "invalid list", + &output, + &[ + "TestExpressionNotBool", + "Expected:", + "bool", + "Found:", + "i32", + ], + ); + assert!( + output.stdout.is_empty(), + "invalid list wrote stdout:\n{}", + String::from_utf8_lossy(&output.stdout) + ); +} + +#[test] +fn list_flag_is_rejected_outside_test_mode() { + let fixture = write_fixture( + "list-outside-test", + r#" +(module main) + +(fn main () -> i32 + 0) +"#, + ); + + let output = run_glagol(["check".as_ref(), "--list".as_ref(), fixture.as_os_str()]); + + assert_failure_stderr_contains( + "check --list", + &output, + &["`--list` is only supported with `test` and `--run-tests`"], + ); +} + +fn write_fixture(name: &str, source: &str) -> PathBuf { + let path = unique_path(name).with_extension("slo"); + fs::write(&path, source).expect("write fixture"); + path +} + +fn write_project(name: &str, modules: &[(&str, &str)], main: &str) -> PathBuf { + let root = unique_path(name); + let src = root.join("src"); + fs::create_dir_all(&src).expect("create project src"); + fs::write( + root.join("slovo.toml"), + format!( + "[project]\nname = \"{}\"\nsource_root = \"src\"\nentry = \"main\"\n", + name + ), + ) + .expect("write project manifest"); + fs::write(src.join("main.slo"), main).expect("write project main"); + for (module, source) in modules { + fs::write(src.join(format!("{}.slo", module)), source).expect("write project module"); + } + root +} + +struct WorkspacePackageSpec<'a> { + member: &'a str, + manifest: &'a str, + modules: &'a [(&'a str, &'a str)], +} + +fn write_workspace( + name: &str, + workspace_manifest: &str, + packages: &[WorkspacePackageSpec<'_>], +) -> PathBuf { + let root = unique_path(name); + fs::create_dir_all(&root).expect("create workspace root"); + fs::write(root.join("slovo.toml"), workspace_manifest).expect("write workspace manifest"); + for package in packages { + let package_root = root.join(package.member); + let src = package_root.join("src"); + fs::create_dir_all(&src).expect("create workspace package src"); + fs::write(package_root.join("slovo.toml"), package.manifest) + .expect("write workspace package manifest"); + for (module, source) in package.modules { + fs::write(src.join(format!("{}.slo", module)), source) + .expect("write workspace package module"); + } + } + root +} + +fn unique_path(name: &str) -> PathBuf { + let id = NEXT_FIXTURE_ID.fetch_add(1, Ordering::SeqCst); + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|duration| duration.as_nanos()) + .unwrap_or(0); + std::env::temp_dir().join(format!( + "glagol-test-discovery-beta19-{}-{}-{}-{}", + std::process::id(), + nanos, + id, + name + )) +} + +fn run_glagol(args: I) -> Output +where + I: IntoIterator, + S: AsRef, +{ + Command::new(env!("CARGO_BIN_EXE_glagol")) + .args(args) + .output() + .expect("run glagol") +} + +fn assert_success(context: &str, output: &Output) { + assert!( + output.status.success(), + "{} failed\nstdout:\n{}\nstderr:\n{}", + context, + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); +} + +fn assert_success_stdout(context: &str, output: Output, expected: &str) { + assert_success(context, &output); + assert_eq!( + String::from_utf8_lossy(&output.stdout), + expected, + "{} stdout mismatch", + context + ); + assert!( + output.stderr.is_empty(), + "{} wrote stderr:\n{}", + context, + String::from_utf8_lossy(&output.stderr) + ); +} + +fn assert_failure_stderr_contains(context: &str, output: &Output, expected: &[&str]) { + assert!( + !output.status.success(), + "{} unexpectedly succeeded\nstdout:\n{}\nstderr:\n{}", + context, + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ); + let stderr = String::from_utf8_lossy(&output.stderr); + for needle in expected { + assert!( + stderr.contains(needle), + "{} stderr missing `{}`:\n{}", + context, + needle, + stderr + ); + } +} diff --git a/docs/POST_BETA_ROADMAP.md b/docs/POST_BETA_ROADMAP.md index feac0a0..35180c1 100644 --- a/docs/POST_BETA_ROADMAP.md +++ b/docs/POST_BETA_ROADMAP.md @@ -41,6 +41,21 @@ Released in `1.0.0-beta.1`: `glagol run`, `glagol clean`, installed std/runtime discovery, README coverage, focused DX tests, and a concise release-gate success line. +Released in `1.0.0-beta.19`: test discovery and user-project conformance +tooling. The scope adds the +`glagol test --list ` command and legacy +`glagol --run-tests --list ` so users and tooling can list +checked/discovered tests without executing test bodies. It preserves existing +single-file, project, and workspace ordering, honors +`--filter `, and keeps the output beta-scoped rather than a stable +public schema. + +Beta19 non-scope: no parallel test execution, retries, tags/groups, coverage +reports, event streams, stable artifact-manifest or Markdown schema freeze, +LSP/watch behavior, SARIF/daemon protocols, JSON expansion, runtime helper +names, source-language syntax, package registries, semver solving, or +performance claims. + Why first: it reduces friction for every later feature and gives users a better way to exercise the beta. diff --git a/docs/compiler/RELEASE_NOTES.md b/docs/compiler/RELEASE_NOTES.md index d71267a..162a412 100644 --- a/docs/compiler/RELEASE_NOTES.md +++ b/docs/compiler/RELEASE_NOTES.md @@ -10,10 +10,41 @@ integration/readiness release, not the first real beta. ## Unreleased -Next scoped Glagol work is expected to continue after the `1.0.0-beta.18` -JSON string token parsing foundation. +No active unreleased compiler scope is documented here yet. -No unreleased compiler scope is committed here yet. +## 1.0.0-beta.19 + +Release label: `1.0.0-beta.19` + +Release date: 2026-05-23 + +Release state: test discovery and user-project conformance foundation + +### Summary + +The beta.19 compiler/tooling contract adds deterministic list-only test +discovery without changing normal test execution output: + +- `glagol test --list ` +- `glagol --run-tests --list ` for legacy single-file test execution + +- Bump the `glagol` compiler package version to `1.0.0-beta.19`. +- Reuse the same checked discovery path as normal `glagol test` for file, + project, and workspace inputs. +- Preserve current file/project/workspace test ordering. +- Honor `--filter ` by marking selected and skipped discovered + tests without executing test bodies or triggering runtime/test side effects. +- Keep normal `glagol test` and legacy `glagol --run-tests` output unchanged + when `--list` is absent. +- Add focused beta19 release-gate coverage for test discovery. + +### Explicit Deferrals + +This release does not implement parallel test execution, retries, tags/groups, coverage, +event streams, stable artifact-manifest or Markdown schema freezes, +LSP/watch/SARIF/daemon protocols, JSON expansion, runtime helper names, +source-language syntax, package registries, semver solving, and performance +claims. ## 1.0.0-beta.18 diff --git a/docs/compiler/ROADMAP.md b/docs/compiler/ROADMAP.md index 42a4b3d..db428af 100644 --- a/docs/compiler/ROADMAP.md +++ b/docs/compiler/ROADMAP.md @@ -22,8 +22,8 @@ general-purpose beta release. A Glagol feature is done only when it has parser/lowerer support, checker behavior, diagnostics for invalid forms, backend behavior or explicit unsupported diagnostics, and tests. -Current stage: `1.0.0-beta.18`, released on 2026-05-23 as a JSON string -token parsing runtime foundation. It keeps the +Current stage: `1.0.0-beta.19`, released on 2026-05-23 as a test discovery +and user-project conformance foundation. It keeps the `1.0.0-beta` language/compiler support baseline and includes the `1.0.0-beta.1` tooling hardening release, the `1.0.0-beta.2` runtime/resource foundation release, the `1.0.0-beta.3` standard-library stabilization release, @@ -89,16 +89,24 @@ diagnostics. It keeps object parsing, array parsing, recursive values, tokenizers, Unicode escape decoding, Unicode normalization, streaming, schema validation, embedded NUL support in the current null-terminated string ABI, and stable JSON APIs deferred. +The beta.19 compiler/tooling slice adds +`glagol test --list ` and legacy +`glagol --run-tests --list ` support. The list action reuses the same +checked discovery path as normal test execution, preserves existing +single-file, project, and workspace ordering, honors `--filter `, +and avoids executing test bodies. It keeps normal test execution output +unchanged when `--list` is absent. -Next stage target: post-`1.0.0-beta.18` developer-experience, package, and -collection/generic planning. Generic vectors, generic collections, maps, sets, -generic stdlib dispatch, runtime collection changes, collection unification, -stable human diagnostic text, stable Markdown schema, LSP/watch protocols, -SARIF/daemon protocols, re-exports/globs/hierarchical modules, registry -semantics, mutable vectors, stable slice/view APIs, tokenizers, broader JSON -parsing, iterators, performance claims, ABI/layout stability, and a stable -stdlib/API compatibility freeze remain unimplemented until a later scoped -contract promotes them explicitly. +Generic vectors, generic collections, maps, sets, generic stdlib dispatch, +runtime collection changes, collection unification, stable human diagnostic +text, stable artifact-manifest or Markdown schema freezes, LSP/watch +protocols, SARIF/daemon protocols, re-exports/globs/hierarchical modules, +registry semantics, semver solving, mutable vectors, stable slice/view APIs, +tokenizers, broader JSON parsing, runtime helper names, source-language +syntax, parallel test execution, retries, tags/groups, coverage/event streams, +performance claims, ABI/layout stability, and a stable stdlib/API +compatibility freeze remain unimplemented until a later scoped contract +promotes them explicitly. The final experimental precursor scope is `exp-125`. Its unsigned direct-value flow, parse/format runtime lanes, and matching staged stdlib helper breadth diff --git a/docs/language/RELEASE_NOTES.md b/docs/language/RELEASE_NOTES.md index 98cac8f..06b2b62 100644 --- a/docs/language/RELEASE_NOTES.md +++ b/docs/language/RELEASE_NOTES.md @@ -8,7 +8,7 @@ Historical `exp-*` releases listed here are experimental maturity milestones. The pushed tag `v2.0.0-beta.1` is historical. It is now documented as an experimental integration/readiness release, not as a beta maturity claim. -The current release is `1.0.0-beta.18`, published on 2026-05-23. It keeps the +The current release is `1.0.0-beta.19`, published on 2026-05-23. It keeps the `1.0.0-beta` language surface, includes the first post-beta tooling/install hardening bundle from `1.0.0-beta.1`, and adds the first runtime/resource foundation bundle from `1.0.0-beta.2` plus the first standard-library @@ -28,11 +28,41 @@ collection alias unification and generic reservation slice from collection ledger from `1.0.0-beta.15`, plus the string scanning and token boundary foundation from `1.0.0-beta.16`, and the JSON primitive scalar parsing foundation from `1.0.0-beta.17`, plus the JSON string token parsing -foundation from `1.0.0-beta.18`. +foundation from `1.0.0-beta.18`, and the test discovery and user-project +conformance foundation from `1.0.0-beta.19`. ## Unreleased -No unreleased language scope is committed here yet. +No active unreleased language scope is documented here yet. + +## 1.0.0-beta.19 + +Release label: `1.0.0-beta.19` + +Release name: Test Discovery And User-Project Conformance Foundation + +Release date: 2026-05-23 + +Status: released beta tooling/conformance foundation on the `1.0.0-beta` +language baseline. + +The beta19 contract is tooling/conformance only. It adds deterministic test +discovery listing for: + +- `glagol test --list ` +- `glagol --run-tests --list ` for the legacy single-file test path + +List mode parses, lowers, type-checks, and discovers tests through the same +front-end path as normal test execution, then prints the discovered/selected +tests without evaluating their bodies. It preserves current single-file, +project, and workspace test ordering, honors `--filter `, and +leaves normal `glagol test` execution output unchanged. + +This release does not add source-language syntax, runtime helper names, JSON +expansion, parallel test execution, retries, tags/groups, coverage reports, +event streams, stable manifest schemas, stable Markdown schemas, LSP/watch +behavior, SARIF/daemon protocols, package registries, semver solving, or +performance claims. ## 1.0.0-beta.18 diff --git a/docs/language/ROADMAP.md b/docs/language/ROADMAP.md index 75df531..40a0d82 100644 --- a/docs/language/ROADMAP.md +++ b/docs/language/ROADMAP.md @@ -10,8 +10,9 @@ Long-horizon planning lives in release train from the historical `v2.0.0-beta.1` tag toward and beyond the first real general-purpose beta Slovo contract. -Current stage: `1.0.0-beta.18`, released on 2026-05-23 as a post-beta JSON -string token parsing foundation. It keeps the `1.0.0-beta` language +Current stage: `1.0.0-beta.19`, released on 2026-05-23 as a post-beta test +discovery and user-project conformance foundation. It keeps the +`1.0.0-beta` language contract and includes the `1.0.0-beta.1` tooling hardening release, the `1.0.0-beta.2` runtime/resource foundation release, the `1.0.0-beta.3` standard-library stabilization release, the `1.0.0-beta.4` @@ -25,7 +26,8 @@ documentation, `1.0.0-beta.12` concrete vector helper parity, benchmark suite catalog and metadata gate, `1.0.0-beta.15` reserved generic collection boundary hardening and collection ledger, and `1.0.0-beta.16` string scanning and token boundary helpers, `1.0.0-beta.17` JSON primitive -scalar token parsing, and `1.0.0-beta.18` JSON string token parsing. +scalar token parsing, `1.0.0-beta.18` JSON string token parsing, and +`1.0.0-beta.19` test discovery and user-project conformance tooling. `1.0.0-beta.16` adds `std.string` source facades and examples for `byte_at_result`, `slice_result`, `starts_with`, and `ends_with`. These helpers @@ -56,16 +58,23 @@ APIs, additional runtime names, Unicode/grapheme string semantics, timing publication, performance claims, stable benchmark JSON schema, and package registry semantics remain deferred. -Next stage target: post-`1.0.0-beta.18` continuation from -developer-experience, -package, benchmark metadata, collection, or string-processing planning without -claiming executable generics, an LSP/watch protocol, SARIF/daemon protocol, -stable Markdown schema, registry semantics, stable benchmark JSON schema, -stable `1.0.0` diagnostics freeze, stable standard-library/API compatibility -freeze, mutable vectors, language slice/view APIs, additional runtime names, -Unicode/grapheme semantics, full JSON parsing, maps/sets, iterators, or -performance claims until the exact scope is frozen from the manifest and -roadmap. +`1.0.0-beta.19` is a tooling/conformance slice, not a new source-language +feature. It adds the `glagol test --list ` command +plus legacy `glagol --run-tests --list `: parse, lower, type-check, and +discover tests through the same front-end path as normal test execution, then +list the discovered/selected tests without evaluating test bodies. The mode +preserves current single-file, project, and workspace ordering, honors +`--filter `, and leaves normal `glagol test` execution behavior +unchanged. + +The beta19 tooling scope does not claim executable generics, maps/sets, +iterators, runtime helper names, source-language syntax, JSON expansion, +parallel test execution, retries, tags/groups, coverage/event streams, +LSP/watch protocols, SARIF/daemon protocols, stable artifact-manifest or +Markdown schemas, stable benchmark JSON schema, stable `1.0.0` diagnostics +freeze, stable standard-library/API compatibility freeze, registry semantics, +semver solving, performance claims, mutable vectors, language slice/view APIs, +additional runtime names, or Unicode/grapheme semantics. The final experimental precursor scope is `exp-125`, defined in `.llm/EXP_125_UNSIGNED_U32_U64_NUMERIC_AND_STDLIB_BREADTH_ALPHA.md`. Its diff --git a/docs/language/SPEC-v1.md b/docs/language/SPEC-v1.md index d607b5a..40371cc 100644 --- a/docs/language/SPEC-v1.md +++ b/docs/language/SPEC-v1.md @@ -9,8 +9,9 @@ diagnostic catalog and schema policy update, and `1.0.0-beta.14` benchmark suite catalog and metadata gate, `1.0.0-beta.15` reserved generic collection boundary hardening and collection ledger, `1.0.0-beta.16` string scanning and token boundary foundation, `1.0.0-beta.17` JSON primitive scalar parsing -foundation, and `1.0.0-beta.18` JSON string token -parsing foundation. The language contract +foundation, `1.0.0-beta.18` JSON string token parsing foundation, and +`1.0.0-beta.19` test discovery and user-project conformance tooling. The +language contract integrates promoted language slices through `exp-125` and the historical publication baseline through `exp-123`. `1.0.0-beta` is the first real general-purpose @@ -217,6 +218,16 @@ Current v1 release surface and explicit experimental targets: APIs, recursive `JsonValue`, Unicode escape decoding/normalization, embedded NUL policy, stable ABI/layout, performance claims, and stable stdlib/API freeze remain out of scope +- `1.0.0-beta.19` test discovery and user-project conformance target: + `glagol test --list ` and legacy + `glagol --run-tests --list ` list checked/discovered tests without + executing test bodies; list mode preserves current test ordering, honors + `--filter `, and remains beta tooling rather than a stable + schema. This target does not add source-language syntax, runtime helper + names, JSON expansion, parallel test execution, retries, tags/groups, + coverage/event streams, stable artifact-manifest or Markdown schemas, + LSP/watch behavior, SARIF/daemon protocols, package registries, semver + solving, or performance claims - `exp-1` owned runtime strings: compiler-known `std.string.concat` accepts two `string` values and returns an immutable runtime-owned `string`; existing string equality, length, printing, locals, parameters, returns, and calls work diff --git a/docs/language/STDLIB_API.md b/docs/language/STDLIB_API.md index 0b85460..7b27915 100644 --- a/docs/language/STDLIB_API.md +++ b/docs/language/STDLIB_API.md @@ -6,7 +6,7 @@ Do not edit this file by hand. ## Stability Tiers - `beta-supported`: exported from `lib/std` and covered by source-search, promotion, or facade gates in the current beta line. -- `experimental`: not used for exported `lib/std` helpers in `1.0.0-beta.18`; future releases may mark new helpers this way before they graduate. +- `experimental`: not used for exported `lib/std` helpers in `1.0.0-beta.19`; future releases may mark new helpers this way before they graduate. - `internal`: helper names that are not exported from their module; they are intentionally omitted from this catalog. The catalog is a beta API discovery aid, not a stable `1.0.0` standard-library freeze. diff --git a/scripts/release-gate.sh b/scripts/release-gate.sh index 01aa069..ca21ed4 100755 --- a/scripts/release-gate.sh +++ b/scripts/release-gate.sh @@ -69,6 +69,7 @@ cargo test --test reserved_generic_collection_beta15 cargo test --test standard_string_scanning_beta16 cargo test --test standard_json_scalar_parsing_beta17 cargo test --test standard_json_string_parsing_beta18 +cargo test --test test_discovery_beta19 # Full cargo test includes unignored integration gates such as dx_v1_7, # beta_v2_0_0_beta_1, and beta_1_0_0. cargo test