slovo/docs/POST_BETA_ROADMAP.md

595 lines
28 KiB
Markdown

# Post-Beta Roadmap
Slovo `1.0.0-beta` is the first usable general-purpose beta baseline. The next
work should broaden the language and toolchain in connected beta slices without
pretending that ABI, package, runtime, or standard-library contracts are stable
yet.
## Direction
The best continuation is not a single feature lane. Slovo needs alternating
language, compiler, runtime, stdlib, and tooling slices so each new capability
can be used in real programs and verified by the release gate.
Each slice should ship only when it updates:
- language and compiler docs
- source examples and standard-library modules when relevant
- formatter behavior
- diagnostics and negative tests
- hosted runtime behavior when relevant
- benchmarks or conformance tests when relevant
- release notes and the public roadmap
## Immediate Sequence
### 1. Tooling And Release Hardening
Goal: make the public monorepo easy to build, test, install, and verify.
Work:
- add `glagol run` for the common build-and-execute loop
- add `glagol clean` for generated build artifacts
- improve `glagol new` templates for library, binary, and workspace projects
- document install paths for `glagol`, `runtime/`, and `lib/std`
- make release gates print a concise final summary
- keep PDF rendering explicit and non-mutating by default
Released in `1.0.0-beta.1`: `glagol run`, `glagol clean`,
`glagol new --template binary|library|workspace`, `scripts/install.sh`,
installed std/runtime discovery, README coverage, focused DX tests, and a
concise release-gate success line.
Released in `1.0.0-beta.19`: test discovery and user-project conformance
tooling. The scope adds the
`glagol test --list <file|project|workspace>` command and legacy
`glagol --run-tests --list <file>` so users and tooling can list
checked/discovered tests without executing test bodies. It preserves existing
single-file, project, and workspace ordering, honors
`--filter <substring>`, and keeps the output beta-scoped rather than a stable
public schema.
Beta19 non-scope: no parallel test execution, retries, tags/groups, coverage
reports, event streams, stable artifact-manifest or Markdown schema freeze,
LSP/watch behavior, SARIF/daemon protocols, JSON expansion, runtime helper
names, source-language syntax, package registries, semver solving, or
performance claims.
Why first: it reduces friction for every later feature and gives users a better
way to exercise the beta.
### 2. Runtime And Resource Foundation
Goal: create a safer foundation for host resources before adding networking or
larger IO APIs.
Work:
- define beta resource-handle policy for files, processes, sockets, and future
host objects
- make host errors consistently return concrete `result` values where possible
- audit allocation failure behavior for strings and vectors
- add runtime conformance tests for cleanup paths
- document that resource handles are beta values, not stable ABI promises
Released in `1.0.0-beta.2`: read-only text file resource handles are
implemented with `std.fs.open_text_read_result`,
`std.fs.read_open_text_result`, and `std.fs.close_result`. The same release
also includes narrow filesystem status and mutation calls:
`std.fs.exists`, `std.fs.is_file`, `std.fs.is_dir`,
`std.fs.remove_file_result`, and `std.fs.create_dir_result`. Matching
`lib/std/fs.slo` facades and focused runtime tests are in place. Writable
handles, directory handles/enumeration, process handles, sockets, platform
error codes, and async resources remain deferred.
Why second: networking, file IO, process IO, and HTTP all need a common story
for handles, errors, cleanup, and platform variance.
### 3. Standard Library Stabilization Pass
Goal: make existing `lib/std` modules feel coherent before adding many new
surface areas.
Work:
- mark each public helper as beta-supported, experimental, or internal
- normalize naming across `std.string`, `std.num`, `std.math`, `std.io`,
`std.fs`, `std.env`, `std.process`, `std.cli`, `std.time`, and `std.random`
- add examples that compose multiple modules in one realistic command-line
program
- add stdlib documentation generation coverage
- identify helpers that should wait for generics instead of being copied across
concrete type families
Released in `1.0.0-beta.3`: `docs/language/STDLIB_API.md` is
generated from `lib/std/*.slo` and guarded by `scripts/release-gate.sh`.
`examples/projects/stdlib-composition` adds a realistic command-line project
that composes `std.fs`, `std.string`, `std.math`, and `std.io` through explicit
standard imports, with focused check/test/doc/run coverage.
Released in `1.0.0-beta.20`: source-authored `std.string` search and ASCII
trim helpers. The scope adds `contains`, `index_of_option`,
`last_index_of_option`, `trim_ascii_start`, `trim_ascii_end`, and
`trim_ascii` over existing byte-oriented string primitives, with explicit
`std.string` import examples and focused compiler gates. Empty needles match
at first index `0` and last index `(len value)`; ASCII trimming removes only
bytes `9`, `10`, `11`, `12`, `13`, and `32`. Unicode/grapheme semantics,
case folding, regexes, tokenizers, mutable strings, slice/view syntax, new
runtime names, and stable stdlib/API promises remain deferred.
Released in `1.0.0-beta.23`: the public
[`docs/language/STDLIB_TIERS.md`](language/STDLIB_TIERS.md) ledger defines the
current standard-library tier labels `beta-supported`, `experimental`, and
`internal`, and aligns the docs around the generated
[`docs/language/STDLIB_API.md`](language/STDLIB_API.md) signature catalog.
JSON, loopback networking, random/time, and filesystem resource-handle helpers
are documented as experimental domains. Concrete vector modules remain
beta-supported concrete lanes, not a generic collections freeze. The slice is
documentation/catalog tooling clarity only: no syntax, helper, runtime,
manifest-schema, Markdown-schema, ABI/layout, or stable stdlib/API behavior
changes. It updates generated catalog output and the release gate so tier
metadata is visible and checked.
Why third: stdlib growth is already broad enough that naming and stability tiers
matter more than adding another isolated helper group.
### 4. Language Usability Slice
Goal: reduce awkwardness in common programs without destabilizing the typed core.
Candidate features:
- `glagol run`-friendly `main` conventions and clearer entry-point diagnostics
- better `match` diagnostics and exhaustiveness checks where the current enum
model allows it
- concrete type aliases for long concrete types such as vectors, options, and
results
- destructuring for simple structs and enum payloads
- additional numeric completeness such as `f32`, only if it can be tested across
checker, formatter, runtime, and docs
Why fourth: these features improve real Slovo code while keeping generics and
traits deferred until the core has more feedback.
Released in `1.0.0-beta.4`: project/workspace build and run entry
diagnostics now use entry-specific codes and explicitly show the required
`(fn main () -> i32 ...)` contract. Non-exhaustive `match` diagnostics now use
clearer missing-arm wording and deterministic found-arm output.
Concrete aliases were split into the follow-up `1.0.0-beta.8` language slice so
the syntax, formatter, diagnostics, and source fixtures could be gated directly.
### 5. Package And Workspace Discipline
Goal: make multi-package local development predictable before remote registry
work.
Work:
- document package identity, version, and local dependency rules
- decide whether a lockfile belongs in the beta package model
- add project-wide `fmt`, `check`, `test`, `doc`, `build`, and `run` behavior
- add diagnostics for ambiguous package roots and dependency cycles
- keep remote registry, semver solving, and publishing out of scope
Released in `1.0.0-beta.5`: local workspaces can declare `default_package` to
select the build/run entry package when multiple packages have entry modules.
Duplicate normalized member paths and missing default-package references are
dedicated diagnostics. Workspace documentation now includes package and local
dependency summaries, generated workspace templates declare
`default_package = "app"`, and `docs/language/PACKAGES.md` records the beta
local-package rules. Lockfiles, remote registries, semver solving, publishing,
optional/dev/target dependencies, and stable package ABI/layout remain out of
scope.
Released in `1.0.0-beta.24`: package manifest identity and dependency
diagnostics are tightened without changing the package model. Duplicate
package manifest keys, invalid dependency keys, and duplicate dependency keys
are explicit diagnostics. The slice adds no remote registry, lockfile,
semantic-version solving, package publishing, optional/dev/target
dependencies, stable package ABI/layout, source-language change, runtime
change, or standard-library change.
Why fifth: stable package rules are a prerequisite for a usable public language,
but remote publishing can wait.
### 6. Networking Foundation
Goal: add minimal blocking local networking after resource and error policy is
in place.
Work:
- add `std.net` with blocking TCP client/server primitives
- use loopback-only deterministic tests
- expose host failures through result values
- keep TLS, async, DNS policy, HTTP server frameworks, and event loops deferred
- add small examples such as echo client/server or local request/response
Released in `1.0.0-beta.6`: `lib/std/net.slo` now provides explicit wrappers
for loopback TCP connect, listen, bound-port lookup, accept, read-all,
write-text, and close result calls. The source fixtures use invalid port/handle
checks for deterministic result-shape coverage, and the compiler/runtime tests
cover lowering plus hosted loopback client/server smoke when the local sandbox
allows loopback sockets. DNS, TLS, UDP, async IO, non-loopback binding, HTTP
frameworks, rich host-error ADTs, and stable socket ABI/layout remain deferred.
Why sixth: networking is useful, but doing it before resources and host errors
would create unstable API debt.
### 7. Serialization And Data Interchange
Goal: make Slovo useful for command-line and network programs without requiring
maps/sets immediately.
Work:
- add string scanning/tokenizing helpers
- add simple JSON text construction helpers first
- defer full JSON object parsing until maps, richer strings, or generic
collections exist
- add benchmark cases for parsing and formatting
Released in `1.0.0-beta.7`: `lib/std/json.slo` now provides explicit helpers
for compact JSON text construction over strings, booleans, numbers, null,
fields, small arrays, and small objects. `std.json.quote_string` is a
compiler-known runtime helper so JSON string escaping is correct before Slovo
has later byte-oriented string scanning helpers. Matching explicit
std/local source fixtures and a `json-quote-loop` benchmark scaffold are in
place.
Released in `1.0.0-beta.17`: `lib/std/json.slo` now provides primitive scalar
JSON token parse facades for booleans, concrete numeric primitives, and exact
`null`. Broader JSON parsing beyond primitive scalar tokens remained deferred
for the next slices.
Released in `1.0.0-beta.18`: `lib/std/json.slo` adds
`parse_string_value_result` for one already-isolated ASCII JSON string token.
It requires exact quotes, rejects leading/trailing whitespace, decodes the
simple JSON escapes `\"`, `\\`, `\/`, `\b`, `\f`, `\n`, `\r`, and `\t`, and
returns `err 1` for ordinary parse failure. Full JSON document parsing,
object/array parsing, tokenizer objects, Unicode escape decoding or
normalization, embedded NUL policy, streaming, schema validation, and stable
ABI/API guarantees remain deferred.
Released in `1.0.0-beta.21`: `lib/std/json.slo` adds source-authored scalar
document parse facades for string, bool, `i32`, `u32`, `i64`, `u64`, `f64`,
and exact `null`. Each helper trims ASCII whitespace around the whole document
with `std.string.trim_ascii`, then delegates to the already released exact
value-token parser. This intentionally remains scalar document parsing only:
object/array parsing, recursive JSON values, parser/tokenizer objects,
maps/sets, streaming, new compiler-known runtime names, broader Unicode escape
policy, embedded NUL policy, and stable ABI/API guarantees remain deferred.
Why seventh: networking and CLI tools need data interchange, but a complete JSON
library depends on collection work.
### 8. Concrete Type Alias Foundation
Goal: reduce concrete type repetition without introducing generics or changing
runtime representation.
Work:
- add top-level `(type Alias TargetType)` declarations for aliases whose targets
are already supported concrete Slovo types
- resolve aliases before typed-core lowering, checked import signatures, backend
layout, ABI decisions, and runtime behavior
- keep aliases module-local: no alias exports, imports, re-exports, or
cross-module alias visibility
- update formatter and diagnostics for malformed, duplicate, unsupported,
cyclic, exported, or imported aliases
- exercise aliases sparingly in JSON source facades and explicit source
fixtures without adding compiler-known runtime names
Released in `1.0.0-beta.8`: concrete aliases such as `(type JsonText string)`
are transparent names for existing concrete types. The compiler parses,
formats, checks, lowers, and erases aliases before backend behavior, while
project imports of functions that used local aliases see concrete target
types. Alias export/import attempts and unsupported targets are diagnostics.
Generic aliases, parameterized aliases, aliasing maps/sets, stable ABI/layout
names, and runtime changes remain deferred.
Why eighth: concrete aliases remove real noise from current stdlib and fixture
code while deliberately postponing generic type parameters until the compiler
and standard library have stronger design pressure.
### 9. Collection Alias Unification And Generic Reservation
Goal: apply concrete aliases to the existing collection/value facades and
reserve the generic/map/set direction without changing executable semantics.
Work:
- use beta.8 concrete aliases sparingly inside current vector, option, and
result source facades to reduce repeated long concrete types
- preserve all public helper names, exports, imports, runtime calls, and
concrete cross-module signatures
- document that current vectors/options/results are still concrete families
and that local aliases erase before lowering
- reserve executable generics, maps, sets, traits, inference,
monomorphization, iterators, and ABI stability for later gated releases
Released in `1.0.0-beta.9`: the concrete `std.vec_i32`, `std.vec_i64`,
`std.vec_f64`, `std.vec_bool`, `std.vec_string`, `std.option`, and
`std.result` facades now use module-local transparent aliases internally.
The exported helper surface remains concrete after alias normalization.
The release does not implement executable generics, maps, sets, traits,
inference, monomorphization, iterators, or stable ABI/layout promises.
Why ninth: generics are important, but they should be introduced after the
compiler, docs, tests, and stdlib have enough real pressure to validate the
design.
### 10. Developer Experience API Discovery
Goal: make Slovo comfortable for repeated daily use by making the current
standard-library API surface easier to inspect before deeper editor work.
Work:
- upgrade the generated `lib/std` API catalog from exported names to exact
exported helper signatures
- normalize module-local beta.8/beta.9 concrete aliases in public signatures
so local aliases do not leak into docs
- validate that exported helpers have matching `(fn ...)` forms
- keep non-exported helpers and `(type ...)` aliases out of the public catalog
- language-server diagnostics and document symbols
- editor-facing symbol metadata for files, projects, and workspaces
- project watch mode
- clearer benchmark harness output
- machine-readable diagnostics stability policy
Released in `1.0.0-beta.10`: `docs/language/STDLIB_API.md` now lists exact
exported helper signatures from `lib/std/*.slo`, and the renderer validates
exported helper forms while normalizing module-local aliases to concrete
public types. `glagol symbols <file.slo|project|workspace>` now emits
deterministic `slovo.symbols` metadata for editor integrations without
starting an LSP server. This is beta API discovery only; it does not add
executable generics, maps, sets, runtime changes, or a stable standard-library
API freeze. LSP, watch mode, benchmark-output work, stable Markdown schema,
stable stdlib/API compatibility freeze, SARIF/daemon protocols, and a
machine-readable diagnostics stability policy remain deferred.
Why tenth: editor support matters, but it should follow a stable enough syntax,
project model, and diagnostic model.
### 11. Local Package API Documentation
Goal: extend beta API discovery from `lib/std` and symbol metadata to the
local packages and modules users document with `glagol doc`.
Work:
- make `glagol doc <file|project|workspace> -o <dir>` include deterministic
exported/public API sections for local packages and modules
- list exact exported function signatures
- list exported struct fields
- list exported enum variants and payload types
- keep non-exported functions, structs, enums, tests, and aliases out of the
public API sections
- normalize module-local concrete aliases in public docs so local alias names
do not leak across module/package boundaries
- keep Markdown layout and generated file names beta-scoped rather than stable
Released in `1.0.0-beta.11`: local file, project, package, and workspace docs
generated by `glagol doc <file|project|workspace> -o <dir>` include
deterministic public API sections with exact exported function signatures,
exported struct fields, exported enum variants/payload types, non-export
filtering, and module-local alias normalization. This extends beta10 API
discovery only; it does not freeze the Markdown schema, create a stable
stdlib/API compatibility freeze, add LSP/watch, define SARIF/daemon protocols,
set a diagnostics schema policy, implement executable generics/maps/sets, add
re-exports/globs/hierarchical modules, or define registry semantics.
Why eleventh: local packages are useful only if their public surface can be
reviewed without reading every source file, but the documentation format
should remain flexible until the package and editor stories are stronger.
### 12. Concrete Vector Query And Prefix Parity
Goal: close small source-authored helper gaps in the current concrete vector
facades before returning to larger language and tooling slices.
Work:
- add `count_of`, `starts_with`, `without_prefix`, `ends_with`, and
`without_suffix` to `std.vec_i64`
- add `count_of` to `std.vec_f64`
- keep all helpers source-authored over the already promoted concrete vector
runtime names, equality, `len`, `at`, `take`, `drop`, and recursive helper
style
- add explicit local helper project coverage for repeated count results and
prefix/suffix empty, mismatch, exact, and longer-than-input cases where
applicable
- document the slice as helper parity only, not a language/runtime change
Released in `1.0.0-beta.12`: `std.vec_i64` gains `count_of`, `starts_with`,
`without_prefix`, `ends_with`, and `without_suffix`; `std.vec_f64` gains
`count_of`; and focused Glagol fixture tests require the corresponding explicit
source-helper coverage. The release does not add generics, maps/sets,
iterators, mutable vectors, slice/view APIs, new runtime names, ABI/layout
stability, performance claims, or a stable stdlib/API freeze.
Why twelfth: concrete vectors are already broad enough that parity gaps create
surprising differences, and source-authored helpers can close those gaps
without committing to generic collection design.
### 13. Diagnostic Catalog And Schema Policy
Goal: document the existing diagnostic machine contract before larger tooling
or editor-facing slices depend on it.
Work:
- add [`docs/language/DIAGNOSTICS.md`](language/DIAGNOSTICS.md) as the beta
`slovo.diagnostic` version `1` policy
- document the S-expression and JSON encodings, required and optional fields,
severity/source/range/related-span semantics, JSON-line discipline,
source-less diagnostics, and artifact-manifest diagnostic metadata
- classify diagnostic changes as clarifying, additive, or migration-level,
with human prose remaining beta-flexible unless machine fields, schema
markers, codes, or golden fixture shape change intentionally
- inventory the current diagnostic codes covered by
`compiler/tests/diagnostics_contract.rs` and the matching `.diag` snapshots
- keep LSP/watch, SARIF, daemon protocols, stable Markdown schema, stable
`1.0.0` diagnostics freeze, and runtime/source-language changes out of scope
Released in `1.0.0-beta.13`:
[`docs/language/DIAGNOSTICS.md`](language/DIAGNOSTICS.md) now defines the beta
diagnostic schema policy and catalogs the current golden diagnostic codes. The
release is documentation/tooling policy only; it does not change Glagol
diagnostic output, the source language, runtime, stdlib/API surface, or
ABI/layout behavior.
Why thirteenth: diagnostics already have a machine schema and broad golden
coverage, but the compatibility policy and current code inventory need one
central reference before future tooling or migration work builds on them.
### 14. Benchmark Suite Catalog And Metadata Gate
Goal: document the existing benchmark suite inventory and metadata listing
commands before benchmark tooling grows additional consumers.
Work:
- add [`benchmarks/README.md`](../benchmarks/README.md) as the top-level
benchmark suite catalog
- document `python3 benchmarks/runner.py --suite-list` for the non-JSON suite
inventory
- document `python3 benchmarks/runner.py --suite-list --json` for beta tooling
metadata
- list the current suite inventory without adding new benchmark kernels
- state that benchmark timings are local-machine evidence only
- keep suite-list JSON beta-scoped rather than a stable public schema
- keep timing publication, performance thresholds, source-language/runtime/
stdlib/API/diagnostic-output changes, and ABI/layout changes out of scope
Released in `1.0.0-beta.14`:
[`benchmarks/README.md`](../benchmarks/README.md) now catalogs the current
benchmark suite, documents the root suite-list commands, records local-machine
evidence policy, and names the explicit exclusions. The release is
documentation/tooling metadata only; it does not add kernels, publish timing
numbers, define performance thresholds, define a stable JSON schema, or change
the source language, runtime, stdlib/API surface, diagnostics, or ABI/layout
behavior.
Why fourteenth: the benchmark suite is already part of the public monorepo, but
its suite-level inventory and metadata boundary need one central reference
before future tooling can rely on it.
### 15. Reserved Generic Collection Boundary Hardening And Collection Ledger
Goal: document the current concrete collection and value-family boundary before
executable generics, maps, sets, iterators, mutable vectors, or slice/view APIs
are designed as executable features.
Work:
- add [`docs/language/COLLECTIONS.md`](language/COLLECTIONS.md) as the
collection/value-family ledger
- link to the generated [`docs/language/STDLIB_API.md`](language/STDLIB_API.md)
catalog for exact public helper signatures instead of duplicating generated
counts
- inventory the current concrete vector, option, result, and related
option/result-returning facade surfaces
- record design pressure from duplicated concrete vector, option, and result
helper families
- define prerequisites before executable generics, generic aliases, maps,
sets, iterators, mutable vectors, or slice/view APIs can be promoted
- state that current unsupported diagnostics are boundaries, not behavior
changes
- centralize reserved generic/map/set diagnostics and reword affected
reserved-boundary messages from beta.9-specific text to current-beta wording
- keep source-language/runtime/stdlib/API changes, diagnostic output shape/
code/schema/span/expected/found/hint changes, benchmark metadata schema
changes, ABI/layout changes, performance claims, and stable API freeze out
of scope
Released in `1.0.0-beta.15`:
[`docs/language/COLLECTIONS.md`](language/COLLECTIONS.md) now records the
collection/value-family ledger and links to the generated standard-library API
catalog for exact signatures. The release also centralizes reserved
generic/map/set diagnostics and rewords affected reserved-boundary messages
away from beta.9-specific text. It does not change the source language,
runtime, stdlib/API surface, diagnostic output shape, diagnostic codes,
diagnostic schema, spans, expected/found values, hints, benchmark metadata
schema, ABI/layout behavior, or performance claims, and it does not create a
stable stdlib/API freeze.
Why fifteenth: the concrete vector, option, and result facades have enough
duplication to justify generic collection planning, but the public contract
needs an explicit boundary ledger before executable generic, map, set,
iterator, mutable-vector, or slice/view semantics are promoted.
### 16. String Scanning And Token Boundary Foundation
Goal: add the first byte-oriented string scanning and token-boundary helpers
without promoting Unicode/grapheme semantics, full JSON parsing, or language
slice/view syntax.
Work:
- add `byte_at_result`, `slice_result`, `starts_with`, and `ends_with` to
`lib/std/string.slo`
- mirror those source facades in the explicit local `std-layout-local-string`
fixture
- cover both local and explicit `std.string` imports with examples for success
and ordinary failure cases
- document byte-oriented behavior over current NUL-terminated runtime strings
- require invalid indexes and ranges to return `err 1`
- keep allocation failure on substring creation aligned with the existing
string allocation trap policy
- keep Unicode scalar/grapheme/display-width semantics, full JSON parsing,
object/array parsing, tokenizer objects, language slice/view features,
mutable strings, stable ABI/layout, and stable API freeze out of scope
Released in `1.0.0-beta.16`: `std.string` now exposes source facades and
examples for byte access, substring extraction, and prefix/suffix checks:
`byte_at_result`, `slice_result`, `starts_with`, and `ends_with`. The release is
byte-oriented over the current runtime string representation and uses `err 1`
for ordinary invalid indexes/ranges. It does not add Unicode/grapheme
semantics, full JSON parsing, object/array parsing, tokenizer objects,
language slice/view features, mutable strings, stable ABI/layout, or a stable
stdlib/API freeze.
Why sixteenth: JSON text construction and process/file/network helpers can
produce useful strings, but parsers and tokenizers need a smaller byte-boundary
foundation before richer string or data-interchange APIs are credible.
## Stable `1.0.0` Gate
Slovo should not become stable until all of these are true:
- migration and deprecation policy is documented
- `lib/std` has explicit beta-supported, experimental, and internal tiers plus
a later stable-tier/deprecation policy before `1.0.0`
- package/workspace behavior is deterministic
- package manifest identity and dependency-key failures have explicit
diagnostics
- conformance tests cover user-shaped projects
- release gates are reproducible on a clean checkout
- diagnostics and formatter output are stable for promoted features
- performance publications are repeatable and labeled as local-machine evidence
- benchmark metadata promoted to stable, if any, has an explicit schema policy
- the language can build useful local CLI tools, libraries, file-processing
programs, and basic host-interaction programs without undocumented behavior
## Explicitly Deferred
These should not be early post-beta work unless a smaller prerequisite slice is
complete first:
- full async runtime
- TLS and certificate policy
- remote package registry
- macro system
- stable C ABI/layout guarantees
- optimizing compiler claims
- mutable vectors, slice/view APIs, iterators, maps, sets, and executable
generics
- new runtime helper names or generic stdlib dispatch before an explicit
runtime/language slice
- web framework or HTTP server framework
- broad Unicode/string normalization policy