slovo/benchmarks

Slovo Benchmark Suite Catalog

Release stage: 1.0.0-beta.14.

The benchmark suite is beta-scoped local tooling. It catalogs deterministic same-machine comparison scaffolds for Slovo and sibling implementations. It does not publish benchmark results, set performance thresholds, define optimizer claims, or create cross-machine comparisons.

Suite Listing

From the repository root, list the suite catalog for humans:

python3 benchmarks/runner.py --suite-list

List the same catalog as beta tooling metadata:

python3 benchmarks/runner.py --suite-list --json

The non-JSON listing is for local review. The JSON listing is for local gates and tooling adapters that need the current suite inventory. The JSON field set is not a stable public schema; it may change during beta releases. Both forms verify the required scaffold files for each suite: benchmark.json, run.py, slovo.toml, and src/main.slo.

Each suite still owns its local metadata and run commands:

python3 benchmarks/<suite>/run.py --list
python3 benchmarks/<suite>/run.py --list --json
python3 benchmarks/<suite>/run.py --dry-run

Current Suites

All current suites provide Slovo, C, Rust, Python, Clojure, and Common Lisp/SBCL source slots. Missing local toolchains are skipped by the per-suite runner where possible.

Suite	Focus	Base checksum	Hot-loop checksum	Runtime args
math-loop	arithmetic and scalar accumulation	5000001	50000001	none
branch-loop	deterministic branch-heavy integer loop	1185071	220775	none
parse-loop	repeated signed decimal i32 parsing	345000001	450000001	12345
array-index-loop	immutable fixed-array indexing	3875007	38750007	none
string-eq-loop	fixed string lookup and equality	4600001	46000001	omega
array-struct-field-loop	fixed-array access through an immutable struct field	3875011	38750011	none
enum-struct-payload-loop	enum payload matching with struct and array access	3500013	35000013	none
vec-i32-index-loop	runtime-owned (vec i32) indexing	3875007	38750007	none
vec-string-eq-loop	runtime-owned (vec string) lookup and equality	4600001	46000001	omega
json-quote-loop	compact JSON string quoting	15000001	150000001	slo"vo\path

The base loop count is 1000000 for every current suite. The hot-loop count is 10000000 for every current suite. The runner supplies loop counts and runtime arguments at execution time so native compilers cannot fold the work into a constant answer.

Local Evidence Only

Benchmark output is local-machine evidence only:

• cold-process mode measures execution after each implementation has been built once; it does not include compile time
• hot-loop mode is startup-amortized local evidence and reports total time plus normalized timing for the base loop count
• Clojure timings include JVM and Clojure startup
• Common Lisp timings include SBCL script startup
• reported timings depend on the local CPU, OS, compiler versions, toolchain availability, thermal/load state, and runner configuration

This catalog intentionally publishes no timing numbers.

Exclusions

1.0.0-beta.14 does not add benchmark kernels, publish timings, define performance thresholds, define a stable JSON schema, change the Slovo source language, change runtime behavior, change standard-library or API contracts, change diagnostic output, change ABI/layout behavior, or make cross-machine performance claims.