72 lines
2.7 KiB
Markdown
72 lines
2.7 KiB
Markdown
# JSON Quote Loop Benchmark Scaffold
|
|
|
|
Release: `1.0.0-beta.7`.
|
|
|
|
This benchmark compares compact JSON string quoting plus checksum accumulation
|
|
across Slovo, C, Rust, Python, Clojure, and Common Lisp/SBCL on the same
|
|
machine. It is same machine local evidence only.
|
|
|
|
It is not a published benchmark result, performance threshold, optimizer
|
|
claim, or cross-machine comparison.
|
|
|
|
## Files
|
|
|
|
- `src/main.slo`: Slovo project benchmark fixture
|
|
- `c/json_quote_loop.c`: C comparison implementation
|
|
- `rust/json_quote_loop.rs`: Rust comparison implementation
|
|
- `python/json_quote_loop.py`: Python comparison implementation
|
|
- `clojure/json_quote_loop.clj`: Clojure comparison implementation
|
|
- `common-lisp/json_quote_loop.lisp`: Common Lisp/SBCL comparison implementation
|
|
- `run.py`: build/run/timing harness
|
|
|
|
All implementations print checksum `15000001` for loop count `1000000` and
|
|
target string `slo"vo\path`. Hot-loop mode uses loop count `10000000` and
|
|
checksum `150000001`.
|
|
|
|
## Commands
|
|
|
|
Run from the repository root:
|
|
|
|
```bash
|
|
python3 benchmarks/json-quote-loop/run.py --list
|
|
python3 benchmarks/json-quote-loop/run.py --dry-run
|
|
python3 benchmarks/json-quote-loop/run.py --only python --repeats 3 --warmups 1
|
|
python3 benchmarks/json-quote-loop/run.py --mode hot-loop --only slovo --only c --only rust
|
|
```
|
|
|
|
To include Slovo, build or point at a Glagol binary and make sure host Clang is
|
|
available:
|
|
|
|
```bash
|
|
cargo build --manifest-path compiler/Cargo.toml --bin glagol
|
|
python3 benchmarks/json-quote-loop/run.py --glagol compiler/target/debug/glagol
|
|
```
|
|
|
|
The runner skips missing C/Rust/Slovo/Clojure/SBCL toolchains where possible.
|
|
Use `--only` multiple times to select implementations:
|
|
|
|
```bash
|
|
python3 benchmarks/json-quote-loop/run.py --only slovo --only c --only rust --only clojure --only common_lisp
|
|
```
|
|
|
|
Clojure is detected with `clojure` on PATH, `CLOJURE`, or `CLOJURE_JAR`.
|
|
Common Lisp is detected with `sbcl` on PATH, `SBCL`, or `--sbcl`.
|
|
|
|
## Comparison Method
|
|
|
|
- The runner builds each implementation once before timing. The reported
|
|
numbers measure execution only, not compile time.
|
|
- Slovo timings use `glagol build`, which currently lowers to LLVM and then
|
|
invokes host `clang -O2` with `runtime/runtime.c`.
|
|
- C timings use `clang -O2 -std=c11`.
|
|
- Rust timings use `rustc -C opt-level=3 -C debuginfo=0`.
|
|
- The measured loop quotes one runtime-supplied ASCII string containing a
|
|
quote and a backslash, then accumulates the quoted byte length.
|
|
|
|
Timing is cold-process local-machine evidence only. Clojure timings include
|
|
JVM and Clojure startup, while Common Lisp timings include SBCL script
|
|
startup.
|
|
|
|
Hot-loop mode is startup-amortized local evidence. It runs a larger loop count
|
|
and reports total time plus normalized time for the base `1000000` loop count.
|