Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

• Project website at https://zoosky.github.io/driller, built with Accent CMS from the repository's own markdown (docs/, SYNTAX.md, example/README.md, FORK.md, CONTRIBUTING.md, CHANGELOG.md) plus a landing page and a getting-started guide under site/. The new Pages workflow builds it on every push to main (and checks links on pull requests) and publishes the output to GitHub Pages. The mounted markdown files gained a short frontmatter block that sets each page's title, summary, and position in the docs navigation. site/ is excluded from the crates.io package.

Changed

• A failed request now prints a concise, classified line instead of a raw reqwest::Error Debug dump. The line matches the success line format (step name, URL, then a red ERR <cause> marker where the status would be) -- for example ERR request timed out, ERR connection refused, ERR DNS resolution failed, or ERR TLS error. The cause is classified from reqwest's own predicates plus a walk of the error's source() chain (no new dependency). Under --verbose the full underlying source() chain is appended on a dimmed cause: line, so the detail from the old dump is still available on demand. The suppression gate (--quiet), the synthetic 520 recorded for --stats, and the status-code breakdown are all unchanged.
• Internal: the library error type (driller::Error) now derives its Display/Error implementations with thiserror instead of hand-written impl blocks. User-facing messages and the source() chain are byte-for-byte unchanged, and the public error API is identical (non-breaking). thiserror was already present transitively (via reqwest's hickory-dns feature), so this adds no new crate to the build.

Security

• Bump h2 0.4.14 -> 0.4.19 to clear RUSTSEC-2026-0258, in which the HTTP/2 implementation accepted and queued empty DATA frames without limit -- unbounded memory growth on a stream that is not drained, or a panic from an integer overflow. Pulled in transitively via reqwest, and reachable on any run whose target negotiates HTTP/2, so a hostile or malfunctioning target server could exhaust the load generator. Lockfile-only change; no driller source is affected.

0.13.0 - 2026-07-03

Added

• --stats-format <text|json> selects the statistics output format (default text, unchanged). --stats-format json emits the same global and per-step statistics as a single JSON document to stdout -- and nothing else on stdout: the run banner and any warnings are routed to stderr and per-request progress is suppressed (as with --quiet), so driller run ... --stats-format json | jq . always sees valid JSON. It implies --stats. The document carries a top-level integer schema version; each bucket reports totals, a status_counts map (exact code to count, including the synthetic 520), a class_rollup (2xx/3xx/4xx/5xx plus a separate connection_errors for 520), and latency_ms percentiles as raw milliseconds (--nanosec does not affect the JSON). --report is unaffected; --stats-format json cannot be combined with --compare (both write to stdout), the same restriction --stats already has.
• driller run - reads the ad-hoc target URL from standard input, so a single-endpoint load test composes in a shell pipeline -- for example echo http://localhost:9000/health | driller run - --duration 10s --stats. The URL is the first non-empty line of stdin (trimmed of surrounding whitespace and a leading UTF-8 BOM) and runs through the same synthetic-GET path as driller run <URL>. Because - is an ad-hoc source it cannot be combined with --benchmark (that pairing exits 1 with a clear message); empty stdin prints the standard error: either a URL or --benchmark is required; and unreadable or non-UTF-8 stdin exits 1 with error: couldn't read URL from stdin: ... rather than a misleading missing-URL message.

Changed

• The ad-hoc runner now rejects a target URL without a scheme (for example example.com) up front with error: URL must include a scheme, e.g. http://example.com and exit 1, instead of building a malformed base URL that fails every request. Applies to a positional driller run <URL>, a URL piped via driller run -, and a --base-url override; driller drives HTTP(S) only.
• Library API (source-breaking): RunOptions (re-exported from the crate root) gains a public machine_readable field, so downstream code that constructs it with a struct literal must set the new field. This is why 0.13.0 is a minor rather than a patch: the library surface is not source-compatible with 0.12.x, even though every CLI flag and plan file is.

0.12.0 - 2026-07-01

Added

• A reusable library target (src/lib.rs). driller was a binary-only crate; the load-testing engine is now exposed as a library with a single public entry point, driller::run(&RunOptions) -> Result<BenchmarkResult, driller::Error>, alongside the actions, benchmark, checker, config, expandable, reader, and tags modules. Integration tests, benchmarks, and sibling crates can drive the engine directly instead of shelling out to the binary. The CLI (src/main.rs) keeps its behavior and now drives the engine through this public API.

Changed

• The library no longer calls std::process::exit on bad input. The engine (driller::run and the reader/config/tags/checker helpers) now returns a typed driller::Error for an unreadable or malformed plan, an invalid configuration, an empty plan, an empty/malformed --compare baseline, and an empty tag listing, leaving the exit decision to the binary. Embedding driller as a crate no longer risks having the whole host process terminated from deep inside the engine. Fatal-input messages printed by the CLI are now uniformly prefixed with error: (for example a missing --benchmark file still prints error: couldn't open <path>: ... and exits 1, with no Rust panic backtrace).

Fixed

• driller --version now embeds the correct commit hash for installs from crates.io. The hash was derived only from git rev-parse, which returns nothing when building the published tarball (it carries no .git), so cargo install driller reported unknown (and once printed a bare 0.11.0 ()). The publish step now writes the release commit's short hash into the packaged tarball and build.rs prefers it, so an installed binary is traceable back to its source. The hash is still resolved via git rev-parse first in a normal checkout, so developer builds are unaffected.

Security

• Bump quinn-proto 0.11.14 -> 0.11.15 to address RUSTSEC-2026-0185, a high-severity remote memory exhaustion from unbounded out-of-order QUIC stream reassembly. Pulled in transitively via reqwest's optional QUIC support; driller is not directly affected, but the bump keeps --locked builds and the security audit clean.
• Bump anyhow 1.0.102 -> 1.0.103 to clear RUSTSEC-2026-0190, an unsoundness in Error::downcast_mut(). Present only transitively through build-time wit-bindgen/getrandom tooling; driller does not use anyhow directly.

0.11.1 - 2026-06-01

Changed

• --report now runs the full benchmark and writes every request (all iterations, in completion order) to the report file, honoring concurrency/iterations/duration like any other run. Previously report mode executed a single hard-coded iteration and ignored those properties, so the report captured only one request per plan step (fcsonline/drill#87). --report composes with --stats, which now reports over the full run.
• --compare now averages both the baseline and the current run per request name and compares each name's mean duration, instead of comparing by position in the file. This keeps the verdict stable regardless of iteration count or the order concurrent iterations finished in (with concurrency > 1 positions are not reproducible). A request with no matching baseline name is skipped, records missing a name/duration are skipped rather than panicking, and an empty or malformed baseline file now exits with a clean error instead of silently reporting success.

Fixed

• A failed assert no longer aborts the run with a Rust panic and backtrace hint. Instead, driller prints a single FAIL: <key> -- expected <x>, got <y> line, continues the run so any remaining checks still report, and finishes with a non-zero exit code so CI can detect the failure. A passing run still exits 0. Strict-equality semantics are unchanged.
• --stats --report together no longer prints NaN requests-per-second and an all-zero stats block. Report mode now produces real timing data, and the requests-per-second divide is guarded against a zero-duration run (fcsonline/drill#87).
• --report no longer silently writes an empty file when a run completes no requests (e.g. a plan with no request items, or a --duration shorter than a single request); it prints a warning and skips the write instead.
• {{ index }} now resolves in a plain request (one with no with_items/with_items_range/with_items_from_csv/with_items_from_file). Previously it only existed inside those expansions, so a plain plan that referenced {{ index }} panicked in the default strict mode ("Unknown 'index' variable") or interpolated to an empty string under --relaxed-interpolations (fcsonline/drill#186). In a plain request index is the iteration counter; inside an items expansion it remains the item's position in the list.

Added

• Documented the built-in interpolation variables (base, index, iteration, item) in SYNTAX.md.

0.11.0 - 2026-05-31

Changed

• Request latency now measures time-to-last-byte. driller reads the full response body before stopping its timer, matching wrk, k6, vegeta and other load-testing tools. Previously the timer stopped as soon as the response headers arrived, and the body was only read when a request used assign, so endpoints serving non-trivial bodies (files, large JSON) were reported as far faster than they really were (fcsonline/drill#74). Reported latencies for body-heavy endpoints will increase to reflect true end-to-end time.
  • Because the body read is now part of the timed request, a response whose body does not finish transferring within --timeout is reported as a connection error (synthetic status 520 / the conn total) rather than its HTTP status. Previously such a request reported its status (e.g. 200) because the body was never read. Loosen --timeout if body transfer for a slow or large-bodied endpoint legitimately needs more time.
  • assign bodies are decoded using the response's Content-Type charset (defaulting to UTF-8), preserving the previous charset-aware behaviour now that driller drains the body itself instead of calling reqwest's text().
  • The body is streamed and discarded chunk by chunk; it is only buffered in memory when a request uses assign. Peak memory therefore stays bounded per in-flight request rather than scaling with the full response size, so testing large-body endpoints at high concurrency does not balloon memory.
• In --duration mode, iterations that complete before the deadline are now counted even when the deadline falls mid-batch; previously the entire in-flight batch was discarded when the duration elapsed. Only requests still in flight at the deadline are dropped.
• In --verbose mode, connection and body-read failures now also print the <<< response marker (with no body), so failed requests are visible in the request/response log instead of only the inline error line.
• cargo-deny now bans native-tls, openssl, and openssl-sys, so CI fails if OpenSSL is ever pulled back into the dependency tree. TLS stays on rustls; this guards against the prebuilt-musl OpenSSL segfault class (fcsonline/drill#168, #190).

0.10.3 - 2026-05-30

Added

• --stats output now includes a per-status-code breakdown: each HTTP status mapped to its request count, followed by a 2xx/3xx/4xx/5xx class rollup. The synthetic status 520 is labelled as a connection error and reported as a separate conn total (not folded into 5xx), so dropped connections are distinguishable from server 5xx responses (e.g. example/benchmark.yml now shows its 202 "failures" as 200 expected 404s + 2 flaky 500s). With --verbose each plan step also prints a compact per-step breakdown.

Changed

• Example server is now a small Rust (axum) binary at example/server, serving the responses/ fixtures and a few dynamic endpoints; running the examples needs only cargo. The previous Node/Express example server, its npm dependency tree, and its Docker files were removed.
• CI: a new examples job builds the example server and runs every standalone example/*.yml plan against it, gating on a clean exit and no connection errors -- turning the example suite into a regression test.

Fixed

• --version no longer prints an empty commit hash in release binaries (e.g. driller 0.10.2 ()). build.rs now requires a successful, non-empty git rev-parse and otherwise falls back to $GITHUB_SHA (then unknown), so CI-built binaries always embed a real commit identifier. A Cross.toml passes GITHUB_SHA into the musl container build for the same reason.
• Release workflow: build the x86_64-apple-darwin target on macos-latest (Apple-silicon, cross-compiling) instead of the frequently-unavailable macos-13 runner, which had left the Intel macOS asset missing from the 0.10.2 release.
• example/headers.yml: corrected the base URL port (3000 -> 9000) so the custom-headers example reaches the example server instead of failing with connection-refused.
• example/benchmark.yml: fixed the CSV quote_char ("\'" -> "'", which had decoded to a backslash) so the CSV-driven POST step issues requests instead of silently parsing nothing; corrected the matching quote_char example in SYNTAX.md.

Documentation

• README.md: document the --worker-threads / -w flag and link docs/cli-reference.md for the full flag list and the runtime workload-tuning guide.
• example/README.md: document running the examples against the Rust server with driller run --benchmark … --stats.

0.10.2 - 2026-05-29

Added

• driller run --worker-threads N (short -w N): selects the tokio runtime. N = 1 (default) uses the current-thread runtime; N >= 2 uses the multi-thread runtime with N worker threads. N = 0 is rejected at CLI parse time. See docs/cli-reference.md for the workload-vs-N guidance table.

Changed

• Default tokio runtime is now explicitly current_thread. This matches the behavior every previous release shipped with -- the prior derivation min(num_cpus, concurrency) was paired with Builder::new_current_thread(), which silently ignored the computed worker count. Behavior is therefore identical for users who do not pass --worker-threads; only the manifest is now honest.

Removed

• num_cpus dependency. No longer needed now that the worker count is taken directly from the CLI flag.

Changed

• actions::request: shrink the connection-pool Mutex window to cover only the HashMap lookup and a cheap reqwest::Client clone (the inner state is Arc-shared). The per-request RequestBuilder is now constructed after the lock is released. Originally pursued as a candidate fix for a multi-thread-runtime throughput regression at moderate response sizes; a clean-machine sweep of the patched binary did not show the regression closing, so this lands as a cleanup rather than a perf fix.
• Cargo.toml: declare tokio's rt and rt-multi-thread features explicitly. The runtime builder requires both, and they were previously available only via reqwest's transitive feature enablement.

Added

• Cargo.toml: a profiling cargo profile that inherits from release and keeps debug symbols (debug = true, strip = false). Use with cargo build --profile profiling or cargo install --path . --profile profiling --force for samply / instruments stack walking. Default release build is unchanged.

0.10.1 - 2026-05-28

Fixed

• --threshold rejects non-numeric values at CLI parse time instead of after running the benchmark, with an error that hints at the bundled-short-flags gotcha (e.g. -stats is parsed as -s -t ats, not as --stats).
• File-not-found and YAML/CSV parse errors in reader.rs are now reported as clean error: ... lines on stderr with exit code 1, instead of Rust panics with backtrace hints. Affects --benchmark, --compare, and any benchmark step that reads an iterate / csv source file.

Changed (release pipeline)

• Release workflow rewritten to use taiki-e/upload-rust-binary-action, replacing the previous Docker-only action that failed on macOS and Windows runners. The 0.10.0 release shipped without binary assets as a result; 0.10.1 restores cross-platform artifacts for x86_64-unknown-linux-musl, x86_64-apple-darwin, aarch64-apple-darwin, and x86_64-pc-windows-msvc.

0.10.0 - 2026-05-28

Added

• driller run <URL> subcommand for ad-hoc HTTP testing without a benchmark file
• CLI override flags: --concurrency, --iterations, --duration, --rampup, --base-url
• Duration-based runs (--duration 30s) that loop the plan for a fixed wall-clock period
• Three-layer config precedence: hard-coded defaults < YAML file < CLI flags
• docs/cli-reference.md with full CLI documentation

Changed

• benchmark::execute() accepts a RunOptions struct instead of positional parameters
• Tags struct owns its data (removed lifetime parameter)
• Synthetic plan built programmatically via Request::simple_get instead of YAML construction
• Duration loop bounded by tokio::time::timeout to prevent overshooting the deadline
• Terminal output colors changed from purple to cyan
• Concurrency > iterations validation produces a clear error message instead of a panic
• checker::compare() accepts threshold as f64 (parsed at CLI boundary)
• Positional URL split into base and path components for correct request targeting
• README quick-start section tightened; example updated to use run subcommand
• Upgrade reqwest 0.12 to 0.13, bump MSRV to 1.95
• Upgrade colored 2 to 3, rand 0.8 to 0.10
• Add cargo-deny configuration for license and advisory auditing

Fixed

• Histogram panic on response durations above 3.6 seconds (upper bound raised to 1 hour)
• Duration mode no longer overshoots by a full batch latency

Added (infrastructure)

• SECURITY.md, CONTRIBUTING.md, issue templates, CODEOWNERS
• Cross-platform release workflow (Linux, macOS, Windows)
• Security audit and cargo-deny CI checks

0.10.0-alpha.2 - 2026-05-25

Changed

• Upgrade clap 2 to 4 (colored help output, better error messages, typed derive API)
• Upgrade to Rust edition 2024, set MSRV to 1.85
• Update release workflow toolchain from 1.83.0 to 1.85.0
• Bump all dependencies via cargo update

Fixed

• Clear all RUSTSEC vulnerabilities: bytes (integer overflow), rustls-webpki (4 CVEs), time (stack exhaustion DoS), rand (unsound with custom logger)
• Remove unmaintained ansi_term and atty transitive dependencies (were pulled in by clap 2)
• Fix clippy unnecessary_unwrap lint in request handling

0.10.0-alpha.1 - 2026-05-22

Friendly fork of fcsonline/drill 0.9.0. See FORK.md for rationale and migration instructions.

Changed

• Renamed crate and binary from drill to driller
• Updated package metadata (repository, description, authors)
• Trimmed publish payload (exclude .github/, example server)

Added

• FORK.md explaining the fork's purpose and relationship to upstream
• Local CI script (local-ci.sh)

Unchanged

• License remains GPL-3.0-or-later
• Benchmark YAML format and CLI flags are fully compatible with drill 0.9.0
• Full upstream git history preserved