Performance Budgets

The tiers

Reference baselines (2026-08-30, rousseau-cachyos-geekom-a9, Ryzen AI 9 HX 370)

All medians in milliseconds. Budget = 2× median (or the tier ceiling, whichever is larger).

INSTANT tier

FAST tier

Currently empty.

dot status and dot diff were placed here at 2000ms against reference medians of 1510 ms and 1420 ms — 1.3× and 1.4× headroom, short of the ≥ 2× this document requires of every budget. They have since moved to MEDIUM; see that section for the measurements that prompted it.

The baselines of 28 ms and 32 ms recorded before those were not real. The perf sandbox never created chezmoi's source directory, so both commands aborted immediately with "no such file or directory" — and _measure discarded exit codes, so the gate timed the failure path and called it excellent. The sandbox now links the repo into XDG_DATA_HOME.

The QA gates and test suites that used to sit in this tier moved to GATES below: they are not interactive operations, so a ceiling defined by human-perceived latency never described them.

MEDIUM tier

The dot status and dot diff baselines are the worst of the two hosted runners, not the reference machine: macOS measured 2290 ms / 2157 ms and Ubuntu 2329 ms / 2136 ms, against 1139 ms / 1174 ms locally. Two unrelated platforms agreeing within 8% is a property of the commands rather than of one slow runner — both shell out to chezmoi, which walks the whole source tree, and that is disk-bound. A budget taken from the faster machine would have been a gate that only ever fired on other people's hardware.

dot doctor and bench.sh --quick are diagnostics that report findings through their exit status — dot doctor exits 1 whenever it finds issues, and bench.sh exits 1 when a shell breaches its own startup threshold. They are gated with _gate_diag, which permits exit 1 but still fails on 2+ (not-found, permission, signal, syntax error). That is far narrower than the blanket || true it replaced.

GATES tier (CI quality gates and test suites)

Budgets are 2× the median measured on the slowest supported platform, not the fastest. Medians below: rousseau-mbp-m1, macOS 26 (Darwin 25.6), 2026-08-30.

† The manifest test hashes an entire sandbox tree before and after every scenario, so it is disk-bound in a way the other gates are not. Its budget is set at 3.2× the macOS median rather than the 2.0× used above, deliberately: dot status and dot diff were first budgeted at 1.3× and 1.4×, and both breached on the first CI run that measured them. The Linux median is left unfilled until CI reports one — guessing it would defeat the point of a table of measurements.

These gates run 3–6× slower on macOS than on Linux — fork/exec is markedly more expensive there and every one of them is fork-heavy shell. CI covers both platforms, so the original Linux-only calibration could not hold, and five of these sat red as a result. Re-capture the Linux column when convenient; it is carried over from the 2026-08-30 Ryzen baseline and is not the binding constraint.

ACCEPTED-SLOW (documented, not gated per-run)

OUT-OF-SCOPE (not gated per-run)

Ratchet vs aspiration

The budgets above are regression gates, not aspirations. If a real optimisation lowers a baseline, edit the doc + the perf test to lower the budget too. If a change pushes something over the budget, the test fails and CI blocks the merge.

The aspirational shell-startup target (<30ms) is tracked separately in benches/bench.sh — that's a bench, not a budget.

Adding a new operation

When you add a new script that runs at a shell prompt:

1. Time it 5 runs on a warm system: for _ in {1..5}; do time bash your-script; done
2. Take the median.
3. Place it in the tier where budget ≥ 2 × median. If a median is 400ms it goes in FAST (500ms is uncomfortably tight); if it's 300ms, INSTANT is fine.
4. Add it to benches/test_perf_budgets.sh in the correct tier section.
5. Add its baseline to this doc.

Where the enforcement lives

• Per-op budget test: benches/test_perf_budgets.sh
• Suite-level wall-clock ratchet: benches/test_help_gates_wall_clock.sh
• CI wiring: .github/workflows/ci.yml, job quality-performance — runs on ubuntu-latest and macos-latest for every PR, with no || true and no budget scaling. On the gates the hosted macOS runner is comparable to or faster than the reference machine — docs-coverage 679 ms vs 969, traceability 1775 vs 2389, help-registry 3318 vs 4449, dot doctor 1746 vs 4423. The exception is anything that drives chezmoi over the whole source tree: dot status and dot diff measured ~2× the local median on both hosted platforms, which is why they sit in MEDIUM rather than FAST. Budget a new operation against the slowest platform it will run on, not against this machine.

How the time is measured

_measure uses the time keyword with TIMEFORMAT='%3R', not date +%s%N.

%N is a GNU extension. BSD date — macOS 14 and earlier — copies the literal N through, so every arithmetic conversion failed, _measure returned nothing, and an empty median compares as 0 against any budget. The gate reported every budget met on those machines, for any command, including one that could not parse. time is a shell builtin, is millisecond-accurate in bash 3.2, and needs no external clock at all.

Two consequences worth keeping:

• _gate_max_rc rejects a non-numeric median outright rather than comparing it. Both fields go through -gt / -le, where bash reads a non-numeric operand as 0 — so a gate that cannot measure would otherwise report success.
• tests/regression/test_gate_integrity.sh puts a date without %N first on PATH and requires the same verdicts. A future timing rewrite that reintroduces the dependency fails there rather than going quiet.

Environment knobs

When a budget fires

The error looks like:

✗ instant_dot_help: median=612ms EXCEEDS budget=500ms

Steps:

1. Bisect the change that pushed it over.
2. Fix the regression, or
3. If the increase is legitimate (real new work), move the operation to the next tier + update this doc + test_perf_budgets.sh in the same PR.

Never silently bump the budget. The tier a thing lives in is a promise to users.