Golden tests
Pin a strategy's BacktestReport and fail the build when it changes beyond a tolerance you set — absolute or relative, per field, exact by default.
Golden-pin your strategy's backtest report, catch regressions in CI, and property-test it against fuzzed market data — in ten languages, plus a reusable GitHub Action.
There is no test API to learn. A test is JSON: which strategy, over which dataset, pinned to which report, under which tolerances.
{
"id": "ema_crossover",
"dataset_ref": "BTCUSDT",
"strategy": {
"symbol": "BTCUSDT",
"timeframe": "1h",
"indicators": {
"fast": { "type": "Ema", "params": [12] },
"slow": { "type": "Ema", "params": [26] }
},
"entry": { "cross_above": ["fast", "slow"] },
"exit": { "cross_below": ["fast", "slow"] },
"sizing": { "type": "fixed_fraction", "fraction": 0.95 }
},
"expected": { "…": "the pinned BacktestReport, written by `bless`" },
"tolerances": {
"*": { "kind": "rel", "value": 0.0001 },
"metrics.sharpe": { "kind": "abs", "value": 0.01 }
},
"property_checks": [
{ "kind": "no_nan" },
{ "kind": "max_drawdown_le", "value": 25.0 }
],
"fuzz": { "seed": 42, "runs": 8, "perturbation": { "kind": "jitter", "amount": 0.001 } }
}The strategy block is opaque: Strategy-CI never parses it, it forwards it verbatim to the wickra-backtest engine and pins whatever report comes back. So a spec that engine understands is a spec this runner can pin, with no release here in between.
| Axis | The question it answers |
|---|---|
| Golden | Did the numbers change? Both reports are flattened to numeric leaves, rounded to eight decimals, and compared per field — reporting mismatches, fields that vanished, and fields that appeared. |
| Property | Is the run sane at all? Invariants that hold for any run, independent of a pinned value. |
| Fuzz | Does it survive a history it has not seen? Seeded perturbations, re-run, properties re-checked. The golden is not re-checked under fuzz — perturbed data produces a different report by design. |
The same runner from every language — native Rust, Python, Node.js and WASM, plus a C ABI for C, C++, C#, Go, Java and R.
cargo install wickra-strategy-ci# Run a directory of strategy tests against a directory of OHLCV data.
# Exits non-zero the moment a report drifts, so CI fails on it.
wickra-strategy-ci run tests/ --data data/
# Re-pin the goldens after a change you meant to make.
wickra-strategy-ci bless tests/ --data data/
# List the test ids found under a path.
wickra-strategy-ci list tests/Write the strategy, run bless once to pin the report, and commit the file. From then on run fails the build whenever the numbers move further than you allowed.
The diff works on numeric leaves. Both reports are flattened to a sorted map of numbers — metrics.sharpe, equity[3].equity — and compared. Strings, booleans and nulls are not pinned, so a report field that is text is outside what a golden can catch. That is a real limit, and it is better stated than discovered.
Wickra Strategy-CI is part of the Wickra ecosystem. It is the test harness for the deterministic wickra-backtest engine, whose indicators are the same O(1) kernels that wickra-core computes live — so a pinned report is pinned against exactly the numbers a live strategy would see.
Wickra Strategy-CI is research and engineering tooling, not financial advice. A passing test attests only that a strategy's backtest report matches its pinned expectation under the given data — it makes no claim about the quality, profitability or future performance of any strategy. Trading carries risk; you are responsible for your own decisions.