Options backtesting software: what to check before you trust one

Why the tool matters

Two backtesters can run the same rule on the same years and disagree on the sign. The difference is almost never the strategy. It is the fill model, the data, and how the result is summarised. Before you compare results across tools, or trust one tool's chart, check how it answers the six questions below.

Established backtesters such as Option Omega, ORATS and the tastytrade backtester are popular with index and 0DTE traders, and each makes its own choices on these points. Read their documentation for the answers rather than assuming them.

1. Does it fill at the bid and ask, or the midpoint?

A seller receives the bid. A backtest that fills at the midpoint hands every trade half the spread, and a four-leg trade collects that gift four times. On thin chains the gift can be larger than the edge. Ask what the tool fills at, whether it charges fees per leg, and whether you can change both.

2. Which years does the data cover?

A strategy tested only on 2023 to 2025 has seen one kind of market. Selling puts behaves very differently in 2018, 2020 and 2022. Ask when the option quotes start, whether they are real quotes or modelled prices, and whether the tool tells you when it had to fill a gap in the data.

In our ten-year replays of index put credit spreads, the losing years were 2018 and 2022. A test window that skips both will look better than the strategy is.

3. Does it test on data the rule was not tuned on?

If you adjust the delta, the exit and the filter until the curve looks good, the curve describes the tuning, not the future. A good tool lets you hold out years, tune on some and score on the rest, and reports the held-out result separately. If every number you see was produced by the same data you tuned on, assume it is optimistic.

4. How does it count trades that share a market?

Ten puts sold in the same week of a selloff are one event, not ten. A tool that treats them as independent will report error bars far tighter than the truth. Look for intervals that group trades by month or by expiry, and for a lower bound, not only an average. A strategy whose interval includes zero is not established, however good the average looks.

5. Does it show the worst case and the sizing?

A win rate and an average return say little about whether you survive. Ask for the worst single trade, the worst month, and the drawdown at the size you plan to run. Ask whether it notices that QQQ, TQQQ and NVDA are closer to one bet than three. Our replays found that the same put spread risked at 10% of an account never produced a 20% drawdown within a year, while risked at 30% it did so about 38% of the time.

6. Does the backtest use your real exit rule?

Many backtests hold every trade to expiry, while most sellers close at half the premium or defend a threatened position. Those are different strategies with different results. The backtest, the paper record and the live record should all use the same exit rule, which our lesson on exit rules covers.

Testing a strategy described in plain English

Some tools now let you describe a strategy in words, for example sell a 25-delta put on QQQ 30 to 45 days out, only when it is above its 200-day average, and turn it into rules. That is convenient, and it raises one more question: show me the rule you actually ran. The words are ambiguous; the rule is not. Check the translated rule before reading any result.

Vesper works this way: you describe the idea, it shows the exact rule, replays it on real option quotes at the bid and ask, scores it on years the rule was not tuned on, shows the low end of the range rather than the average, and lets you paper trade it forward. You can also do this from Claude, ChatGPT or Cursor, which our AI agent guide explains. Whatever tool you use, the six questions above are the ones to ask.