How to Backtest a Trading Strategy: 7 Steps, a Split Test and a 22-Year Walk-Forward on SPY
Michael Whitman
·
Trading since 2017, full-time since 2020
·
YouTube creator, 137K subscribers
·
Published 2026-09-06
·
Updated
·
Reviewed
To backtest a strategy, split the history in two before looking at either half, tune the rule on the first half only, then run it once on the second with real costs included. The held-out half is the only part of the test that means anything.
To backtest a trading strategy, split the price history before you look at it, tune the rule on the first part only, then run it once on the rest with costs and a control. Then walk that split forward through time, because one lucky split can pass a rule that a second one would not.
A backtest measures the rule and the process that chose its settings. If that process saw the answers, the result describes the search. Two real tests on SPY below show what each step catches.
Before you start
Charting software with history going back at least a few hundred bars on the timeframe you trade, or a spreadsheet and a free daily price download. Both tests on this page use free daily data.
A rule already written down, in the sense the algorithmic trading page means: a stranger could apply it and get the same trades you would.
Your real round-trip cost: spread plus commission plus an allowance for slippage. Not the advertised spread.
If any of the three is missing, stop here. A vague rule tested without costs can only produce a flattering number.
A rule that led once can trail later: on the S&P 500 the 10-day average rule that led 25 lengths from 1958 to 1992 ranked last from 1992 on, as technical vs quantitative analysis shows.
The steps
1. Write the rule until it has no judgment left in it
Entry condition, stop, target, and what happens if neither is hit. “Wait for confirmation” is not a rule; “enter on the close above the prior high” is.
2. Split the history in two and write down where
Do this before you look at either part. The split date goes in your notes, because a split chosen after seeing the data is not a split.
3. Tune on the first part only
Try whatever you like here. This is the part you are allowed to fit.
4. Add your costs to every trade
Deduct the round trip from each one. Say a round trip costs $0.02 and an ordinary bar spans $1: that is 2% of a bar, and across 100 trades it hands over two whole bars of movement before any result is counted.
5. Run once on the second part
Once. If you adjust anything after seeing it, that part is tuned too and you have no test left.
6. Run two controls: the same trades on a schedule, and simply holding
Take the same number of entries every Nth bar, ignoring the rule, with the same exits and costs. Then compare against buying at the start of the test period and holding to the end.
7. Repeat the split forward one year at a time
Tune on the years before each year, trade that year, then move forward and tune again. Join the years together and compare the result with the same controls.
8. Read six numbers from the report, side by side with the control
Ending value, yearly growth rate, worst fall, number of trades, time in the market and settings used. The section below reads each one on a real result.
How to tell it worked
The test passes when four things are true. The held-out result is positive after costs. It beats both controls. The walk-forward result agrees with the single split. And there are enough trades to mean something: with 25 trades, a 50% win share has a 95% margin of about 19.6 percentage points either way, with 100 trades about 9.8, and with 400 trades about 4.9.
The SPY tests below pass the first condition and fail the second, which is the most common way a careful backtest still misleads. Both use only the buy-and-hold control; the scheduled-entries control was not run, so holding is the only yardstick they are judged against.
The original data
Test one, the split. The rule: hold SPY while its close is above its own simple moving average, sit in cash while it is not.
Sixty settings, every length from 10 to 305 days in steps of 5. Trades happen at the close, each buy or sell costs 0.05% of the account, cash earns nothing, and closes are adjusted for dividends. The first half runs from 13 April 1994 (the first day the 305-day average exists) to 16 July 2010, the second from 16 July 2010 to 26 October 2026, 4,094 trading days each, split at the midpoint by day count.
The tuning half picked the 300-day average. It turned $1 into $4.43 from 1994 to 2010, against $3.18 for holding, and 14 of the 60 settings beat holding in that half. The held-out half agreed: the 300-day setting ranked 3rd of 60 from 2010 to 2026, and the rank correlation between the halves was 0.87.
And yet none of the 60 settings beat holding in the second half. The 300-day rule turned $1 into $5.49 while holding turned it into $9.55; the best of all 60 reached $5.70, 59.66% of holding’s ending value.
Checks on test one. With no costs the 300-day setting still tops the first half and ranks 3rd in the second, and 0 of 60 beat holding there. Letting cash earn the 3-month Treasury bill rate (FRED series DTB3) lifts the best second-half result to $5.91, still 0 of 60. Nasdaq’s SPY closes for 2016-2026 give the same 300-day position on all 2,213 days that can be compared. The rows are in the settings file and the buy-and-hold file.
Test two, the walk-forward. Same rule, same 60 settings, same 0.05% cost, data to 30 Oct 2026. At the last close of each year from 2004 to 2025, pick the setting that grew $1 the most over the previous 1, 2, 3, 5 or 10 years, or over all history back to April 1994, and trade it for the next year. That gives 22 years held out in every version, 31 Dec 2004 to 30 Oct 2026, 5,492 sessions.
The shortest tuning window did worst here, and all history did best. Re-tuning on the past year changed the setting at all 21 year-ends and turned $1 into $2.42. Tuning on all history settled on 295 to 305 days, changed only 3 times and made $6.54.
| Tuning window | $1 grew to | Per year | Worst fall | Buys | Switches | In market | Settings |
|---|---|---|---|---|---|---|---|
| Past 1 year | $2.42 | 4.14% | -45.93% | 193 | 385 | 71.45% | 17 |
| Past 2 years | $4.70 | 7.34% | -25.64% | 122 | 243 | 74.03% | 19 |
| Past 3 years | $4.26 | 6.87% | -21.31% | 103 | 205 | 77.93% | 14 |
| Past 5 years | $4.13 | 6.71% | -19.63% | 104 | 207 | 75.87% | 12 |
| Past 10 years | $5.20 | 7.85% | -21.31% | 56 | 111 | 81.65% | 9 |
| All history | $6.54 | 8.98% | -21.31% | 37 | 73 | 83.32% | 3 |
| Hold SPY | $9.29 | 10.75% | -55.19% | 1 | 0 | 100% | - |
Holding beat every version and every fixed setting. None of the 60 fixed settings beat it over the same period either (the best, 290 days, reached $7.01; the median $4.59). All figures are in the report file.
Checks on test two. At no cost the six versions make $2.58, $5.38, $4.76, $4.24, $5.15 and $6.96; at 0.10% a trade, $3.59, $4.16, $4.15, $4.39, $5.58 and $6.30. One-year tuning stays lowest and all history highest at all three costs (summary file).
And the prices. Nasdaq’s SPY closes for 1,275 sessions from October 2021 match Yahoo’s unadjusted closes exactly, with the same 200-day position on all 1,076 comparable days. Every yearly pick is in the walk-forward file.
This is one fund and one family of rules, simulated on past data. It says nothing about how any setting will behave from here, and on another market or timeframe the ordering could differ.
How to read a backtest report
Ending value first, and only next to the control. The all-history version turned $1 into $6.54. On its own that looks like a result; beside holding’s $9.29 it is 70% of the simpler choice.
Then the yearly rate. 8.98% a year against 10.75%. Ending values compound, so a gap of under two points a year became a gap of $2.75 per $1 over 22 years.
Then the worst fall, from a peak in the equity curve to the lowest point before a new peak: 21.31% for the all-history version (19 Feb to 21 May 2020) against 55.19% for holding (9 Oct 2007 to 9 Mar 2009).
The rule lost less in the worst fall and still ended with less. The report shows that trade-off existed on this history; it does not say which side of it suits you.
Then the number of trades. 37 buys in 22 years for the all-history version is a small sample, so its result rests on few decisions. The one-year version’s 193 buys gave a larger sample of a different rule, one that paid more costs and spent less time in the market.
Then time in the market. 83.32% for the all-history version, 71.45% for the one-year version. Every day out is a day the rule has to make up for.
And how many settings it used. 3 settings over 21 re-picks means the choice was stable; 17 means the pick changed with each year’s conditions. The win rate page covers the per-trade side of the same report.
Across our study of 24,971 trading videos, 462 have backtest or backtesting in the title. The median one gets 2,234 views. Of the 133 with description text, 10 mention invalidation or failure.
What a backtest cannot tell you
Your fill and your size. A backtest fills any quantity at the printed price. A real order meets the liquidity that was there, and a fast market moves before you do.
A gap. The move happens between two bars, so no stop helps; the gap trading page has what that costs.
And whether you would have taken them. A hundred mechanical trades on a screen is not a hundred trades you sat through. That is what forward testing is for.
When it fails
The settings and the test share a history
This is the failure, and every step above is one procedure for avoiding it. A result chosen from sixty candidates on the same data it is then judged against is a search result. The overfitting page covers why the best-looking candidate is the one most likely to disappoint.
The test passed and the control was skipped
Test one is the example. Its held-out half agreed with its tuning half. Only the buy-and-hold control shows it finished 42.51% short of holding’s ending value.
The tuning window was too short
Test two is the example. The one-year version picked a new setting every year and ended with about a quarter of holding’s result.
You re-tuned after a bad stretch
A rule adjusted whenever it disappoints has never been tested at all. It becomes a record of many rules with a few trades each, the trading journal failure applied to a spreadsheet.
The costs were left out
On any rule that trades often, a backtest without costs measures nothing useful. Costs are the largest single line in the frequency table.
Questions people ask
How many years of data do you need to backtest?
Enough to hold several different market conditions in both the tuning part and the test. On SPY daily, one year of tuning gave the lowest walk-forward result of the six versions tested, and all history the highest.
Can I backtest for free?
Yes. Both tests here use free daily closes and a short script; a spreadsheet can run one setting the same way, row by row.
Can ChatGPT or another AI backtest a strategy?
An AI assistant can write the code or the spreadsheet formulas. It does not change the method: you still set the split first, include costs, compare against holding, and check a handful of its trades by hand.
Related
How to automate a strategy is the next step once a rule survives this, and where the parameter count starts to matter.
How to build a trading plan is where the costs and sample size on this page come from.
And do trading bots work runs the split test in full, with the grid and the correlation written out.
I decide the split before I open the chart, and I write the date down. Without that, a backtest is really a search for the settings that fit what already happened, and you cannot tell the difference from the inside.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.