Do Trading Bots Work?
A trading bot executes a written rule without judgment, which removes hesitation and adds nothing else. The hard part is not the code but knowing whether the settings were chosen on the same data they are being judged on, because tuning a rule to past bars is what makes a backtest look better than the future.
The interesting question is not whether a computer can follow a rule. It is whether the rule was chosen honestly, and there is a standard test for that which almost no backtest screenshot has passed.
How it works
A bot is a rule with no discretion in it. The rule below has two parameters and nothing else: buy when RSI is below a level, sell a fixed number of bars later.
Two parameters is a small bot. A real one has ten or twenty, and every one of them is a dial that can be turned until the past looks good.
The test that matters
Sixty settings were tried on the first 288 bars — six entry levels crossed with ten holding periods. The best made +7.53 after costs, on 11 trades.
That is the number a backtest screenshot shows you.
The same setting on the next 288 bars made +5.85 — and ranked 19th of the 60.
The best setting on the second half was a different one entirely, and it made +11.22.
Nothing was hidden and nothing was cherry-picked. The winner was simply chosen on data it was then judged against, which is the one thing a test has to avoid.
How much the first half tells you
Across all 60 settings, first-half performance and second-half performance correlate at r = +0.38.
Some information, and not much. A correlation of +0.38 explains about a seventh of the variation, so a setting that came top in the first half is more likely than average to do well in the second and is nowhere near certain to.
And with two parameters. The relationship falls further as parameters are added, because more dials mean more ways to fit noise, and noise does not repeat.
One caveat stated plainly: every one of the 60 settings made money out of sample, from −0.06 to +11.22, because the rule is long-only and this history drifts upward. The comparison between settings is what this section is about, not the returns.
Why more dials make it worse
A grid of 60 settings contains 60 chances for one of them to look good by accident.
That is the mechanism, and it is arithmetic rather than opinion. Each setting produces a result that is part signal and part luck; searching for the maximum finds the setting with the most luck in it as reliably as the one with the most signal.
Add a third parameter with ten values and the grid is 600. Add a fourth and it is 6,000. The best result gets better every time you add a dial, and none of that improvement is real.
Which gives a rule of thumb worth more than any indicator choice: count the parameters, and count the trades. Sixty settings tested on 11 trades each, as above, is already thin. A twenty-parameter system fitted to a few hundred bars is fitting noise with great precision.
What automation actually buys
Consistency. The rule is applied identically every time, which is the only condition under which a record means anything — the same argument the trading journal page makes.
Speed and attention. It watches everything, all the time, without deciding it deserves a break.
And no hesitation, which matters most at exactly the moments a person hesitates.
What it does not buy is judgment. Every judgment is still being made — it is being made once, in advance, by whoever chose the parameters.
What it cannot do
Notice that conditions changed. A rule tuned to a trending stretch keeps firing in a range, and why traders lose money has the cost of that in a table.
React to a gap. The move happens between two bars, so speed is worth nothing; the leverage page has what that costs.
Or know that its own settings were fitted. The bot has no way of telling that it was the winner of a search rather than the discovery of an edge.
A worked example
Split your data before you look at it. First half to tune, second half untouched.
Tune on the first half only.
Run once on the second half. Once. If you go back and adjust after seeing it, the second half is now in-sample too, and you have no test left.
Then count the parameters. Two is testable, ten is not — with ten dials and a few hundred bars, something will always fit.
The original data
Across our study of 24,971 trading videos, 157 cover trading bots. The median one gets 9,862 views, 73% never pass 50,000, and the median length is 8.4 minutes.
The corpus carries description text for only seven of those 157, which is too thin to say anything about how the topic is written, and this page does not.
What is worth putting side by side is the pair of fields. 157 videos about the bot, against 496 on algorithmic trading — the rule-writing that has to happen first. Three times as many people write about the rule as about the robot, which is the right way round and not what the marketing suggests.
When it fails
The backtest and the settings share a history
This is the failure, and it has a name: fitting. Everything above is one demonstration of it, on the smallest possible bot.
You re-tune after every bad month
Then every month is in-sample. A rule adjusted whenever it disappoints has never been tested at all, which the trading journal page describes as a record of many rules with one trade each.
The costs were not in the test
Every figure on this page has 0.02 deducted per round trip. A backtest without costs on a high-frequency rule is not optimistic, it is meaningless — costs took two thirds of the gross in the cost table.
You judged it from where it ended
Sixty bars is not a sample, and a bot that is losing over sixty bars is indistinguishable from one that has stopped working. Deciding that in advance — how many trades before you switch it off — is the only version of the question with an answer.
Related
Algorithmic trading is writing the rule down, which is the part that has to work before code is relevant.
Do indicators work is the same discipline applied to a single signal, tested against a control.
And the trading journal is why changing the rule mid-test destroys the test.
I do not run a bot and the reason is not distrust of code. It is that the version of me who would pick the settings is the same person whose judgment I would be trying to remove, so all I would be doing is moving the guessing one step earlier and making it harder to see.
— Michael Whitman, from this video
This page is educational, not financial advice. Test every idea on your own charts before risking money.