Standard Deviation: The Tails Are Fatter
Standard deviation measures how widely returns are spread around their own average, and it is the number behind Bollinger Bands, volatility readings and most risk models. Financial returns do not follow the bell curve it assumes, which is why extreme moves arrive more often than the maths predicts.
How it works
Standard deviation measures spread. Two markets can have the same average return and be completely different to hold; the deviation is what separates them.
The calculation squares each difference from the mean, averages those squares, and takes the square root. Squaring is what makes large deviations dominate the result — a move twice as far contributes four times as much.
On this site’s shared 576-bar history the bar-to-bar deviation is 0.347%. The largest single move up was 1.065% and the largest down was 1.194%.
Where the bell curve breaks
The number is usually paired with a normal distribution. That model says 68.3% of observations fall within one deviation of the mean, 95.4% within two and 99.7% within three.
The measured shares were 74.1%, 93.2% and 98.6%. More inside one deviation than predicted, and fewer inside two and three.
Both at once is the signature. A distribution can be more peaked in the centre and heavier in the extremes than a bell curve, with the shortfall coming from the shoulders in between. That is the standard shape of financial returns and it is exactly what these numbers show.
Eight of the 575 bar-to-bar moves exceeded three deviations. A normal distribution across that many observations would produce fewer than two. Extreme moves arrived roughly four times more often than the model allows, which is the entire practical content of this page.
In practice
It is a price statistic. A wide range on no volume and one on heavy volume produce the same reading and mean different things.
Scaling across timeframes uses the square root of time. A daily figure multiplied by the root of 252 gives an annual one — a convention that assumes independence between periods, which markets do not entirely have.
Because differences are squared, one gap can move the whole figure. A single outlier in a hundred observations can raise the deviation noticeably, which makes short windows unstable.
This is the practical consequence for a stop. A level chosen to be breached 4.6% of the time was breached 6.8% of the time here — roughly half again as often as intended.
Costs are unaffected by any of it. 2% of a median bar’s range per round trip on this history, whether the deviation reading is high or low.
Every deviation figure describes a window that has already closed. Using it as a forecast assumes the next period resembles the last, which is a reasonable working assumption and not a fact.
What to do about the fat tails
The fix is not a better distribution, it is a smaller position. Any model that treats a three-deviation move as impossible will be surprised several times a year, and the only defence that works regardless of which model is right is a size that survives being surprised.
The second adjustment is to stop reading probability into band touches. A price outside two deviations is not a 4.6% event; on this series it was closer to a 7% one, and on a real instrument with news risk it can be higher still. Treat deviation bands as a description of recent range and not as odds, and most of the trouble the measure causes disappears.
One more property is worth knowing because it explains a lot of odd behaviour: the measure is not robust. Because differences are squared before averaging, a single unusual bar contributes far more than its share, and a twenty-bar window containing one outlier can report a figure double the one it reported the day before.
That instability is why short windows produce bands that lurch. The alternative measures — average true range, or the average absolute deviation — do not square anything and are far steadier for the same window length. Neither is better in general; they simply weight extremes differently, and knowing which one your indicator uses explains most of the disagreements between two volatility readings on the same chart.
What standard deviation is not
It is not a probability. The odds come from the distribution you assume.
It is not direction. A high reading says nothing about which way.
It is not risk. It measures variability, and losing money is a different thing.
And it is not stable. One outlier moves it, because the differences are squared.
When it fails
In a range the reading falls to its lowest just before the break. Quiet conditions produce a small deviation, a small deviation produces narrow bands and confident sizing, and the move that follows is measured against a figure computed when nothing was happening.
The second failure is assuming normality. The bell curve is a convenience, and the measured shares here show what it costs.
A third is a window too short to be stable. Twenty observations produce a figure one outlier can double.
A fourth is comparing readings across instruments. A percentage deviation is comparable; a price one is not.
And a fifth is confusing it with risk. An asset that rises steadily has a deviation, and no drawdown.
The original data
On this site’s shared 576-bar history the 575 bar-to-bar returns have a mean of 0.0068% and a standard
deviation of 0.347%, annualising to about 5.5%. The largest single moves were +1.065% and -1.194%. Measured
shares within one, two and three deviations were 74.1%, 93.2% and 98.6%, against the normal distribution’s
68.3%, 95.4% and 99.7%, with 8 moves beyond three deviations. The figures are in
research/series-measurements.json, produced by site/measure_series.py.
The eight beyond three deviations is the number to carry. It is a synthetic series with no news in it, and it still produced four times the tail events a normal distribution allows — which suggests the excess is a property of how prices move rather than of any particular market event. Before using any deviation-based level, count how often your own instrument exceeded it, and size the position against the count rather than the theory.
Related
Bollinger Bands are the most common trading application. Historical volatility is the same figure plotted over time. And risk management is where the sizing consequence lives.
The practical lesson took me a while. It is not that standard deviation is wrong, it is that the bell curve attached to it is. Once you accept that the extremes arrive more often than the model says, position sizing stops being about the typical day and starts being about surviving the atypical one.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.