My Forecast Tool Has Been Wrong for a Year — Always in the Same Direction
Every month I run an "8 ball" — it takes how a given calendar month has historically behaved and uses that as a humble preview of the month ahead. Not a prediction. A tendency. Shake it, get the odds.
Here's the uncomfortable, interesting thing: it's been wrong for over a year. And it's wrong the same way almost every time.
Look at the scoreboard. Every one of these is the 8 ball's historical tendency versus what the month actually did:
| Month | 8 ball said | Reality | Miss |
|---|---|---|---|
| Aug 2024 | quiet (−10%) | +73% | massive |
| Sep 2024 | chop (−8%) | +24% | wide |
| Oct 2024 | normal (−1%) | +26% | wide |
| Nov 2024 | quiet (−14%) | +14% | hot |
| Dec 2024 | quietest (−20%) | calmer, but still | over |
| Jan 2025 | warm (+21%) | +73% | massive |
| Feb 2025 | warm (+21%) | +54% | over |
Read the "reality" column. Month after month, the market ran wider than history said it should. Not once did the 8 ball say "quiet" and get a genuinely quiet month in return. The misses aren't random noise scattered both directions — they're stacked on one side.
Why one-sided misses matter
If a forecast is wrong randomly — too high sometimes, too low others — that's just noise, and you shrug. But when a model is wrong in the same direction over and over, that's not the model failing. That's the model telling you something has structurally changed.
The 8 ball is built on years of history — including the sleepy, low-volatility stretches of 2019 and mid-2021. It carries the memory of a calmer market. So when it keeps lowballing the range, the honest interpretation isn't "the tool is broken." It's:
The market has been running in a higher-volatility regime than its own history — and has been for over a year.
That's a genuinely useful thing to know, and I only know it because I kept an honest scoreboard of my tool being wrong. If I'd quietly updated the baseline every month or stopped tracking the misses, the pattern would've been invisible. The persistent error is the finding.
What I do with it
Two things, practically:
Trust the miss, adjust the read. Until the pattern breaks, I mentally shade the 8 ball's range estimate up. If it says "quiet, −10%," I read that as "probably normal-to-wide," because that's what's actually been happening. The tendency is the floor, not the forecast.
Watch for the regime to flip back. One-sided error doesn't last forever. The first month the market comes in at or below the 8 ball's quiet call will be a real signal — it'll mean the high-volatility regime may finally be releasing. I want to catch that turn, and the only way to catch it is to keep scoring the misses honestly, month after month.
The bigger point
This is the whole philosophy, turned on my own tool. It would be easy to only publish the 8 ball when it looks smart and bury it when it whiffs. Instead the miss gets its own article, because a model that's wrong the same way every time is more valuable than one that's occasionally right by luck. The error has a shape, and the shape is information.
My forecast tool has been wrong for a year. That's not the embarrassing part of the project. That might be the most useful thing it's told me.
Sharpen your edge by telling yourself the truth — especially about your own tools.
— 23HT
Ranges are median daily MNQ range vs. the running all-days baseline, per the monthly Crystal Ball & 8 Ball reports. "8 ball said" = the historical tendency for that calendar month as of that date; "reality" = what the month actually printed.