Forecast Accuracy Analysis

Finalert runs that measurement as a standing routine. We back-test past forecasts against actuals, score them on a consistent method, identify who habitually sandbags and who habitually promises too much, and feed the findings into the next cycle so the forecasting process improves rather than simply repeating with fresh optimism.

The measurement has to be fair to be useful. That means fixing the forecast version being graded, comparing on the same basis as actuals, and accounting for genuine surprises that nobody could have seen. Done carelessly, accuracy scoring turns into blame and submitters respond by padding, which makes the next forecast worse rather than better.

Forecast accuracy analysis starts by deciding what is being graded. A forecast that was revised four times during the quarter has four versions, and grading the last one flatters everybody. We fix the versions to be scored, typically the original submission and each subsequent revision, and we lock them so they cannot be quietly restated later. We also fix the horizon, since a forecast made three months out and one made three weeks out are different promises and should never be compared against each other on the same chart.

Then each version is compared to actuals on a consistent basis. That sounds simple and rarely is, because reorganisations, changed account mappings, acquisitions and reclassifications mean the actuals arrive in a shape the forecast was never written in. We restate one side to match the other and document what was restated. Genuine one-off events are flagged rather than silently excluded, so the scoring shows both the raw result and the result adjusted for things nobody could reasonably have forecast.

Separating bias from error

Bias is the average signed difference over a run of periods. If a line comes in above forecast eight quarters out of ten, that is not bad luck, it is a lean, and it can be corrected in the next submission with a simple adjustment. Error is measured on absolute differences, so the misses do not cancel each other out, and it tells you how tight the forecast can realistically be even after the lean is removed. A line can be unbiased and still wildly inaccurate, or very accurate with a persistent lean, and those two conditions need completely different responses.

We cut both measures three ways. By line, because revenue, headcount cost, materials and capital spend usually behave very differently and a single company-level accuracy figure hides all of it. By period, because some months and quarters are reliably harder than others and seasonality shows up in accuracy as well as in the numbers. And by submitter, because forecasting is a human activity and the same individual tends to lean the same way across cycles, which is the most actionable pattern the analysis produces.

Back-testing and feeding it into the next cycle

Back-testing puts the whole history on one view. We rebuild the forecasts you submitted over past periods, score them against what actually happened, and show accuracy as a trend rather than a single verdict. That answers questions worth knowing: whether the process is improving, at what horizon accuracy falls off sharply, which lines are genuinely unforecastable and should be handled with a range instead of a point, and whether last year's process change made any measurable difference.

The output is only worth producing if it reaches the next cycle. So each forecasting round opens with the accuracy pack for the previous one: each submitter's own history, the bias adjustment suggested for their lines, and the horizon beyond which point forecasts stopped being credible. The tone matters, because scoring used as blame produces padding and padding destroys accuracy. A limit is worth stating plainly: we build and run the analysis, your finance leadership owns the forecast. We do not sign filings, issue audit or attest opinions, give legal advice or act as your accountant of record.