Experiments

What we have tried in building Weather Footprints, and what we found: what worked, and as importantly what did not. Each page says what the question was, what data it was answered on, how the answer was judged, and what, if anything, we changed because of it.

These pages are more technical than the rest of the site. Scores are mostly the Brier skill score (BSS: 0 is no better than always forecasting the usual frequency, 1 is perfect, below 0 is worse than that) and the ROC area (AUC: how well a forecast ranks the days that turned out to have the weather above those that did not; 0.5 is no better than chance, 1 is perfect). Unless a page says otherwise, skill is cross-validated: each week is scored by a fit made without it.

Planned experiments are listed with their question before they are run, so that the way they are judged is fixed before anyone sees the result.