Checked by something we didn’t write

The same classifier exists three times: a reference in Python, the backend in C#, and the one that actually runs in your browser, in JavaScript. Three implementations of one algorithm is three chances to be subtly different, and for months they were. Run over the same stored epochs, the C# and JavaScript classifiers agreed with each other on 52.3% of them — mean confidence 0.895 against 0.453 — and nothing in the build noticed.

What fixed it was not more care. It was one fixture all three answer to: a golden file generated from the Python reference, asserted against the C# port to twelve decimal places, and read by a browser harness holding the JavaScript to the same file and tolerance. Every case checks the label, the refusal, the distance, the confidence, all five state probabilities, and the refusal reason string verbatim — because that text is the only thing a person ever sees to explain why a reading was withheld.

It currently reports 39 of 39 cases agreeing, with max |Δconfidence| = 0.000e+00 and all 195 state probabilities bit-identical — not within tolerance, identical. The tolerance exists only so that a last-bit difference between two runtimes’ logarithms would not be reported as a broken port.

Underneath the classifier, the signal processing is checked the same way against a numpy/scipy oracle, which is the subject of the methodology post. The pattern is the same in both: a shared fixture, generated from a reference, asserted by every port. A fixture nobody checks is a fixture that drifts.

• 39/39 cases
• 12 decimal places
• 195/195 probabilities exact
• Python · C# · JavaScript

Three implementations agreeing to the last bit proves that they are the same algorithm. It does not prove the algorithm is right. Three ports of a mistake agree perfectly.

So the second check has to come from outside the project entirely, and it has to be a question whose answer is known before anyone looks.

Close your eyes and alpha rises. Berger reported it in 1929 and it has replicated ever since; it is the nearest thing electroencephalography has to a fact you can check a machine against.

The recording is PhysioNet’s EEG Motor Movement/Imagery dataset, under the Open Data Commons Attribution licence: two one-minute resting runs, eyes open and eyes closed, from each of five subjects, epoched into 140 four-second windows at the production rate. Nobody here produced that data and nobody here can tune it. It went through the shipped feature extractor unmodified.

Relative alpha rises from 0.0516 to 0.1555 — a factor of 3.01, Cohen’s d of 1.69 — and it holds on all five subjects taken separately, at ratios from 2.06 to 4.83. Everything else the pipeline computed there is physiologically ordinary: a 1/f exponent near 1.1, an individual alpha frequency of 9.6–9.8 Hz, robust amplitude 33–38 µV.

The specific accusation this answers is that the pipeline cannot see alpha and reports it as dullness. Given real human EEG at Muse-equivalent electrode sites it recovers the largest and most reliable effect in the field, in the right direction, at better than three to one, on five subjects out of five. That was the outcome that had to be reported honestly either way.

A Muse records at TP9, AF7, AF8 and TP10. The public dataset uses a 64-channel montage that has AF7 and AF8 exactly, and nothing at TP9/TP10, so a substitute had to be chosen — and the choice decides the headline number.

T9 and T10 were taken, on measured scalp distance: 30.1 mm and 29.6 mm from the Muse sites, the nearest available. They are also the conservative choice, because they sit low on the same contour, and alpha is stronger further up. Moving to TP7/TP8, 9 mm further away and 36 mm higher, would have bought 4.41× instead of 3.02×. Going occipital — where the eyes-closed effect is biggest, and where a Muse has no electrode at all — would have bought 7.42× and a headline relative alpha of 0.49.

All three numbers are published in the same document as the result, which is what makes the restraint auditable rather than merely asserted. The fixture takes the smallest.

Very little, directly — and that is worth saying rather than dressing up. What it buys you is the ability to rule one thing out. If your reading looks wrong, the cause is not that Delve is blind to alpha, because that has been tested against a recording it cannot influence.

What remains on the table is the band on your head. Dry electrodes on hair, over a brow that moves, is the fragile part of this chain, and it is the part you can actually do something about: seat the band on skin, keep the arms behind the ears clear of hair, and give it a few seconds before you start counting the reading.