Benchmark PPS jitter honestly

Every claim on this site was produced this way. If you want to disagree with us, disagree with these numbers.

Don’t use chrony’s stats for this

chronyc sourcestats gives you a windowed, median-filtered, slowly-converging estimate. That is exactly what you want for disciplining a clock and exactly what you don’t want for measuring a change you just made. It lags, it smooths, and it will happily hide a regression for several minutes.

Measure the kernel’s PPS timestamps directly instead.

The measurement

Each PPS assert should land exactly 1.000000000 s after the last one. The deviation from that is the jitter — nothing else.

sudo apt install pps-tools

sudo timeout 62 ppstest /dev/pps0 | awk '
/assert/ {
  split($0, a, "assert "); split(a[2], b, ","); t = b[1] + 0;
  if (prev > 0) {
    d = (t - prev - 1.0) * 1e9;
    n++; sum += d; sumsq += d*d;
    if (n == 1 || d > max) max = d;
    if (n == 1 || d < min) min = d;
  }
  prev = t
}
END {
  mean = sum/n; sd = sqrt(sumsq/n - mean*mean);
  printf "n=%d  jitter_sd=%.0f ns  p2p=%.0f ns\n", n, sd, max-min
}'

62 seconds gives you ~60 intervals. That’s enough to see a 3× effect and not enough to see a 5% one — size your window to the effect you’re hunting.

Always run A/B/A

This is the part people skip, and it’s the part that makes the number mean something.

A time server’s behaviour drifts on the scale of minutes: the board warms up, satellites rise and set, the DOP changes. If you measure A then B and B is worse, you cannot distinguish “B is worse” from “the last five minutes were worse.”

So measure A → B → A:

round 1:  feature OFF   →  1304 ns
round 2:  feature ON    →  1912 ns
round 3:  feature OFF   →  1169 ns   ← agrees with round 1. Now believe round 2.

If the two A rounds disagree with each other by more than the A-to-B difference, you have measured nothing and you should go again with a longer window.

We caught our dashboard’s 36% tax exactly this way, and we disbelieved two other apparent wins when the bracketing rounds refused to agree.

Sanity check: is it even connected?

$ sudo ppstest /dev/pps0
source 0 - assert 1783875773.176667184, sequence: 28
source 0 - assert 1783875774.176667944, sequence: 29

Sequence incrementing once a second = you have a pulse. No output = you have a blinking LED and a wire that goes nowhere. See Hardware.