Why PREEMPT_RT made it worse

Installing a realtime kernel is the prestige move in every Pi time-server guide. It is also, on its own, the single most damaging thing we did.

	raw PPS jitter (σ)	peak-to-peak
Stock kernel	2134 ns	11 µs
PREEMPT_RT	6947 ns	38 µs

Three times worse. Not marginally, not within noise — three times.

Why

PREEMPT_RT achieves its determinism by force-threading interrupt handlers. Instead of running in hard-IRQ context (immediately, uninterruptibly, nanoseconds after the electrical edge), a handler becomes a schedulable kernel thread that the scheduler runs when it gets around to it.

For most drivers this is a good trade: you lose a little latency, you gain the ability to preempt long-running handlers, and the worst case improves. That’s the entire pitch of realtime Linux, and it’s a good pitch.

But look at what pps-gpio actually does in its handler:

static irqreturn_t pps_gpio_irq_handler(int irq, void *data)
{
  ...
  pps_get_ts(&ts);              /* ← THE TIMESTAMP IS TAKEN HERE */
  pps_event(info->pps, &ts, ...);
  ...
}

The handler is the measurement. pps_get_ts() is the whole point of the driver — it captures when the pulse arrived. Everything downstream, every nanosecond of accuracy chrony reports, descends from that one call.

So when PREEMPT_RT threads this handler, it doesn’t defer some work. It defers the act of looking at the clock. The timestamp is no longer taken at the electrical edge; it’s taken after thread-wakeup latency — microseconds later, and variably later, which is worse.

We didn’t make the system more deterministic. We inserted a scheduler between the pulse and the clock.

You can see it happen

On a stock kernel, the PPS interrupt has no thread at all:

$ ps -eo pid,class,rtprio,psr,comm | grep irq/41
(nothing — it runs in hard-irq context)

Boot PREEMPT_RT and it materialises:

$ ps -eo pid,class,rtprio,psr,comm | grep irq/41
  239 RR      50   3 irq/41-pps@12.-1

That thread is the problem. It is also — and this is the cruel part — exactly what the guides tell you to go and pin to an isolated core. You can only taskset a thread. The advice to isolate the PPS IRQ requires the very threading that destroys the timestamp.

The trap, stated plainly

You can pin the PPS interrupt, or you can timestamp it fast. You cannot do both. Threading is the price of pinning, and on our board that price was 12× the accuracy. A hard-IRQ handler on a busy CPU 0 beat a threaded-and-pinned one on a quiet, isolated CPU 3 — by a mile.

The fix

Tell the kernel this particular handler must not be threaded:

flags |= IRQF_NO_THREAD;

That’s it. The timestamp goes back to hard-IRQ context, at the electrical edge, while the rest of the system keeps every benefit of PREEMPT_RT.

	RMS offset	raw PPS jitter
Stock kernel	440 ns	2134 ns
PREEMPT_RT (unpatched)	2468 ns	6947 ns
PREEMPT_RT + IRQF_NO_THREAD	199 ns	2568 ns

The patch is four lines and it’s here. It is, as far as we can tell, not applied anywhere — which means anyone running GPIO-based PPS on a realtime kernel today is silently eating microseconds of jitter and has no reason to suspect it, because everything looks fine. chrony still says Stratum 1. The dashboard still says locked. The number is just quietly worse.

The lesson underneath

The realtime kernel is not “the fast kernel.” It is the predictable kernel, and it buys predictability by making things schedulable. If the thing you care about is a measurement taken inside an interrupt handler, making it schedulable is precisely the wrong move.

Nothing about that is obvious from the outside. It is only obvious from a number.