Infrastructure
PTP or Die: Hardware Timestamping for Regulatory-Grade Time Sync
Why NTP is insufficient for HFT compliance, and how to implement IEEE 1588 PTPv2 with hardware timestamping to achieve sub-100ns accuracy.
MiFID II’s RTS 25 requires order timestamps to be accurate within 100µs of UTC for systematic internalisers and regulated venues. Firms running NTP in production are not compliant with this requirement — NTP over typical datacenter networks has milliseconds of error due to asymmetric queuing.
This post documents how to implement sub-100ns time synchronization using IEEE 1588 PTP with hardware timestamping.
1. The Physics of Software Clocks
Your server’s TSC (Time Stamp Counter) drifts. Silicon oscillators are affected by temperature, voltage, and manufacturing variance. A typical drift rate is 1-100 parts per million (PPM).
At 100 PPM, your clock drifts 8.6 seconds per day.
Why NTP Falls Short for Compliance
NTP corrects drift by querying upstream servers over the network. The problem: asymmetric network jitter.
NTP assumes the request and response take equal time. In reality, datacenter switches have asymmetric queuing. This introduces milliseconds of error — three orders of magnitude worse than MiFID II requires.
- Source: RFC 5905 (NTPv4) - Section 8, “Clock Filter Algorithm”
- Regulatory Source: ESMA MiFID II RTS 25 - Article 3
2. Choosing an Approach
| Approach | Accuracy | Cost | Verdict |
|---|---|---|---|
| A. NTP (Public Servers) | ±50ms | Free | Non-compliant. |
| B. NTP (Private Stratum 1) | ±1ms | $5K | Still non-compliant under RTS 25. |
| C. PTP (Software Timestamping) | ±10µs | $10K | Borderline — risky under audit. |
| D. PTP (Hardware Timestamping) | ±50ns | $25K | Compliant with large margin. |
Why Hardware Timestamping? Software timestamping occurs in the kernel. By the time the kernel sees the packet, the NIC has already buffered it for some microseconds. Hardware timestamping stamps the packet at the PHY layer, before the kernel is involved, eliminating that source of error.
3. PTP on Solarflare
Solarflare X2522 NICs support hardware timestamping and have a dedicated PTP clock on the card.
Step 1: Configure the PTP Grandmaster
You need a GPS-disciplined PTP Grandmaster (e.g., Meinberg M1000). This device has an atomic clock and a GPS antenna. It broadcasts PTP packets to your network.
Step 2: Install and Configure ptp4l
# Install LinuxPTP
sudo apt install linuxptp
# Configure /etc/ptp4l.conf
# Key settings:
# tx_timestamp_timeout: Increase for busy NICs
# delay_mechanism: E2E for most setups
# priority1: Lower = more likely to be elected master
```text
## Step 3: Run ptp4l in Slave Mode
```bash
sudo ptp4l -i enp3s0f0 -m -H
# -H = Hardware timestamping (mandatory)
```python
## Step 4: Discipline the System Clock with `phc2sys`
`ptp4l` synchronizes the NIC's hardware clock (PHC). You must then discipline the system clock to the PHC.
```bash
sudo phc2sys -a -r -r -m
```text
**Verification:**
```bash
cat /var/log/ptp4l.log
# Look for: "rms 5 max 12" (rms < 100ns = success)
```text
## 4. Auditing Your Time Sync State
Before a compliance audit, verify your configuration:
```bash
# Verify ptp4l is running
systemctl status ptp4l
# Check NIC hardware timestamping support
ethtool -T enp3s0f0
# Check current offset
pmc -u -b 0 'GET CURRENT_DATA_SET'
5. Trade-offs
- Single Point of Failure: Your Grandmaster is now critical infrastructure. Budget for a redundant pair.
- Network Topology: PTP requires symmetric network paths. Asymmetric hops (e.g., load balancers) destroy accuracy. Run PTP on a dedicated, flat VLAN.
- Observability: Standard monitoring tools don’t understand PTP. You need custom dashboards showing
phc_offset_secondsfrom the PTP metrics.
6. The Core Reality
Time is not a feature. It is a physical constraint that determines the legality of your operation.
In financial markets, the order of events is legally binding. If your clock is wrong, your audit trail is fiction. MiFID II compliance requires you to treat time synchronization as infrastructure, not an afterthought.
Invest in PTP like you invest in fire suppression: hoping you never need it, but certain you will.
Up Next in Linux Infrastructure Deep Dives
CPU Isolation for HFT: The isolcpus Lie and What Actually Works
Why the standard 'isolcpus' kernel parameter doesn't fully isolate your critical threads, and the combination of settings required for true deterministic scheduling.
Continue Reading
Enjoyed this?
Get one deep infrastructure insight per week.
Free forever. Unsubscribe anytime.
You're in. Check your inbox.