Infrastructure
The 100ms Tax: Why Most Block Builders Lose Before They Start
A first-principles breakdown of why infrastructure latency, simulation failure rates, and observability gaps eat block builder margin — and how to fix it.
At a builder processing many bundles per block, simulation failure rate matters enormously. The bundles that fail simulation are disproportionately the most valuable ones — complex arbitrages with tight timing windows. When your simulation times out on a 12-hop DEX route, you don’t just lose a bundle. You lose the only bundle that mattered that block.
This post documents the infrastructure failures that turn profitable MEV strategies into losses, and the observability patterns required to find them.
1. The Physics of Block Propagation
Ethereum produces a block every 12 seconds. Builders compete to have their block included by proposers. The competition is decided by bid value — the amount the builder pays the proposer.
But bid value is a function of bundle quality, and bundle quality degrades rapidly:
The “Stale State” Problem:
If a builder simulates a bundle against state that is 200ms old, the bundle may reference a DEX pool that has already been arbitraged. The simulation “succeeds” but the execution reverts.
- Source: Flashbots MEV-Boost Architecture
2. Simulation Infrastructure Options
| Approach | Simulation Latency | State Freshness | Verdict |
|---|---|---|---|
| A. Single Geth Node (Default) | ~50ms | Stale by 1-2 blocks | High revert rate. |
| B. Erigon + In-Memory Cache | ~10ms | Stale by 200ms | Better. Still loses tail bundles. |
| C. Custom State Trie + Block Sync | <5ms | Fresh to pending block | Required for competitive building. |
The Math:
In a 12-second block time, a builder receiving bundles at t=11.5s has 500ms to simulate and bid. If simulation takes 50ms and you process sequentially, you handle 10 bundles. If it takes 5ms, you handle 100. The builder with better infrastructure wins by volume.
3. The “Silent Killer”: Partial Fills and Revert Rates
Most builders track “bundles received” and “bundles included.” They rarely track:
- Simulation Timeout Rate: Bundles that took >100ms to simulate and were dropped.
- Execution Revert Rate: Bundles that simulated successfully but reverted on-chain.
- Partial Fill Rate: Multi-leg bundles where only some transactions succeeded.
Why This Matters
A non-trivial execution revert rate directly reduces margin. On-chain reverts are visible in receipts and can be measured — if you’re instrumenting your pipeline correctly. Most builders aren’t.
4. The “Bundle Half-Life” Metric
A useful observability metric: Bundle Half-Life.
Definition: The median time from searcher submission to block inclusion (or drop).
This single metric captures:
- Network propagation latency (searcher → builder)
- Simulation queue depth
- Block propagation latency (builder → proposer)
Implementation
# Pseudocode for Bundle Half-Life tracking
bundle_events = []
def on_bundle_received(bundle_id, timestamp):
bundle_events.append({"id": bundle_id, "received": timestamp})
def on_bundle_included(bundle_id, block_number, timestamp):
event = find_event(bundle_id)
event["included"] = timestamp
event["half_life"] = event["included"] - event["received"]
emit_metric("bundle_half_life_seconds", event["half_life"])
```bash
**Dashboard Thresholds:**
* P50 Half-Life > 1s: **Warning.** You're losing to faster builders.
* P99 Half-Life > 5s: **Critical.** Your tail bundles are worthless.
## 5. The Observability Stack
The key metrics for MEV infrastructure are not the standard RED metrics (Rate, Errors, Duration). They are:
| Metric | Source | Threshold |
| :--- | :--- | :--- |
| **Bundle Acceptance Rate** | Builder API logs | < 95% = investigate |
| **Simulation Success Rate** | EVM tracer | < 99% = infra problem |
| **Execution Revert Rate** | On-chain receipts | > 1% = state freshness issue |
| **Bundle Half-Life (P99)** | Custom tracing | > 2s = you're losing |
```bash
# Example Prometheus queries
sum(rate(bundle_received_total[5m])) by (searcher)
histogram_quantile(0.99, rate(bundle_half_life_seconds_bucket[5m]))
6. Trade-offs
- Infra Cost vs. Margin: Running a custom state trie requires significant engineering investment. Make sure your bundle volume justifies it.
- Latency vs. Correctness: Faster simulation can mean less accurate gas estimation, leading to underpriced bundles.
- Observability Overhead: Every metric you emit adds latency. Instrument selectively.
7. The Core Insight
In MEV, infrastructure is not a cost center. It is a profit function.
The difference between a profitable builder and a money-losing one is often not strategy — it’s the simulation latency that causes reverts. DevOps in this domain is not “keeping the lights on.” It is directly moving the P&L.
Need Help With MEV Infrastructure?
Building block infrastructure or optimizing your MEV strategy? I help trading firms and protocols design low-latency systems. Let’s discuss your architecture →
Up Next in On-Chain Infrastructure: MEV
Antifragile MEV: How We Profit When the Network Breaks
Applying Nassim Taleb's antifragility to blockchain execution infrastructure. Why reorgs are profit opportunities, multi-builder hedging is arbitrage, and chaos engineering is a competitive advantage.
Continue Reading
Enjoyed this?
Get one deep infrastructure insight per week.
Free forever. Unsubscribe anytime.
You're in. Check your inbox.