The Connection That Lies to You
A quick note on language and scope before we get into it. I write this series in English because Polymarket isn’t accessible in every jurisdiction, and the audience for this kind of technical detail is scattered globally. I also stay deliberately vague about anything that constitutes edge — which wallets I follow, how I filter entries, how I size positions. That part stays private. What I share freely is the engineering: the failures, the fixes, and what they forced me to understand. This entry is purely infrastructure. There is no alpha in connection management; there is only reliability or the lack of it.
What a Quiet Failure Looks Like
Prediction markets don’t close at 4 PM. They run through weekends, through holidays, through whatever hours you happen to be asleep. A bot meant to operate continuously has to stay connected to a live data feed for all of that time. The naive assumption — the one I made and then paid for — is that a broken connection announces itself. It doesn’t, not always.
The failure mode I eventually ran into wasn’t a crash. It wasn’t a refused connection or a stack trace in the logs. It was silence. The WebSocket stayed open. The connection object reported a healthy state. No exception was raised. But messages had stopped arriving. The feed had gone deaf, and the bot had no idea, because from where the bot was sitting, the socket existed and looked fine. The world was moving — orders were being placed, probabilities were shifting, positions were being opened and closed — and the bot was watching a frozen picture of the market from several minutes ago, cheerfully unaware.
This is the scariest failure in any system that depends on a continuous stream: not the loud crash you can detect and page yourself about, but the quiet stall that passes every surface-level health check while the underlying truth has quietly rotted.
Heartbeats: Proving the Pipe Is Actually Flowing
The fix for silent staleness is to stop trusting the connection object’s status and start trusting observed behavior instead. Specifically: if data is actually flowing, messages arrive. If no message has arrived within some defined window, assume the pipe is dead regardless of what the connection object says, and reconnect.
This is a heartbeat check, and it’s conceptually simple. You record a timestamp every time a message comes in. On a separate timer, you compare that timestamp to now. If the gap exceeds your threshold, you treat it as an outage and tear the connection down deliberately, then open a new one.
import asyncio
import time
HEARTBEAT_TIMEOUT = 30 # seconds without a message before we act
class FeedConnection:
def __init__(self):
self.last_message_at = time.monotonic()
self.ws = None
def on_message(self, raw):
self.last_message_at = time.monotonic()
# ... parse and handle raw ...
async def heartbeat_loop(self):
while True:
await asyncio.sleep(10)
elapsed = time.monotonic() - self.last_message_at
if elapsed > HEARTBEAT_TIMEOUT:
print(f"No message for {elapsed:.1f}s — assuming dead socket, reconnecting")
await self.reconnect()
async def reconnect(self):
if self.ws is not None:
try:
await self.ws.close()
except Exception:
pass # already dead, doesn't matter
await self.connect()
One subtlety worth noting: the threshold needs to be calibrated to actual traffic patterns. During busy market hours, messages arrive frequently and a thirty-second gap is unambiguous. During a slow period — an obscure market with very little activity — the feed can legitimately go quiet for a stretch. I handle this by also sending explicit WebSocket ping frames and expecting pong responses; the application-level heartbeat covers semantic staleness, while the protocol-level ping covers socket-layer liveliness. Both are necessary because they catch different failure modes.
Backoff: Not Hammering a Server That’s Already Struggling
Once you’re willing to reconnect, you need to reconnect responsibly. If the upstream service is having an incident — a restart, a brief outage — and you immediately hammer it with reconnection attempts, you’re adding load to something already stressed, and you’ll likely trigger rate limiting on top of the original problem.
Exponential backoff with jitter is the standard answer. Each successive failed attempt waits longer before retrying, and you add a small random offset so that a fleet of clients (or in my case, multiple bot instances) don’t all retry in lockstep.
import random
async def connect_with_backoff(self, max_attempts=10):
delay = 1.0
for attempt in range(max_attempts):
try:
await self.connect()
print(f"Connected on attempt {attempt + 1}")
return
except Exception as e:
jitter = random.uniform(0, delay * 0.3)
wait = delay + jitter
print(f"Attempt {attempt + 1} failed ({e}), retrying in {wait:.1f}s")
await asyncio.sleep(wait)
delay = min(delay * 2, 60) # cap at 60 seconds
raise RuntimeError("Could not reconnect after maximum attempts")
The cap at sixty seconds is a deliberate choice. I don’t want the bot sitting idle for five minutes on the off-chance the service is down; I’d rather fail loudly and alert myself at that point. Backoff is for transient blips, not extended outages. Those deserve human attention.
The Gap Problem: Being Honest About What You Missed
Reconnecting is the easy half. I didn’t fully appreciate this until I had reconnection working and then watched the bot make decisions based on state that was already stale.
Here’s the problem. Say the connection drops for ninety seconds. During those ninety seconds, a market moves significantly. Positions change. The probability you were tracking shifts. When the connection comes back, the live feed starts delivering current events — but your internal representation of the market still reflects the state from before the outage. You are live again but wrong.
The gap has to be reconciled before you trust your own state. How you do this depends on what the API offers. The Polymarket CLOB API exposes REST endpoints that let you query current order book state and recent trade history. On reconnect, before resuming normal operation, I pause the event-processing pipeline and do a full state refresh via REST: pull the current book, pull recent fills, reconcile against what I had, and only then resume listening to the live stream.
async def reconnect(self):
await self.close_existing()
await self.connect_with_backoff()
await self.reconcile_missed_state() # REST-based catch-up
self.resume_processing()
async def reconcile_missed_state(self):
print("Reconciling state after reconnect...")
# Fetch current snapshots for all tracked markets via REST
for market_id in self.tracked_markets:
snapshot = await self.rest_client.get_market_snapshot(market_id)
self.state.apply_snapshot(market_id, snapshot)
print("Reconciliation complete, resuming live feed")
The order matters enormously. Connect first, then reconcile, then resume. If you reconcile before connecting, you might miss events that arrive in the window between your REST snapshot and your WebSocket subscription. If you resume before reconciling, you’re processing live deltas against stale base state. The sequence is: socket open, REST snapshot, then live events layered on top.
This also means accepting that there’s a brief window during reconciliation where you’re deliberately not acting. That’s intentional. Acting on incomplete information is worse than pausing. The bot needs to be honest about when it doesn’t have a reliable picture of the world, and the reconciliation step is the mechanism for enforcing that honesty structurally rather than relying on me to remember it.
What This Changed About How I Think About the System
Before working through this, I thought of connection reliability as binary: either the bot was connected or it wasn’t, and the task was to stay connected. What I understand now is that connectivity and correctness are separate properties. You can have one without the other. A connected-but-stale system can be more dangerous than a cleanly disconnected one, because at least a disconnected system knows it’s offline.
The real work wasn’t writing the reconnection logic. It was building in the discipline to distrust the bot’s own state after an interruption and force a reconciliation before acting. That instinct — assume you might be wrong after any gap, and verify before proceeding — turns out to apply more broadly than just WebSocket connections. It’s a reasonable posture for any system that depends on external state it doesn’t fully control.
Next up: what happens when the reconciliation step itself fails, and how to handle a world where both the live feed and the REST fallback are returning inconsistent data at the same time.