All Posts
Backend 10 min read

Scaling WebSockets to 50,000 concurrent users without breaking the bank

#websockets#node.js#redis

Live bidding apps live and die by latency. If bidder A submits $450 at millisecond 100, bidder B at millisecond 120 must see that bid before they place their own. Our client ran an estate auction site that worked fine with 800 users. Then a celebrity collection went live, and 48,000 users joined within ten minutes. The Node.js event loop froze, socket handshakes timed out, and memory crested past 6GB.

The architectural bottleneck

The initial design held all client socket connections on a single large EC2 instance. Broadcast messages iterated through an in-memory Map of sockets. As the connection list grew, JSON serialization and looping over 40,000 connections during rapid-fire bids took over 400ms per broadcast. That blocked the event loop from answering incoming TCP handshakes, causing a cascading socket disconnect storm.

Decoupling connection state from business logic

We re-architected the system into three distinct layers:

1. Socket Gateway Nodes: Lightweight Node.js / uWebSockets.js instances whose only job is maintaining TLS socket connections and message framing. No database queries, no business rules. 2. Message Broker: Redis Cluster running Pub/Sub channels partitioned by auction room IDs. 3. Core Auction Engine: Dedicated Go services that process incoming bids atomically using Redis Lua scripts, ensuring absolute ordering and preventing race conditions.

Optimizing payloads and connection churn

Broadcasting large JSON blobs 10 times a second per room saturated network egress bandwidth quickly. We made two crucial optimizations:

- Binary delta compression using Protocol Buffers instead of verbose JSON payloads, reducing packet size by 74% - Heartbeat batching: staggering client ping/pong intervals over random jitter windows to eliminate synchronized keep-alive thundering herds - Connection throttling at the NGINX reverse proxy to queue TLS handshakes during massive traffic spikes

Production results

During the subsequent high-profile auction with 52,000 simultaneous connections across four gateway nodes:

- P99 broadcast latency stayed under 28 milliseconds - CPU utilization per gateway node hovered steadily under 35% - Total infrastructure spend was reduced by 40% compared to running monolithic oversized compute instances.

Have a similar challenge?

We build production-grade software for companies that need it done right.

Let's Talk