Nov 18, 2026, 14:49–20:55 UTC (peak impact 15:45–17:15 UTC), with a related smaller disruption on Nov 19, 14:10–15:20 UTC.
At peak on Nov 18, we observed substantial error rates outside of TTS, STT, and Agent conversation endpoints, Text to Speech experienced elevated time-to-first-audio and 3.4% failure rate, inbound calls to ElevenAgents that used SIP and Twilio Native experienced high call connection rate failures between 15:45-17:15 UTC, with other Agent conversation methods remaining available and unimpacted throughout. 12%–50% of requests failed across Voices, Creative, Studio, Music, Reader, and login/workspace routes. Impact was primarily in the US, with limited voice failures in EU and India residencies limited to specific voices that were shared from the US environment. (Singapore was unaffected).
A burst of concurrent requests triggered repeated full cache rebuilds on the voice caching cluster, exhausting its memory and client buffers. Because backend services had no upper bound on cache connections, slowed responses caused application servers to open connections until the cluster hit its maximum client limit, failed health checks, and entered a repeating restart cycle with an empty cache. Queued requests on application servers then cascaded this connection exhaustion to other caching clusters.
On 19 November, a server hosting one of our caches reached its limit for new network connections, and connections to that cache timed out. This was related to the 18 November incident but much smaller in scale. Calls that ended during this window were not consistently removed from ElevenAgents’ active-call counts which was stored in the impacted cache, so those counts reached customers’ concurrency limits, and about 10% of new conversations overall were refused between 14:10 and 15:20 UTC. We cleared the stale counts at 15:17 and have since raised this connection limit on all servers that host these caches.
– We have already increased the capacity of our caching clusters and added limits within our services to ensure our clusters are not overloaded in this same way.
– We are conducting a full review of our caching layer across all engineering teams to add additional hardening where needed. This includes validating isolation between clusters, ensuring correct rate-limiting is in place and improving health checks.
– We will be conducting a review with our dev ops team to improve how caching/datastore services are provisioned in future, ensuring all future stores follow best practise.
– We will remove EU/IN voice dependencies on the US environment providing greater resiliency to these environments.