Agents, STT, and Creative Platform: Increased Latency, Conversation Failures, and STT Failures

Write-up

Root Cause Analysis: Increased Latency and Failures Across Agents, Speech to Text, and Creative Platform

Duration:

September 29, 2026, 15:56 UTC to 19:36 UTC

Description of the Issue:

Customers experienced increased latency and failed requests on ElevenAgents conversations, Speech to Text, and Creative Platform products, including Sound Effects.

Root Cause:

Processing a request included recording and storing its usage metrics. Under peak traffic the internal usage datastore became overloaded and stopped accepting new connections. Because requests waited on this step to complete, the slowdown affected every product that records usage in it.

Timeline & Actions Taken:

15:56 UTC: Latency and error rates begin rising across affected products.

16:34 UTC: Alert fires; engineering begins investigating.

17:32 UTC: Issue localized to the internal usage datastore.

18:26 UTC: Incident declared as impact on Agents and Speech to Text is confirmed.

19:36 UTC: Usage metric datastore recovers; error rates return to baseline.

20:23 UTC: Full recovery confirmed across all affected products.

Preventative Actions & Learnings:

Decoupling usage recording from request processing, so the usage metric datastore can no longer affect request latency or availability.