Write-up
Degraded RAG performance

Summary: On February 10, 2026, between 22:30 and 01:48 UTC, users experienced elevated latency when using RAG, with RAG chunk retrieval in conversations responding more slowly than normal. The root cause was identified as an internal, unoptimized database query that increased database load and degraded performance.

Timeline:

22:30 UTC: Elevated latency begins affecting RAG chunk retrieval in conversations.

23:36 UTC: The issue is identified from an alarm as increased latency tied to a slow internal database query.

23:36–01:48 UTC: Database performance is optimized and mitigations are applied to reduce load, while we continue monitoring latency.

01:48 UTC (Feb 11): Latency returns to normal levels and the incident is considered resolved.

Preventative actions & learnings:

• We are improving our detection and testing so slow-running queries are caught earlier, before they can impact production performance.
• We are adding additional safeguards around query performance to reduce the chance of similar latency regressions in the future.