Gladia - Real Time Instabilities – Incident details

App experiencing degraded performance

Real Time Instabilities

Resolved
Operational
Started 21 days agoLasted about 10 hours

Affected

Real-Time v1

Major outage from 4:00 AM to 4:40 AM, Operational from 4:40 AM to 2:10 PM

Real-Time v2

Major outage from 4:00 AM to 4:40 AM, Operational from 4:40 AM to 2:10 PM

Updates
  • Postmortem
    Postmortem

    Between 03:30 and 11:30 UTC, our real-time (RT) WebSocket service experienced intermittent connection issues due to an infrastructure scaling problem. A temporary workaround was deployed to stabilize performance while the underlying issue was identified and resolved. The root cause was a change in our cloud provider’s API that affected our autoscaling logic. The issue has since been fully resolved, and we have implemented safeguards to prevent recurrence. We sincerely apologize for the disruption and appreciate your patience.

  • Resolved
    Resolved

    A fix has been deployed. This incident has been resolved.

  • Identified
    Identified

    A workaround is in place to allow more capacity until we fix the root cause.

  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    Identified

    We have an issue scaling ressources to handle Real Time requests.

  • Investigating
    Investigating

    We are currently investigating this incident.