Performance Event: 2022-09-20

Issue: Widespread outage across districts; various 502 Bad Gateway errors.

Event 1 - Students experiencing slowness while Online Testing
Event 2 - Users receiving 502 Bad Gateway error messages

Event 1
September 20, 2022 10:27-11:20 AM CDT

Event 2
September 20, 2022 11:20-11:55 AM CDT

Status: Resolved

Summary of most recent events

Error rates for Online Testing as well as one of our Database Cluster’s CPU capacity increased at the same time that container instances were scaling up. The system went down for approximately 35 minutes due to a combination of the events above as well as a single Database Cluster failing over on its own under load. We restored full functionality to software on September 20, 2022 at 11:55 AM.

Immediate next steps taken by Eduphoria

We will continue to investigate what caused the increase of container instances and error rates. Furthermore, efforts are underway to establish solutions that will better isolate customer processes to limit cascading failures.