Performance degradation
Incident Report for awork.com
Postmortem

🟢 Summary

On March 6, 2023, some awork users had performance issues that got worse twice during the day. The issue caused slow loading times and temporary service outages from awork. It was fixed during the day.

We apologize for the trouble we caused you on Monday and thank you for your patience and understanding.

🔍 Explanation

As part of the latest update, major changes were made to core services to improve the overall performance of awork. Although we tested and approved all changes carefully, a very high number of requests resulted in inefficient use of the underlying database. This was identified as the root cause of the issue.

To tackle this, we scaled up the services and started to further investigate and fix the responsible requests and database queries. The process required several attempts to clearly identify which operations were responsible and needed to be resolved.

After doing so, recovering the core services led to the initially expected, greatly improved query performance. This status was reached around 6 PM on March 6, 2023.

Posted Mar 06, 2023 - 23:38 CET

Resolved
This incident has been resolved.
Posted Mar 06, 2023 - 18:30 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 06, 2023 - 18:26 CET
Investigating
We are currently investigating this issue.
Posted Mar 06, 2023 - 17:39 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 06, 2023 - 16:06 CET
Investigating
We are currently investigating this issue.
Posted Mar 06, 2023 - 15:06 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 06, 2023 - 12:18 CET
Investigating
We are currently investigating this issue.
Posted Mar 06, 2023 - 12:06 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 06, 2023 - 11:14 CET
Investigating
We are seeing loading issues - investigating the root cause.
Posted Mar 06, 2023 - 09:30 CET
This incident affected: Web-App and API (Tasks, Projects, Users).