Resolved
All behavior remains normal, we are considering this incident resolved.
Identified
We are seeing a recovery in all workflow and broadcast processing delays. We are monitoring closely.
Identified
We've taken steps to shuffle around work in our system which has meant that we're processing new work for all customers again. We still have some queued workflow executions that we're working through and expect to be fully caught up in the next 40 minutes.
Identified
One of our workflow queues is currently still backlogged, but we're seeing good resolution across other areas of our service and we have effectively cordoned off the source of the issue. We're burning through remaining work and are exploring ways to shed the load further to get back to stability.
Investigating
We're seeing some recovery on our services but are dealing with delays in some workflow and broadcast processing still. Our logging service is also currently delayed, so may not reflect the latest as we catch up and work through a huge influx of work.
Investigating
We're seeing an extremely high amount of load on our services which is causing workflow and broadcast delays. We're working to relieve the pressure immediately.
Investigating
We're currently investigating an issue related to slow notification processing and message delivery. Some workflows will be delayed