Intermittent Connection Delays

Incident Report for Todyl

Resolved

This incident has been resolved.
Posted Dec 15, 2023 - 03:26 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Dec 14, 2023 - 22:22 UTC

Update

95% of all devices currently connected. The SRE team is working to address the remaining devices.
Posted Dec 14, 2023 - 22:10 UTC

Update

Our SRE Team is working to establish the remaining connections.
Posted Dec 14, 2023 - 21:24 UTC

Update

The SRE team continues to increase the rate at which devices are connecting to the SGN. We will continue to increase this threshold, allowing more connections faster.
Posted Dec 14, 2023 - 20:47 UTC

Update

The SRE team is increasing the rate at which devices are connecting.
Posted Dec 14, 2023 - 20:08 UTC

Update

Todyl's SRE team is continuing to work to increase the rate of connections, as well as number of devices connecting to the SGN.
Posted Dec 14, 2023 - 19:42 UTC

Update

The team is continuing to monitor and allow additional connections.
Posted Dec 14, 2023 - 19:26 UTC

Update

The team is continuing to monitor and allow more connections.
Posted Dec 14, 2023 - 18:55 UTC

Update

The team is continuing to allow additional connections.
Posted Dec 14, 2023 - 18:15 UTC

Update

The team is continuing to allow additional connections at an increased rate.
Posted Dec 14, 2023 - 17:48 UTC

Update

There is no major update at this time. The team is continuing to increase the number of connected devices. We will update in 15 minute intervals as connection counts continue to increase.
Posted Dec 14, 2023 - 17:34 UTC

Update

The team is continuing to increase the number of connected devices. We will update in 15 minute intervals as connection counts increase.
Posted Dec 14, 2023 - 17:17 UTC

Update

At 13:33 UTC our US-EAST AWS Cache suddenly went offline causing new SASE connections fail. At the present time several other major SaaS vendors are experiencing degraded service and outages, we are unsure if related at this time. The SRE Team has recovered and stabilized the connection infrastructure, and is currently increasing the number of devices allowed to connect. We will update in 15 minute intervals as connection counts increase.
Posted Dec 14, 2023 - 16:53 UTC

Update

Engineering is continuing to monitor the connections of devices and address this issue. An update will be posted within the next 30 minutes. We are working to resolve this issue as quickly as possible.
Posted Dec 14, 2023 - 16:31 UTC

Update

We are continuing to investigate this issue. Engineering is continuing to monitor the connections of devices. An update will be posted within the next 30 minutes. We are working to resolve this issue as quickly as possible.
Posted Dec 14, 2023 - 16:00 UTC

Update

The team is continuing to resolve connectivity issues to the SGN. An update will be posted within 30 minutes if one is not available sooner.
Posted Dec 14, 2023 - 15:42 UTC

Update

The team is continuing to address this issue. An update will be posted within the next 30 minutes if one is not available sooner.
Posted Dec 14, 2023 - 15:05 UTC

Identified

The issue has been identified and a fix is being implemented.
Posted Dec 14, 2023 - 14:37 UTC

Monitoring

Devices should be connecting to the SGN, if they have not already.
Posted Dec 14, 2023 - 14:21 UTC

Investigating

Some devices may be experiencing a delay in connecting to the SGN. We are currently investigating this and an update will be posted in 30 minutes if one is not available sooner.
Posted Dec 14, 2023 - 13:54 UTC
This incident affected: Secure Global Network (SASE) (Secure Global Network).