{
"vendor": "Workspot",
"slug": "workspot",
"platform": "statuspage",
"status_url": "https://status.workspot.com",
"last_checked": "2026-09-16T12:28:20Z",
"last_state": "ok",
"history_backfilled": true,
"first_watched": "2026-09-04T07:06:16Z",
"incidents": [
{
"body": "Update from Microsoft: \n\nCurrent Status: Resolved\n\nWhat went wrong and why?\nDue to an increase in demand for compute resources in a particular zone in the region, there were capacity constraints on available compute infrastructure. This in turn resulted in service management operation and allocation failures for VM resources.\n\nHow did we respond?\nWe\u2019ve load balanced the capacity demands and restored the capacity buffers to a healthy state. With the congestion resolved, services should now function as usual.\n\nWhat happens next?\n\u2022\tEnsuring capacity for our customers is a top priority for Microsoft. We sincerely apologize for any impact this may have caused, but be assured, we are continuously improving tools around monitoring usage, forecasting stock out, maintaining healthy buffers and improving cycle times to reduce this type of issue from happening.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-01T21:47:51.704Z",
"resolved_inferred": false,
"started_at": "2026-05-21T16:05:26.000Z",
"state": "resolved",
"title": "Azure East US Capacity Constraints Impacting Virtual Desktop Availability",
"updated_at": "2026-06-01T21:47:51.718Z",
"url": "https://stspg.io/n4cbgp4d0gcr"
},
{
"body": "Workspot investigated the restart of the Control Dynos reported by our platform provider. The system is stable and we are closing the incident on status, but continuing to actively monitor and investigate the root cause.  \nWe will share the RCA and updates as soon as they become available. Thank you.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-06T04:32:45.501Z",
"resolved_inferred": false,
"started_at": "2026-05-05T12:08:30.488Z",
"state": "resolved",
"title": "Intermittent Restarts of Control Dynos",
"updated_at": "2026-05-06T04:32:45.519Z",
"url": "https://stspg.io/83zmh55s13tv"
},
{
"body": "Update:\n \nWe have received the Preliminary Post Incident Review(PIR) from Microsoft for the Azure East US control plane issue that impacted your environment on April 24.\n \nHere is a summary of what happened and why:\n \nSTATUS:  RCA\n \nCOMMUNICATION:\n \nThis is our Preliminary PIR to share what we know so far. After our internal retrospective is completed (generally within 14 days) we will publish a Final PIR with additional details.\n \nWhat happened?\n \nBetween 11:30 UTC on 24 April and 00:15 UTC on 25 April 2026, customers may have experienced failures or delays when attempting to provision, scale, or update resources in East US. Beyond this, a smaller subset of impacted customers may have experienced intermittent connectivity issues on existing workloads (including Virtual Machines and Azure Virtual Desktop sessions) for scenarios dependent on unhealthy internal service dependencies.\n \nThe issue initially began with impact to a subset of customers in a single Availability Zone (physical AZ-01) but as demand shifted, similar symptoms were observed impacting a subset of customers in AZ-02 and AZ-03. While none of these zones were impacted for the full duration of the incident, customers experienced periods of impact in each zone for portions of the incident.\n \nThe following services were among those affected: Azure Application Gateway, Azure App Service, Azure Batch, Azure Cache for Redis, Azure Data Explorer, Azure Data Factory, Azure Databricks, Azure Health Data Services, Azure Kubernetes Service (AKS), Azure Red Hat OpenShift, Azure Service Fabric, Azure Synapse Analytics, Azure Virtual Desktop, Azure Virtual Machines, Azure Virtual Network Manager, Azure VMware Solution, Oracle Database@Azure, Virtual Machine Scale Sets \u2013 and potentially additional services that were dependent on new compute allocations in the region.\n \nNote: Logical availability zones assigned to customer subscriptions may map to different physical availability zones. Customers can use the Locations API to understand this mapping: https://learn.microsoft.com/rest/api/resources/subscriptions/list-locations?HTTP#availabilityzonemappings.\n \nWhat went wrong and why?\n \nThe Azure PubSub service is a key component of the networking control plane, acting as an intermediary between resource providers and networking agents on Azure hosts. Resource providers, such as the Network Resource Provider, publish customer configurations during Virtual Machine or networking create, update, or delete operations. Networking agents (subscribers) on the hosts retrieve these configurations to program the hosts networking stack. Additionally, the service functions as a cache, ensuring efficient retrieval of configurations during VM reboots or restarts. This capability is essential for deployments, resource allocation, and traffic management in Azure Virtual Network (VNet) environments.\n \nDuring normal platform operations, one partition of this PubSub service in AZ-01 became unhealthy and automatically attempted to fail over to a secondary replica. The failover did not complete successfully, resulting in a partial loss of control plane availability within AZ-01. We intervened to investigate and attempted a manual failover of the primary partition, but this attempt was also unsuccessful.\n \nShortly afterward, we observed a similar condition in AZ-03, which led to a partial loss of control plane availability in AZ-03 as well. As the investigation progressed, we suspected that a previously deployed update to a regional control plane dependency had introduced a latent regression. This issue did not surface during earlier validation and only manifested when failover conditions were triggered under sustained production load.\n \nAs part of our mitigation efforts, we identified a version from the prior week that represented a Last Known Good (LKG) state. We first applied this rollback in AZ-03, which successfully restored control plane service health in that zone. Based on this, we began rolling back the affected components in AZ-01. By design, rollback operations are executed in stages by Azure Fabric controllers using update domains to ensure platform safety, while recovery proceeds incrementally.\n \nWhile mitigation was in progress, the platform was unable to maintain two healthy instances of the PubSub service across availability zones simultaneously, which is a requirement for normal replication and control plane operations. This resulted in a loss of quorum of the service. As the system attempted to rebalance, impact shifted between availability zones, leading to periods of degraded behavior across multiple zones.\n \nSimilar failure patterns began to appear in AZ-02 and again in AZ-03, expanding the scope of impact across the region. For AZ-02 we initiated and completed a rollback, and although AZ-03 had previously shown recovery following the rollback, subsequent instability indicated that the rollback in that zone had not fully completed, because of an orchestration fault. As impact reemerged, rollback operations in AZ-03 were restarted and then completed, fully restoring service health.\n \nHow did we respond?\n \n\u2022\t11:30 UTC on 24 April 2026 \u2013 Customer impact began. We observed failures or delays when customers attempted to provision, scale, or update resources in the affected region.\n\u2022\t11:38 UTC on 24 April 2026 \u2013 We detected an issue in AZ-01. A control plane partition became unhealthy and automatic failover attempts did not complete successfully.\n\u2022\t11:38\u201313:40 UTC on 24 April 2026 \u2013 We attempted manual failover in AZ-01. These efforts did not successfully restore service.\n\u2022\t13:40 UTC on 24 April 2026 \u2013 We identified a recently deployed update as the likely cause of the issue.\n\u2022\t13:50 UTC on 24 April 2026 \u2013 We began observing similar symptoms in AZ-03, indicating the issue was affecting multiple availability zones.\n\u2022\t14:07 UTC on 24 April 2026 \u2013 We initiated rollback to a previously known good version in AZ-03.\n\u2022\t15:03 UTC on 24 April 2026 \u2013 We observed significant recovery in AZ-03. Control plane availability exceeded 99%.\n\u2022\t15:04 UTC on 24 April 2026 \u2013 We initiated rollback actions to AZ-01.\n\u2022\t18:52 UTC on 24 April 2026 \u2013 We observed significant improvement in AZ-01 as rollback progressed.\n\u2022\t19:02 UTC on 24 April 2026 \u2013 We confirmed AZ-01 had recovered to greater than 99% availability, while the rollback continued in the background.\n\u2022\t19:05 UTC on 24 April 2026 \u2013 We observed similar symptoms in AZ-02 as load redistributed across the region.\n\u2022\t19:10 UTC on 24 April 2026 \u2013 We initiated rollback to a known good version in AZ-02.\n\u2022\t21:02 UTC on 24 April 2026 \u2013 We observed instability reappear in AZ-03. We determined this was because the rollback had not yet completed across all update domains. Consequently, we manually unblocked the rollback across remaining update domains in AZ03 to ensure stable recovery.\n\u2022\t22:39 UTC on 24 April 2026 \u2013 We confirmed rollback was fully completed in AZ-03.\n\u2022\t23:22 UTC on 24 April 2026 \u2013 We confirmed rollback was fully completed in AZ-02, completing PubSub mitigation across all affected zones.\n\u2022\t00:15 UTC on 25 April 2026 \u2013 We validated downstream service recovery and PubSub health across all zones in the region.\n \nHow are we making incidents like this less likely or less impactful?\n \n\u2022\tWe have assessed the risk of occurrence in other high volume regions, and have taken steps to rollback this PubSub service in these regions out of an abundance of caution. (Completed)\n\u2022\tWe are investing in improving our test coverage surrounding the failure cases and load patterns that contributed to this incident, to catch issues like this one before they reach production. (Estimated completion: TBD)\n\u2022\tWe are working to reduce rollback complexity, to be able to mitigate issues like this more quickly in future. (Estimated completion: TBD)\n\u2022\tThis is our Preliminary PIR to share what we know so far. After our internal retrospective is completed (generally within 14 days) we will publish a Final PIR with additional details.   \n \nHow can customers make incidents like this less impactful?\n \n\u2022\tConsider using Availability Zones (AZs) to run your services across physically separate locations within an Azure region. To help services be more resilient to localized failures like this one (which predominantly impacted zones at different times) many Azure services support zonal, zone-redundant, and/or always-available configurations: https://docs.microsoft.com/azure/availability-zones/az-overview\n\u2022\tFor mission-critical workloads, customers should consider a multi-region geodiversity strategy to avoid impact from incidents like this one that impacted a single region: https://learn.microsoft.com/azure/architecture/patterns/geodes and https://learn.microsoft.com/azure/well-architected/design-guides/regions-availability-zones\n\u2022\tMore generally, consider evaluating the reliability of your applications using guidance from the Azure Well-Architected Framework and its interactive Well-Architected Review: https://aka.ms/AzPIR/WAF\n\u2022\tThe impact times above represent the full incident duration, so are not specific to any individual customer. Actual impact to service availability varied between customers and resources \u2013 for guidance on implementing monitoring to understand granular impact: https://aka.ms/AzPIR/Monitoring\n\u2022\tFinally, consider ensuring that the right people in your organization will be notified about any future service issues \u2013 by configuring Azure Service Health alerts. These can trigger emails, SMS, push notifications, webhooks, and more: https://aka.ms/AzPIR/Alerts",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-28T06:52:36.029Z",
"resolved_inferred": false,
"started_at": "2026-04-24T13:31:39.174Z",
"state": "resolved",
"title": "VM Connectivity Issues \u2013 Azure East US",
"updated_at": "2026-04-28T06:52:36.067Z",
"url": "https://stspg.io/c2k0zn43l7yg"
},
{
"body": "The system has been stable and functioning normally for the past 48 hours. Our internal investigation found no application-level anomalies.\n\nWe are closing this incident while the root cause analysis (RCA) from our infrastructure provider is still ongoing. We will share updates as soon as they become available.\n\nThank you for your patience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-03-24T05:17:11.136Z",
"resolved_inferred": false,
"started_at": "2026-03-20T18:09:24.602Z",
"state": "resolved",
"title": "Workspot Control Reported Five Minutes Downtime",
"updated_at": "2026-03-24T05:17:11.155Z",
"url": "https://stspg.io/fr45njm1g65x"
},
{
"body": "The service has been fully restored after the rollback was performed. We appreciate your patience. \n\nA detailed Root Cause Analysis (RCA) will be shared once available.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-02-04T03:28:59.488Z",
"resolved_inferred": false,
"started_at": "2026-02-02T09:38:47.985Z",
"state": "resolved",
"title": "Workspot Control Experiencing Intermittent Availability",
"updated_at": "2026-02-04T03:28:59.521Z",
"url": "https://stspg.io/lyw2n6x10nty"
},
{
"body": "This issue has been resolved, and customers can now log in to Workspot Control and access non-persistent VMs without any issues. The platform is stable, and normal operations have resumed. We will continue to monitor the environment to ensure stability. \n \nWe will share a post-incident summary once our investigation ends and relevant details are available.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-11-14T18:30:55.947Z",
"resolved_inferred": false,
"started_at": "2025-11-14T15:57:43.419Z",
"state": "resolved",
"title": "Workspot Control Login Instability",
"updated_at": "2025-11-14T18:30:55.962Z",
"url": "https://stspg.io/v00q4hfv7q7z"
},
{
"body": "This incident has been resolved, and no Workspot customers have reported experiencing the issue.\n\nHere is the summary of the AWS incident: https://aws.amazon.com/message/101925/",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-31T06:51:04.014Z",
"resolved_inferred": false,
"started_at": "2025-10-20T17:11:29.178Z",
"state": "resolved",
"title": "AWS Operational Issues - No Impact - Workspot is closely monitoring the situation",
"updated_at": "2025-10-31T06:51:04.029Z",
"url": "https://stspg.io/xvfz06l6vsmk"
},
{
"body": "This issue has been resolved in accordance with the Microsoft article: https://learn.microsoft.com/en-us/windows/release-health/status-windows-11-25h2#issue-details.\n\nThe problem is addressed in the October Windows non-security preview update (KB5067036) and all subsequent updates. Microsoft recommends installing the latest Windows updates, as they include important improvements and fixes, including this resolution.\n\nAdditionally, we have not received any further reports of this issue from customers who have applied the recommended Microsoft updates.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-31T06:45:45.183Z",
"resolved_inferred": false,
"started_at": "2025-10-15T18:58:53.445Z",
"state": "resolved",
"title": "Users are experiencing Authentication failures while using Workspot Windows Client.",
"updated_at": "2025-10-31T06:45:45.198Z",
"url": "https://stspg.io/gnc0wm5vb40f"
},
{
"body": "Microsoft has informed us that the outage in the region has been mitigated. Please, find the summary below, as per Microsoft:\n \nSTATUS: Mitigated 9/10/2025 9:55:15 PM UTC\n\nSUMMARY OF IMPACT:\n\nWhat happened?\nBetween 09:12 UTC and 18:50 UTC on 10 September 2025, a platform issue resulted in an impact on multiple Azure services in the East US 2 region, more specifically, two zones (Az02 and Az03). Impacted customers may have experienced error notifications when performing service management operations, such as creating, deleting, updating, scaling, starting, or stopping, for resources hosted in this region. The primary impacted service was Virtual Machines or Virtual Machine Scale Sets, but this would have resulted in issues for services dependent on such Compute resources, such as Azure Databricks, Azure Kubernetes Service, Azure Synapse Analytics, Backup, and Data Factory.\nCustomers who still see failed or unhealthy resources should attempt to update or redeploy the resource.\n \nWhat do we know so far?\nOur investigation identified that the issue impacting resource provisioning in East US 2 was linked to a failure in the platform component responsible for managing resource placement. The system is designed to recover quickly from transient issues, but in this case, the prolonged performance degradation caused recovery mechanisms themselves to become a source of instability.\nThe incident was primarily driven by a combination of platform recovery behavior and sustained performance degradation. While customer-generated load remained within expected limits, internal platform services began retrying failed operations aggressively when performance issues emerged. These retries, intended to support resilience, instead created a surge in internal system activity.\n \nHow did we respond?\n09:12 UTC on 10 September 2025 \u2013 Customer impact began.\n09:13 UTC on 10 September 2025\u2013 Our monitoring systems observed a rise in failure rates, triggering an alert and prompting our team to initiate an investigation.\n12:08 UTC on 10 September 2025 \u2013 We identified unhealthy dependencies in core infrastructure components as initial contributing factors.\n13:34 UTC on 10 September 2025 \u2013 Began mitigation efforts that included - Restarted critical service components to restore functionality, rerouted workloads from affected infrastructure, initiated multiple recovery cycles for the impacted backend service, on recovery, internal workloads processed through backlogs to get to the current healthy state, and executed capacity operations to free up resources.\n18:50 UTC on 10 September 2025 \u2013 After a period of monitoring to validate the health of services, we were confident that the control plane service was restored, and no further impact was observed on downstream services for this issue.\n \nWhat happens next?\nOur team will be completing an internal retrospective to understand the incident in more detail. We will publish a Preliminary Post Incident Review (PIR) within approximately 72 hours to share more details on what happened and how we responded. After our internal retrospective is completed, generally within 14 days, we will publish a Final Post Incident Review with any additional details and learnings.\nTo get notified when that happens, and/or to stay informed about future Azure service issues, make sure that you configure and maintain Azure Service Health alerts \u2013 these can trigger emails, SMS, push notifications, webhooks, and more: https://aka.ms/ash-alerts\nFor more information on Post Incident Reviews, refer to https://aka.ms/AzurePIRs\nThe impact times above represent the full incident duration, so they are not specific to any individual customer. Actual impact to service availability may vary between customers and resources \u2013 for guidance on implementing monitoring to understand granular impact: https://aka.ms/AzPIR/Monitoring\nFinally, for broader guidance on preparing for cloud incidents, refer to https://aka.ms/incidentreadiness\n\nStay informed about your Azure services\n- Visit Azure Service Health to get your personalized view of possible impacted Azure resources, downloadable Issue Summaries, and engineering updates.\n- Set up service health alerts to stay notified of future service issues, planned maintenance, or health advisories.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-17T04:05:31.114Z",
"resolved_inferred": false,
"started_at": "2025-09-10T14:43:01.297Z",
"state": "resolved",
"title": "Failure in Azure East US and East US2 region",
"updated_at": "2025-09-17T04:05:31.134Z",
"url": "https://stspg.io/bjjsydbyqsdb"
}
]
}