{
"vendor": "Digitalocean",
"slug": "digitalocean",
"platform": "statuspage",
"status_url": "https://status.digitalocean.com",
"last_checked": "2026-09-16T12:28:20Z",
"last_state": "degraded",
"history_backfilled": true,
"first_watched": "2026-09-04T07:06:16Z",
"incidents": [
{
"body": "Our Support Portal is currently experiencing issues due to a third-party service disruption impacting our systems. During this time, customers may be unable to access the Support Portal or create new support tickets for their concerns. Existing support requests may also experience delayed responses from our Support team.\n\nAs a workaround if you're unable to access the Support Portal or need to create a new support ticket, please submit your request via Support Contact form: https://www.digitalocean.com/company/contact/support.\n\nWe apologize for the inconvenience and appreciate your patience while we work to resolve the issue. We'll provide another update as soon as we have more information.",
"first_seen": "2026-09-16T12:28:20Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"started_at": "2026-09-16T09:48:19.081Z",
"state": "investigating",
"title": "Support Portal Service Disruption",
"updated_at": "2026-09-16T09:48:19.174Z",
"url": "https://stspg.io/n6hr8jd64qjl"
},
{
"body": "This incident is now resolved. Power redundancy has been fully restored, and we have confirmed that customer impact has been mitigated. We are closing this incident.",
"first_seen": "2026-09-09T12:30:34Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-09-09T12:25:52.141Z",
"resolved_inferred": false,
"started_at": "2026-09-09T10:47:40.428Z",
"state": "resolved",
"title": "Power Degradation impacting GPU Performance in ATL1",
"updated_at": "2026-09-09T12:25:52.160Z",
"url": "https://stspg.io/g4s2358yl9tt"
},
{
"body": "## 1. What happened\n\nBetween 16:56 UTC and 23:09 UTC on August 24, 2026, customers using DigitalOcean\u2019s Cloud Control Panel, Public API, and dependent product management systems experienced elevated request error rates, peaking at 25%, in two distinct windows totaling 3.5 hours of active degradation within a 6-hour span. Running Droplets, network routing, and block storage \\(Volumes\\) remained fully functional and unaffected throughout the event. No customer data loss occurred.\n\nThe event was initiated during a planned infrastructure migration of the database layer supporting identity, authentication, and authorization services. The migration introduced increased execution latency for a critical authorization query. In response to the elevated query latency, multiple upstream services initiated automated hedged retries. This secondary request amplification led to queue buildup in excess of the authentication service pool's processing throughput, resulting in cascading request timeouts across the authorization path.\n\nTo mitigate the initial degradation, engineering teams rerouted authorization traffic onto a single healthy processing path and temporarily disabled hedging for primary callers, successfully restoring service availability. However, upon returning traffic to the standard multipath routing configuration, lingering query latency combined with retry behavior from unhedged callers caused recurrent queue saturation, leading to the second window of degradation.\n\n## 2. Impact\n\n* Active customer accounts \\(<2%\\) experienced one or more API or Control Panel request failures during the event windows.\n* Aggregate failure rates across both impact windows averaged approximately 10% of authenticated requests, with a peak error rate of 25% during the second window.\n* Affected management control planes included the Cloud Control Panel, Public API, Kubernetes cluster authentication, Container Registry, and Serverless / AI Inference endpoints.\n* Data plane components\u2014including active Droplets, network routing, and block storage volumes\u2014maintained 100% availability, as established connections do not depend on the affected authorization control path. There was zero data loss.\n\n## 3. Current state\n\nAll impacted control plane services have been restored to nominal operational capacity. The database infrastructure migration has been completed. Remediation controls have been validated to ensure system stability against similar query latency scenarios.\n\n## 4. Preventing recurrence\n\nTo prevent future occurrences and enhance systemic resilience, DigitalOcean has implemented the following technical enhancements:\n\n1. Automated Request Throttling and Circuit Breaking:\n\nDeployed automated circuit breakers across upstream services to detect downstream dependency degradation and automatically suppress hedged/retry amplification. Completed a broad architecture audit to ensure retry suppression and circuit breaking are uniformly enforced across all control plane services.\n\n1. Queue Management and Load Shedding:\n\nImplemented fast-fail load-shedding mechanisms to prevent request queue saturation during dependency slowness, accompanied by proactive telemetry alerts to identify queue backing prior to customer impact.\n\n1. Graceful Service Degradation:\n\nConfigured critical authorization components to operate in degraded execution modes utilizing asynchronous, eventually-consistent data replicas during primary database latency events.\n\n## 5. Timeline\n\n| **Time UTC** | **Event** |\n| --- | --- |\n| 16:56 | A specific authorization query began running significantly slower than its historical baseline. |\n| 16:58 | Automated monitoring detected abnormal error rates and latency; detected within 5 minutes of impact starting. |\n| 17:05\u201318:52 | Operators tuned retry configuration and consolidated requests to a healthy service instance \\(first unavailability window closed\\). |\n| 21:30 | Shifting traffic back to the production pattern caused the same saturation problem \\(second unavailability window begins\\). |\n| 21:30\u201323:09 | Teams disabled the retry/amplification behavior across the remaining affected services and added additional read capacity for the slow query. Service was restored to a stable state. |",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-25T19:04:33.091Z",
"resolved_inferred": false,
"started_at": "2026-08-24T17:14:43.701Z",
"state": "postmortem",
"title": "Cloud Control Panel and API",
"updated_at": "2026-09-15T19:11:53.801Z",
"url": "https://stspg.io/8l5c24cxh3hr"
},
{
"body": "Our Engineering team has resolved the issue preventing cluster creation on Managed Databases in the NYC3 and NYC1 regions. Customers can now create new Managed Databases Clusters normally.\n\nOur Engineering team has monitored the environment to ensure stability. We sincerely apologize for any inconvenience this disruption may have caused to your operations. If you continue to experience issues, please open a ticket with our Support team: https://cloudsupport.digitalocean.com/s/",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-23T14:33:31.719Z",
"resolved_inferred": false,
"started_at": "2026-08-23T05:11:37.000Z",
"state": "resolved",
"title": "Managed Databases Creation",
"updated_at": "2026-08-23T14:33:31.735Z",
"url": "https://stspg.io/8ld5fpw01tg4"
},
{
"body": "Our Engineering team has confirmed that the issue impacting Container Registry and Spaces accessibility has been resolved. \n\nSpaces accessibility via the Cloud Control Panel is now restored, and pushing and pulling container images is succeeding as expected. Customers who previously encountered these issues should now be able to access Spaces and deploy their container images without further issues.\n\nIf you continue to experience any problems, please open a ticket with our support team. Thank you for your patience, and we apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-21T14:52:40.373Z",
"resolved_inferred": false,
"started_at": "2026-08-21T12:59:42.692Z",
"state": "resolved",
"title": "Container Registry and Spaces Accessibility",
"updated_at": "2026-08-21T14:52:40.389Z",
"url": "https://stspg.io/2b4phnbmtm49"
},
{
"body": "Our Engineering team has confirmed the regional control plane is healthy and CPU Droplets, Managed databases, Load balancers, Block storage, Kubernetes (DOKS) and Spaces are operating normally.\n\nGPU Droplets continue to be impacted and our teams are working to restore all nodes. We will communicate with GPU Droplet customers separately via Slack and email with more information and regular updates.\n\nThank you for your patience throughout this incident. If you continue to experience any issues, please open a support ticket from within your account.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-19T19:34:54.136Z",
"resolved_inferred": false,
"started_at": "2026-08-19T11:10:06.329Z",
"state": "resolved",
"title": "Major Service Interruption in MKC1",
"updated_at": "2026-08-19T19:34:54.152Z",
"url": "https://stspg.io/gdzqj0v18x0r"
},
{
"body": "Between Aug 13, 09:50 UTC  and 14:30 UTC, customers in the FRA1 region experienced an issue affecting Block Storage Volumes attachment and detachment operations on Droplets and Managed Kubernetes clusters.\n\nDuring this window, customers may have encountered delays or been unable to attach and detach Block Storage Volumes from their resources.\n\nBlock Storage Volumes attachment and detachment operations now appear to be functioning normally, and customers should no longer experience issues with these operations. For Managed Kubernetes clusters specifically, some residual cleanup of affected volume attachments may still be needed, and our team is actively working through this.\n\nWe apologize for any inconvenience this may have caused. If you continue to experience issues with Block Storage Volumes operations, please don't hesitate to open a support ticket for further investigation. Our team will be happy to assist you.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-13T14:59:44.895Z",
"resolved_inferred": false,
"started_at": "2026-08-13T14:59:44.847Z",
"state": "resolved",
"title": "Volume Attach/Detach events in FRA1",
"updated_at": "2026-08-13T14:59:44.904Z",
"url": "https://stspg.io/yldnj7859gcp"
},
{
"body": "Our Engineering team confirms the full resolution of the issue affecting all impacted services, including new account registration, Droplet Creation, Reserved IPs, Automated Backups and Snapshots, Droplet Autoscale, DOKS clusters, GenAI services, the Droplet Console, and Database cluster creation.\n\nWe apologize for the inconvenience. If you continue to experience any issues, please open a support ticket from within your account.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-09T01:01:02.727Z",
"resolved_inferred": false,
"started_at": "2026-08-08T18:57:49.228Z",
"state": "resolved",
"title": "Account Registration, Droplets, and Related Services",
"updated_at": "2026-08-09T01:01:02.747Z",
"url": "https://stspg.io/grfh9nf5bp3f"
},
{
"body": "Our Engineering team has confirmed that the issue affecting Functions on App Platform has been fully resolved. Functions builds, deploys, and requests to Functions endpoints are all working as expected. The fix implemented earlier was successful, and we are no longer seeing any problems affecting Functions.\n\nHowever, if you continue to experience any issues, please don't hesitate to raise a support ticket for further investigation.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-06T21:05:05.672Z",
"resolved_inferred": false,
"started_at": "2026-08-06T19:01:27.576Z",
"state": "resolved",
"title": "Functions on App Platform",
"updated_at": "2026-08-06T21:05:05.690Z",
"url": "https://stspg.io/7pm9l1jwpkr6"
},
{
"body": "Between Aug 6, 06:42 UTC and 09:08 UTC, customers experienced failures with write operations across multiple DigitalOcean services, including Managed Databases, DOKS, Cloud Firewalls, DNS, Spaces, Block Storage Volumes, and event processing.\n\nCustomers attempting to update Cloud Firewall rules, modify DNS records, manage object storage operations, provision or scale DOKS clusters and node pools, and make other infrastructure configuration changes experienced failures. These operations returned errors, preventing customers from managing their resources. Additionally, event processing was delayed or failed during this window, which may have impacted the timeliness of resource status updates and notifications.\n\nThe services have recovered and are now operating normally. Customers should no longer experience issues with write operations, configuration changes, or event processing.\n\nWe apologize for any inconvenience this may have caused. If you continue to experience issues with any of these services, please don't hesitate to open a support ticket. Our team will be happy to assist you.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-06T16:33:14.860Z",
"resolved_inferred": false,
"started_at": "2026-08-06T16:33:14.787Z",
"state": "resolved",
"title": "Multiple Services Availability",
"updated_at": "2026-08-06T16:33:14.872Z",
"url": "https://stspg.io/zscj6w048lfx"
},
{
"body": "As of 21:06 UTC, our Engineering team has resolved the issue affecting Spaces Cold Storage billing. An error in our billing processing system caused incorrect values to be reflected in daily usage and current invoices for customers with cold storage buckets.\n\nOur team identified and corrected an issue with the billing data sync, ensuring accurate usage reporting. If you continue to experience any issues, please open a ticket with our Support team. We apologize for any inconvenience this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-04T21:42:43.197Z",
"resolved_inferred": false,
"started_at": "2026-08-04T16:13:35.593Z",
"state": "resolved",
"title": "Spaces Cold Storage Billing",
"updated_at": "2026-08-04T21:42:43.212Z",
"url": "https://stspg.io/qb6zv0kdjy50"
},
{
"body": "Between 09:36 and 13:30 UTC on August 3, some customers experienced intermittent errors while accessing the DigitalOcean Cloud Control Panel. We renewed the affected internal certificates and restored access. The issue is resolved, and the Cloud Control Panel is operating normally. We apologize for the disruption.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-03T17:59:42.943Z",
"resolved_inferred": false,
"started_at": "2026-08-03T13:58:04.872Z",
"state": "resolved",
"title": "Cloud Control Panel Access",
"updated_at": "2026-08-03T17:59:42.957Z",
"url": "https://stspg.io/045lx5xqfdns"
},
{
"body": "Our Engineering team has resolved the issue causing empty or broken responses from Kimi-K3. If you continue to experience any problems, please open a ticket with our Support team. We apologize for any inconvenience this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-30T06:15:55.395Z",
"resolved_inferred": false,
"started_at": "2026-07-29T17:56:51.011Z",
"state": "resolved",
"title": "Response Degradation Impacting Kimi-K3",
"updated_at": "2026-07-30T06:15:55.419Z",
"url": "https://stspg.io/ynhlvpy6ctcm"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-28T15:24:28.800Z",
"resolved_inferred": false,
"started_at": "2026-07-27T18:31:40.584Z",
"state": "resolved",
"title": "Agent Platform Requests Returning HTTP 500 Errors",
"updated_at": "2026-07-28T15:24:28.816Z",
"url": "https://stspg.io/54mh0lzk9733"
},
{
"body": "Between 07:46\u00a0 UTC & 10:24 UTC today, some users in India accessing resources in the NYC region may have experienced connectivity issues. The impact was limited to users using Airtel as their ISP.\n\nOur investigation determined that the issue was caused by an internal network issue within Airtel and was not related to DigitalOcean network. The connectivity issue now appears to be resolved, and users should no longer experience problems accessing their resources.\n\nWe apologize for any inconvenience this may have caused. However, if you continue to experience any connectivity issues, please don't hesitate to open a support ticket for further investigation; we'll be happy to assist you further.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-24T12:42:25.955Z",
"resolved_inferred": false,
"started_at": "2026-07-24T10:36:52.252Z",
"state": "resolved",
"title": "Network Connectivity from India to NYC",
"updated_at": "2026-07-24T12:42:25.974Z",
"url": "https://stspg.io/y9g7qgc9qs6m"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-23T12:37:00.025Z",
"resolved_inferred": false,
"started_at": "2026-07-23T11:16:59.123Z",
"state": "resolved",
"title": "Llama-4-Maverick Model Availability",
"updated_at": "2026-07-23T12:37:00.044Z",
"url": "https://stspg.io/9pxmkrlffl58"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-22T19:42:17.085Z",
"resolved_inferred": false,
"started_at": "2026-07-22T16:50:42.474Z",
"state": "resolved",
"title": "Gradient AI Serverless Inference Requests Timing Out for qwen3.5-397b-a17b",
"updated_at": "2026-07-22T19:42:17.100Z",
"url": "https://stspg.io/2fdnm4tns3f1"
},
{
"body": "Our Engineering team has confirmed that the issue regarding volume attachment to Droplets in the NYC1, NYC3, SGP1, SYD1, and BLR1 regions has been fully resolved, and all services are now operating normally.\n\nIf you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-19T15:32:39.578Z",
"resolved_inferred": false,
"started_at": "2026-07-19T09:51:19.000Z",
"state": "resolved",
"title": "Block Storage Volume NYC1, NYC3, SGP1, SYD1 and BLR1",
"updated_at": "2026-07-19T15:32:39.594Z",
"url": "https://stspg.io/wv68wmnz55ln"
},
{
"body": "Our engineering team has resolved the outbound network connectivity issue in the BLR1 region. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-19T09:52:00.733Z",
"resolved_inferred": false,
"started_at": "2026-07-19T06:57:41.458Z",
"state": "resolved",
"title": "Networking in BLR1",
"updated_at": "2026-07-19T09:52:00.749Z",
"url": "https://stspg.io/8yc90q76srsl"
},
{
"body": "Our Engineering team has confirmed that the underlying issue affecting Reserved IP routing in the TOR1 region has been fully resolved, and services are now operating normally.\n\nIf you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-16T17:24:35.996Z",
"resolved_inferred": false,
"started_at": "2026-07-16T13:39:47.134Z",
"state": "resolved",
"title": "Reserved IP routing in TOR1",
"updated_at": "2026-07-16T17:24:36.013Z",
"url": "https://stspg.io/x0s0pghvdcgv"
},
{
"body": "The deployed fix has successfully restored full functionality, and our monitoring shows that system performance has completely stabilized. Response times for all Gemma 4 inference workflows have returned to normal baseline levels.\nWe will continue to track platform stability moving forward to ensure long-term reliability. We apologize for any disruption this may have caused to your workflows and appreciate your patience throughout the recovery process.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-15T21:46:35.616Z",
"resolved_inferred": false,
"started_at": "2026-07-13T19:04:10.240Z",
"state": "resolved",
"title": "Serverless Inference - Gemma4 Latency Issues Causing Timeouts & Slow Responses",
"updated_at": "2026-07-15T21:46:35.631Z",
"url": "https://stspg.io/rzs04wclgpfq"
},
{
"body": "The issue affecting Kubernetes deployments in the NYC1 region has been resolved. Our investigation found intermittent DNS timeouts affecting a small number of DOKS clusters, with affected worker nodes running on shared-CPU Droplets. This is a documented limitation for latency-sensitive cluster DNS workloads such as CoreDNS.\n\nThe affected clusters are currently functional. To reduce the risk of recurrence, we recommend running CoreDNS on non-shared/dedicated CPU node pools and using sufficient CoreDNS replicas. \n\nIf you continue to see DNS failures or NodeNotReady events, please open a support ticket so we can investigate that cluster specifically.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-11T01:27:56.795Z",
"resolved_inferred": false,
"started_at": "2026-07-09T20:31:52.156Z",
"state": "resolved",
"title": "Kubernetes Deployments in NYC1",
"updated_at": "2026-07-11T01:27:56.812Z",
"url": "https://stspg.io/p8xctv4k5yjv"
},
{
"body": "Our Engineering team has confirmed that the issue with the Deepseek V4 Pro model in Serverless Inference and Agent Platform has been fully resolved. The model is now operational, and users should be able to use it without experiencing any errors.\n\nIf you continue to experience any problems, please open a ticket with our Support team. We apologize for any inconvenience this may have caused and appreciate your patience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-05T00:53:19.389Z",
"resolved_inferred": false,
"started_at": "2026-07-04T22:13:45.293Z",
"state": "resolved",
"title": "Limited access to Deepseek V4 Pro model",
"updated_at": "2026-07-05T00:53:19.404Z",
"url": "https://stspg.io/wwsc9zft1lxd"
},
{
"body": "Our Engineering team has confirmed that the issue impacting Droplets in the NYC3 region has been fully resolved. Users can now perform actions on their Droplets or create new Droplets in this region without any issues.\n\nWe appreciate your patience while we worked to resolve this issue. If you continue to experience any problems, please open a support ticket from your account so our team can investigate further.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-04T15:10:06.071Z",
"resolved_inferred": false,
"started_at": "2026-07-04T14:42:47.881Z",
"state": "resolved",
"title": "Droplet Actions in NYC3",
"updated_at": "2026-07-04T15:10:06.085Z",
"url": "https://stspg.io/1ggj0xnb3n4v"
},
{
"body": "Our engineering team has resolved the issue with Managed Database Clusters. All database systems should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-03T08:20:23.462Z",
"resolved_inferred": false,
"started_at": "2026-07-02T22:04:53.265Z",
"state": "resolved",
"title": "Managed Database Clusters",
"updated_at": "2026-07-03T08:20:23.479Z",
"url": "https://stspg.io/rpkck6zr7ht5"
},
{
"body": "The intermittent DNS lookup failures between Managed Kubernetes and Managed Database hostnames in the FRA1 region have been resolved for now but please let us know if you see the issue again. Connectivity has remained stable, and all systems are operating normally. If you continue to experience any problems, please open a ticket with our support team. Thank you for your patience, and we apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-01T16:28:25.933Z",
"resolved_inferred": false,
"started_at": "2026-07-01T11:35:28.934Z",
"state": "resolved",
"title": "DNS Lookup Failures for Managed Database",
"updated_at": "2026-07-01T16:28:25.950Z",
"url": "https://stspg.io/r6yw4xc720r8"
},
{
"body": "Our Engineering team has confirmed that the issue impacting Droplet resizes using the \"Downscale Anytime\" option has been fully resolved. Users can now resize their Droplets using the \"Downscale Anytime\" option without any issues.\n\nWe appreciate your patience while we worked to resolve this issue. If you continue to experience any problems, please open a support ticket from your account so our team can investigate further.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-01T13:28:10.841Z",
"resolved_inferred": false,
"started_at": "2026-07-01T11:13:21.135Z",
"state": "resolved",
"title": "Droplet Resize",
"updated_at": "2026-07-01T13:28:10.857Z",
"url": "https://stspg.io/m6y2gmlz3jrv"
},
{
"body": "This incident has been resolved, and our teams continue to monitor the results. If you experience any further issues, please contact support.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-02T03:38:53.338Z",
"resolved_inferred": false,
"started_at": "2026-07-01T06:25:23.501Z",
"state": "resolved",
"title": "Agent Timeouts While Retrieving Data from Knowledge Bases",
"updated_at": "2026-07-02T03:38:53.354Z",
"url": "https://stspg.io/sp8b09xxqvjt"
},
{
"body": "Between 00:00 UTC & 16:00 UTC today, our Engineering team identified an issue affecting backup operations on Droplets.  During this period, backups scheduled within this window may not have been created and may appear as missing.\n\nOur team has taken necessary measures to resolve the issue, and we can confirm that the backup service has been restored and is now functioning normally. Upcoming scheduled backups should be performed as expected.\n\nWe sincerely apologize for any inconvenience this may have caused and appreciate your understanding. However, if you have any further questions or concerns, please create a support ticket for further analysis.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-27T18:24:25.102Z",
"resolved_inferred": false,
"started_at": "2026-06-27T18:24:25.041Z",
"state": "resolved",
"title": "Droplet Backup Service",
"updated_at": "2026-06-27T18:24:25.113Z",
"url": "https://stspg.io/djw8kv2yfvcd"
},
{
"body": "Our Engineering team has resolved the issue that was causing HTTP 400 errors for requests to Anthropic models. Users should now be able to access Anthropic models without any issues.\nIf you continue to experience problems, please open a ticket with our Support team so we can investigate further.\nWe apologize for any inconvenience this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-27T05:15:38.421Z",
"resolved_inferred": false,
"started_at": "2026-06-27T04:43:29.854Z",
"state": "resolved",
"title": "Anthropic Inference Model Availability",
"updated_at": "2026-06-27T05:15:38.440Z",
"url": "https://stspg.io/s2bx05f4wxkb"
},
{
"body": "Between 13:22 & 14:26 UTC today, users have experienced missing monitoring graphs for Droplets (with DO agent installed), Load Balancers, Databases, and other services within the Cloud Control Panel.\n\nOur engineering team has identified the root cause of the issue and has taken appropriate steps to restore functionality. We can confirm that services have been restored and are functioning as expected.\n\nWe apologize for any inconvenience this may have caused. If you continue to experience issues viewing monitoring graphs, please create a support ticket for further analysis. Thank you for your patience and understanding",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-19T16:37:38.303Z",
"resolved_inferred": false,
"started_at": "2026-06-19T14:00:32.960Z",
"state": "resolved",
"title": "Monitoring Graphs in the Cloud Control Panel",
"updated_at": "2026-06-19T16:37:38.319Z",
"url": "https://stspg.io/8jy6pdvsszwm"
},
{
"body": "The connectivity issues affecting our Serverless/GenAI Inference API have been fully resolved.\nOur engineering teams successfully completed the connectivity restoration. All systems should be functioning normally, and the endpoint should be fully operational.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-17T00:29:42.280Z",
"resolved_inferred": false,
"started_at": "2026-06-16T22:00:52.000Z",
"state": "resolved",
"title": "Intermittent 500 Errors on Serverless/GenAI Inference API",
"updated_at": "2026-06-17T00:29:46.450Z",
"url": "https://stspg.io/40ln0gl1x56j"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-11T17:42:05.051Z",
"resolved_inferred": false,
"started_at": "2026-06-11T14:24:20.851Z",
"state": "resolved",
"title": "DeepSeek V4 Pro is returning HTTP 429 \"Rate limit exceeded\"",
"updated_at": "2026-06-11T17:42:05.073Z",
"url": "https://stspg.io/r8mzkb94pf9g"
},
{
"body": "Between 09:41 and 11:24 UTC, our Engineering team identified an issue impacting the DNS API service. During this period, users may have experienced issues performing domain and DNS record management operations through the Control Panel and API. Services dependent on DNS API operations, including Let's Encrypt certificate provisioning, MongoDB cluster creation, App Platform deployments, and DigitalOcean Kubernetes (DOKS) cluster create and delete operations, were also impacted.\n\nOur Engineering team has confirmed that the underlying issue affecting the DNS API service has been fully resolved, and all affected services are now operating normally.\n\nIf you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-04T11:47:05.272Z",
"resolved_inferred": false,
"started_at": "2026-06-04T10:12:10.120Z",
"state": "resolved",
"title": "DNS API Service",
"updated_at": "2026-06-04T11:47:05.291Z",
"url": "https://stspg.io/c9tgc3txntpg"
},
{
"body": "Our Engineering team has confirmed that the issue impacting build failures on App Platform has been resolved at 07:00 am UTC. All App Platform builds are now succeeding as expected. Customers who previously encountered build failures should now be able to deploy their applications without further issues.\n\nIf you continue to experience any problems, please open a ticket with our support team. Thank you for your patience, and we apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-29T07:58:23.373Z",
"resolved_inferred": false,
"started_at": "2026-05-29T00:14:19.240Z",
"state": "resolved",
"title": "App Platform Deployments",
"updated_at": "2026-05-29T07:58:23.388Z",
"url": "https://stspg.io/c5cpdwx9rc37"
},
{
"body": "Between 06:35 UTC & 12:36 UTC today, our engineering team identified an issue impacting the Cloud Firewall. During this period, users might have encountered issues while updating their firewall rules.\n\nOur team has taken appropriate measures to address the issue. We can confirm that service has been restored and is now functioning normally.\n\nWe regret the inconvenience caused and appreciate your patience and understanding. However, if you continue to experience any issues, please create a support ticket for further analysis.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-20T13:03:09.605Z",
"resolved_inferred": false,
"started_at": "2026-05-20T12:08:27.021Z",
"state": "resolved",
"title": "Cloud Firewall",
"updated_at": "2026-05-20T13:03:09.622Z",
"url": "https://stspg.io/5dzt41b39qvv"
},
{
"body": "Our Engineering team has verified that the issue causing incorrect Cloud Firewall rules to display on the cloud panel is fully resolved. The user interface continues to function as expected, and all systems have returned to normal operating conditions.\n\nIf you continue to experience problems, please open a ticket with our support team. Thank you for your patience throughout this incident.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-20T07:48:14.980Z",
"resolved_inferred": false,
"started_at": "2026-05-20T05:35:48.457Z",
"state": "resolved",
"title": "Cloud Firewall",
"updated_at": "2026-05-20T07:48:14.998Z",
"url": "https://stspg.io/6y9xmwmgkyy4"
},
{
"body": "From 00:00 UTC to 01:40 UTC, users may have experienced degraded write performance and intermittent impacts to workloads dependent on Block Storage Volumes in the NYC3 region.\n\nOur Engineering team has confirmed that the underlying infrastructure issue affecting Block Storage Volumes at the storage layer has been fully resolved, and services are now operating normally.\n\nIf you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-20T05:13:50.000Z",
"resolved_inferred": false,
"started_at": "2026-05-20T01:05:16.154Z",
"state": "resolved",
"title": "Block Storage Volume Performance",
"updated_at": "2026-05-20T05:41:39.874Z",
"url": "https://stspg.io/1cr62wpw64kx"
},
{
"body": "As of the current time, our Engineering team has confirmed that the issue with Serverless Inference has been resolved. The root cause of the issue was an outage with our provider, Anthropic, which affected users of Sonnet 4.6 and Opus 4.7 models. According to Anthropic's status page (https://status.claude.com/incidents/8z7l5zcy0v3b), they have resolved the outage. If you continue to experience problems, please open a ticket with our Support team. We apologize for any inconvenience this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-15T02:00:39.064Z",
"resolved_inferred": false,
"started_at": "2026-05-15T01:21:51.685Z",
"state": "resolved",
"title": "Anthropic reported outage that's impacting access to their Serverless Inference models",
"updated_at": "2026-05-15T02:00:39.085Z",
"url": "https://stspg.io/d47rft5t1t68"
},
{
"body": "As of 19:50 UTC, the issue affecting the DNS service has been fully resolved, and all related services, including Let's Encrypt certificate issuance and Managed MongoDB provisioning, are now operating normally.\n\nThe backlog of delayed requests has been successfully processed, and all DNS record updates, pending certificate, and MongoDB operations should now be complete. \n\nWe apologize for the disruption this caused and appreciate your patience while our team worked to restore full functionality.\n\nHowever, if you continue to experience any issues, please don't hesitate to raise a support ticket for further investigation. We'll be happy to assist you.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-14T20:46:40.334Z",
"resolved_inferred": false,
"started_at": "2026-05-14T13:57:38.574Z",
"state": "resolved",
"title": "DNS service, Certificates and Managed MongoDB",
"updated_at": "2026-05-14T20:46:40.349Z",
"url": "https://stspg.io/r6dtr7gc9pch"
},
{
"body": "The remediation for the issue affecting the GradientAI Agent Platform Playground has been fully implemented, and the service is operating normally. We are no longer observing customer impact related to this issue, and the incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-12T11:37:12.000Z",
"resolved_inferred": false,
"started_at": "2026-05-12T10:24:07.025Z",
"state": "resolved",
"title": "GradientAI: Agent Platform Playground Interaction Errors",
"updated_at": "2026-05-12T12:22:23.582Z",
"url": "https://stspg.io/ll1m0fw2m4g9"
},
{
"body": "Between 5:35 and 14:25 UTC today, our Engineering team identified an issue that prevents enabling Two-Factor Authentication (2FA) and Google/GitHub authentication through the Control Panel. During this period, users might have encountered issues while enabling authentication methods and accessing teams with secure sign-in enabled.\n\nOur team has taken appropriate measures to address the issue. We can confirm that service has been restored and is now functioning normally.\n\nWe regret the inconvenience caused. However, if you continue to experience any issues, please create a support ticket for further analysis.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-09T14:42:15.691Z",
"resolved_inferred": false,
"started_at": "2026-05-09T11:15:11.260Z",
"state": "resolved",
"title": "Control Panel Errors - Unable to Enable 2FA and Google/GitHub",
"updated_at": "2026-05-09T14:42:15.705Z",
"url": "https://stspg.io/tyb995vv9ykq"
},
{
"body": "The upstream outage with Let's Encrypt has been resolved. Customers should now be able to issue Let's Encrypt certificates for Spaces, Load Balancers, and App Platform Custom Domains. Our Engineering team has also confirmed that stuck or delayed actions with Mongo, Advanced PG, and Advanced MySQL databases should complete normally now. \n\nWe appreciate your patience. If you continue to experience any issues, please open a support ticket from within your account.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-08T21:54:44.593Z",
"resolved_inferred": false,
"started_at": "2026-05-08T20:46:22.108Z",
"state": "resolved",
"title": "Let's Encrypt Outage Affecting Certificate Issuance and Managed Databases Operations",
"updated_at": "2026-05-08T21:54:44.610Z",
"url": "https://stspg.io/rw9m9n69q6tg"
},
{
"body": "Our engineering team has resolved the issue with multiple services in NYC2 region, and all services should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience and thank you for your patience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-08T19:59:48.942Z",
"resolved_inferred": false,
"started_at": "2026-05-08T18:01:42.481Z",
"state": "resolved",
"title": "Multiple Services in NYC2",
"updated_at": "2026-05-08T19:59:48.955Z",
"url": "https://stspg.io/tb7f2ntnj8hm"
},
{
"body": "All services are operating normally. We will continue to monitor the system to ensure ongoing reliability.\n\nThank you for your patience while we worked to resolve this issue.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-28T19:33:38.769Z",
"resolved_inferred": false,
"started_at": "2026-04-28T13:45:20.299Z",
"state": "resolved",
"title": "Elevated 5xx \u201ccontext canceled\u201d errors impacting serverless inference",
"updated_at": "2026-04-28T19:33:38.782Z",
"url": "https://stspg.io/qf13crcjzkmn"
},
{
"body": "The issue is resolved, and service is operating normally.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-27T11:37:36.467Z",
"resolved_inferred": false,
"started_at": "2026-04-27T10:38:30.000Z",
"state": "resolved",
"title": "Serverless Inference - Intermittent Rate Limiting Affecting Some Customers Using Anthropic Models",
"updated_at": "2026-04-27T11:37:36.483Z",
"url": "https://stspg.io/y8cgtdqrkct1"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-23T23:51:58.178Z",
"resolved_inferred": false,
"started_at": "2026-04-23T22:26:34.000Z",
"state": "resolved",
"title": "Intermittent errors impacting some Serverless Inference models in ATL1",
"updated_at": "2026-04-23T23:51:58.201Z",
"url": "https://stspg.io/z4pxt0jh8n97"
},
{
"body": "Our Engineering team has confirmed that the issue impacting App Platform deployments and Kubernetes (DOKS) nodes has been fully resolved at 09:22 UTC. Users may already notice improvements while deploying apps and DOKS nodes.\n\nAll App Platform deployments are now succeeding as expected. Customers who previously encountered build failures should now be able to deploy their applications without further issues.\n\nIf you continue to experience any problems, please open a ticket with our support team. Thank you for your patience, and we apologize for any inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-23T10:09:54.770Z",
"resolved_inferred": false,
"started_at": "2026-04-23T08:08:37.488Z",
"state": "resolved",
"title": "App Platform Deployments",
"updated_at": "2026-04-23T10:09:54.800Z",
"url": "https://stspg.io/0rby3knkk6s1"
},
{
"body": "Our Engineering team has confirmed full resolution of the issue with Cloud UI for Managed Kubernetes at 09:19 UTC.\n\nAll the services should be functioning as expected.\n\nIf you continue to experience problems, please open a ticket with our support team. Thank you for your patience throughout this incident!",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-23T08:46:39.799Z",
"resolved_inferred": false,
"started_at": "2026-04-23T07:01:46.702Z",
"state": "resolved",
"title": "Cloud UI for Managed Kubernetes",
"updated_at": "2026-04-23T08:46:39.818Z",
"url": "https://stspg.io/3tff32qrsv77"
},
{
"body": "Our Engineering team has resolved the issue that was impacting the DigitalOcean API and the Cloud Control Panel. The system is now operating normally.\n\nDuring the incident, users may have experienced difficulties accessing droplets, viewing the Droplets page, or performing CRUD (Create, Read, Update, Delete) operations via the API. We have verified that the issue has been fully resolved.\n\nWe apologize for the inconvenience this may have caused. If you continue to experience any issues, please submit a support ticket through the Cloud Control Panel so our team can assist you further.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-22T11:47:38.128Z",
"resolved_inferred": false,
"started_at": "2026-04-22T11:06:49.867Z",
"state": "resolved",
"title": "Cloud Control Panel and API",
"updated_at": "2026-04-22T11:47:38.156Z",
"url": "https://stspg.io/2wd37dxsqcrk"
},
{
"body": "Our Engineering team has confirmed that the issue affecting the DNS resolution of .co top-level domain (TLD) has been resolved. The DNS resolution for .co domains is now working as expected.\n\nWe apologize for the inconvenience. If you continue to face any issues, please open a support ticket from within your account.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-15T12:28:18Z",
"resolved_at": "2026-04-18T14:10:35.084Z",
"resolved_inferred": false,
"started_at": "2026-04-17T21:22:52.396Z",
"state": "resolved",
"title": "DNS Resolution for .co TLD",
"updated_at": "2026-04-18T14:10:35.098Z",
"url": "https://stspg.io/2s124dt3v3gl"
},
{
"body": "From 17:22 to 17:46 UTC, our Engineering team observed an issue impacting Spaces availability in the NYC3 region. During this time, customers may have encountered 500 errors and degraded performance while accessing Spaces buckets. The issue has now been fully resolved. If you continue to experience problems, please open a ticket with our support team from within your Cloud Control Panel. We apologize for any inconvenience caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-08T12:29:58Z",
"resolved_at": "2026-04-17T18:27:09.864Z",
"resolved_inferred": false,
"started_at": "2026-04-17T18:27:09.813Z",
"state": "resolved",
"title": "Spaces availability in NYC3",
"updated_at": "2026-04-17T18:27:09.873Z",
"url": "https://stspg.io/nftmlzbn2kd4"
}
]
}