{
"vendor": "Upstash",
"slug": "upstash",
"platform": "statuspage",
"status_url": "https://status.upstash.com",
"last_checked": "2026-09-16T12:28:20Z",
"last_state": "ok",
"history_backfilled": true,
"first_watched": "2026-09-04T07:06:16Z",
"incidents": [
{
"body": "Upstash Vector experienced a brief DNS resolution issue in the US East (us-east-1) region lasting approximately 20 minutes. During this period, new DNS resolution attempts may have failed.\n\nExisting connections and clients with cached DNS records were not affected.\n\nThe issue has been resolved, and DNS resolution is operating normally.",
"first_seen": "2026-09-10T12:15:40Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-09-09T12:00:00.000Z",
"resolved_inferred": false,
"started_at": "2026-09-09T12:00:00.000Z",
"state": "resolved",
"title": "Intermittent DNS Resolution Errors for Upstash Vector in US East (us-east-1)",
"updated_at": "2026-09-09T13:01:03.143Z",
"url": "https://stspg.io/yfkkt5363477"
},
{
"body": "We experienced a brief issue affecting access to the Upstash Console. Our team quickly identified the cause and resolved the issue within minutes.\n\nThe console is now operating normally.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-09-03T08:00:00.000Z",
"resolved_inferred": false,
"started_at": "2026-09-03T08:00:00.000Z",
"state": "resolved",
"title": "Upstash Console login issue",
"updated_at": "2026-09-03T08:29:36.492Z",
"url": "https://stspg.io/dmdmfrm8lycz"
},
{
"body": "Between 22:20 and 22:33 UTC, QStash clients in us-east-1 experienced disruptions related to message persistence. The team applied a fix, and service has been restored.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-08-28T22:20:00.000Z",
"resolved_inferred": false,
"started_at": "2026-08-28T22:20:00.000Z",
"state": "resolved",
"title": "Message Persistence Issue \u2014 QStash us-east-1",
"updated_at": "2026-08-28T22:57:24.342Z",
"url": "https://stspg.io/69bd84bqrpcq"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-22T10:05:17.914Z",
"resolved_inferred": false,
"started_at": "2026-07-22T08:50:12.940Z",
"state": "resolved",
"title": "Fly.io infrastructure disruption affecting some Upstash Redis databases on Fly.io DFW Region",
"updated_at": "2026-07-22T10:05:17.932Z",
"url": "https://stspg.io/0l5sf44ng7bn"
},
{
"body": "Partial outage due to http parsing errors. Problem was resolved couple minutes later. If problem continues, refreshing the DNS cache is recommended.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-07-16T06:30:00.000Z",
"resolved_inferred": false,
"started_at": "2026-07-16T06:30:00.000Z",
"state": "resolved",
"title": "QStash us-east-1 - URL Publish Errors",
"updated_at": "2026-07-16T08:35:07.559Z",
"url": "https://stspg.io/9qvjxdfprs57"
},
{
"body": "This incident has been resolved, we will publish RCA soon.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-25T16:36:00.855Z",
"resolved_inferred": false,
"started_at": "2026-06-25T15:23:44.611Z",
"state": "resolved",
"title": "Upstash Redis Partial Service Disruption",
"updated_at": "2026-06-25T16:36:00.871Z",
"url": "https://stspg.io/9gynqrnvcpyj"
},
{
"body": "### **Summary**\n\nOn June 23, 2026, QStash users in the EU-CENTRAL-1 region experienced elevated latency and degraded performance. The issue was caused by a regression introduced in a recent enhancement to Flow Control scheduling logic. The change increased resource consumption under load, leading to reduced performance on affected shards. The incident was resolved by rolling back to the previous stable version.\n\n### **Impact**\n\n* Service: QStash \\(EU-CENTRAL-1\\)\n* Start: 2026-06-23 16:22 UTC\n* Resolved: 2026-06-23 17:14 UTC\n* Duration: ~52 minutes\n* Impact: Increased latency and degraded performance for workloads routed to the affected instance in the EU. Multiple customers in the region experienced slower request processing during the incident window.\n\n### **Root Cause**\n\nWe recently deployed an enhancement to our Flow Control feature, designed to improve fairness between independent flow controls when unused global parallelism capacity was available.\n\nUnder production load, the new behavior introduced a regression that significantly increased resource utilization. The elevated resource consumption caused performance degradation on the affected QStash instamce, impacting all workloads sharing that instance.\n\n### **Resolution**\n\nAfter identifying the regression as the source of the slowdown, we rolled back the deployment to the last known stable version.\n\n* Rollback completed: 2026-06-23 19:18:46.84 UTC\n\nFollowing the rollback, system performance returned to normal levels and service stability was restored.\n\n### **Preventive Actions**\n\nTo reduce the likelihood of similar incidents in the future, we are taking the following actions:\n\n* Expand performance and load testing coverage for Flow Control changes.\n* Add resource utilization regression checks to the deployment pipeline.\n* Improve monitoring and alerting for abnormal CPU and memory consumption patterns.\n* Introduce additional canary validation before wider production rollout of scheduler-related changes.\n\nWe apologize for the disruption and appreciate our customers\u2019 patience while we resolved the issue.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-23T17:14:46.066Z",
"resolved_inferred": false,
"started_at": "2026-06-23T16:22:25.859Z",
"state": "postmortem",
"title": "QStash EU Region \u2014 Degraded Performance",
"updated_at": "2026-06-24T12:30:46.041Z",
"url": "https://stspg.io/r5k8bdfqm35c"
},
{
"body": "Between 01:18 and 04:03 UTC, customers were unable to access running boxes or perform create, read, update, or delete operations. Running containers were not affected and continued operating throughout. The issue has been fully resolved and all box operations are functioning normally.\n\nWe are taking all necessary measures to prevent this from happening again. We apologize for the disruption.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-23T06:05:57.252Z",
"resolved_inferred": false,
"started_at": "2026-06-23T06:05:57.207Z",
"state": "resolved",
"title": "Box Access and Management Operations Unavailable",
"updated_at": "2026-06-23T06:05:57.261Z",
"url": "https://stspg.io/y6kf1kvffvjj"
},
{
"body": "## What happened\n\nA configuration change applied to QStash's EU networking layer introduced an outbound connectivity fault. Between **09:11 and 09:45 UTC**, a subset of QStash instances had degraded egress connectivity, and some dispatched messages may have failed on their initial attempt. A related side effect caused some servers to egress from IP addresses outside QStash's advertised outbound range until **11:24 UTC**; during that window, endpoints enforcing QStash IP allowlists may have rejected requests from those servers.\n\nQStash's automatic retries delivered many affected messages on subsequent attempts once connectivity was restored, and QStash is now operating normally.\n\n## Customer impact\n\nImpact was limited to the EU region and to two effects within the window above:\n\n* **Delivery delays / failures** _\\(09:11 \u2013 09:45 UTC\\)_ \u2014 messages dispatched through affected instances may have failed on their first attempt. Thanks to automatic retries with backoff, most were re-delivered once connectivity recovered; messages that exhausted their retry policy during the window followed their configured failure path \\(e.g., DLQ / failure callback\\).\n* **Allowlist rejections** _\\(09:11 \u2013 11:24 UTC\\)_ \u2014 customers who restrict inbound traffic to QStash's outbound IP ranges may have seen requests from the affected servers rejected. Customers who do not enforce source-IP allowlisting were unaffected by this.\n\n## Root cause\n\nA routine networking configuration change contained an error that:\n\n1. disrupted outbound connectivity on the affected nodes, and\n2. caused affected nodes to acquire outbound IPs outside the advertised range.\n\nThe underlying gap was systemic: the change procedure had no automated validation gate to catch the faulty state before it reached production.\n\n## What we've done\n\n* **Fail-fast pre-checks** \u2014 automated validation now halts the change procedure _before it runs_ if a configuration would degrade connectivity or violate expected state.\n* **Egress IP-range enforcement** \u2014 outbound IP assignments are now checked against the advertised QStash range, so a node can no longer come online with an unadvertised IP.\n* **Tighter post-change verification** \u2014 completion now confirms outbound reachability and egress-IP conformance across affected nodes.\n\n## Customer action\n\nNone required. All QStash traffic again originates from the advertised outbound IP ranges.\n\nWe apologize for the disruption.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-19T09:11:00.000Z",
"resolved_inferred": false,
"started_at": "2026-06-19T09:11:00.000Z",
"state": "postmortem",
"title": "QStash Service Disruption in the EU Region",
"updated_at": "2026-06-19T13:59:02.732Z",
"url": "https://stspg.io/5m39sbnstm49"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-06-04T14:09:13.505Z",
"resolved_inferred": false,
"started_at": "2026-06-04T10:10:36.337Z",
"state": "resolved",
"title": "Fly.io - Upstash Vector service disruption on IAD region",
"updated_at": "2026-06-04T14:09:13.519Z",
"url": "https://stspg.io/w6vy8lf5kbyx"
},
{
"body": "The issue affecting some Upstash Vector indexes in the US-EAST-1 region has been resolved.\n\nOur team investigated the incident and identified the conditions that were contributing to elevated memory pressure on the affected servers. We mitigated those conditions by reducing memory utilization on the impacted nodes, rebalancing affected workloads where needed, and increasing available headroom capacity across the region.\n\nAll affected indexes should now be operating normally, and based on the mitigations applied, we do not expect this issue to recur. We apologize for any inconvenience this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-30T07:51:00.748Z",
"resolved_inferred": false,
"started_at": "2026-05-29T20:18:02.657Z",
"state": "resolved",
"title": "Intermittent slowness in Vector US-EAST-1 region",
"updated_at": "2026-05-30T07:51:00.763Z",
"url": "https://stspg.io/d5zkphtnpb9j"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-23T00:00:03.229Z",
"resolved_inferred": false,
"started_at": "2026-05-22T23:27:48.418Z",
"state": "resolved",
"title": "New Database Creation Failing Due to Upstream Provider Issue",
"updated_at": "2026-05-23T00:00:03.245Z",
"url": "https://stspg.io/fcgvdbq1n5d9"
},
{
"body": "Earlier today, unexpected load on our proxies caused intermittent connection issues for Upstash Redis in the following regions: \nus-east-1, us-west-1, ap-southeast-2, and ap-south-1. \n\nDuring this period, some clients may have seen connection timeouts or elevated error rates when reaching their databases.\nOur team identified the issue quickly and applied workarounds to relieve pressure on the affected proxies. Connection health has since been restored and we've been monitoring the regions to confirm everything is stable. All systems are now operating normally.\n\nWe appreciate your patience and apologize for any disruption this may have caused.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-14T17:31:50.861Z",
"resolved_inferred": false,
"started_at": "2026-05-14T15:47:23.000Z",
"state": "resolved",
"title": "Upstash Redis \u2013 intermittent connection issues in some regions",
"updated_at": "2026-05-14T17:31:50.883Z",
"url": "https://stspg.io/qyl44kv5lw09"
},
{
"body": "On May 12th and 13th at various times, a subset of Upstash Redis instances on [Fly.io](http://Fly.io) experienced intermittent hangs and elevated error rates. The Redis process would stall inside a logging syscall \u2014 alive but not making progress \u2014 which made the issue hard to spot from our usual telemetry. After investigating with Fly's team, we identified the root cause as a bad interaction between a recent guest kernel update on Fly's newer machines and an upstream Cloud Hypervisor bug \\([cloud-hypervisor#7672](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7672)\\) affecting log writes from inside the VM. We mitigated by disabling the affected logging paths, and Fly has since rolled out a hypervisor-side patch, fully resolving the issue. No data was lost. Sorry for the disruption.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-12T13:48:53.804Z",
"resolved_inferred": false,
"started_at": "2026-05-12T09:49:32.897Z",
"state": "postmortem",
"title": "Fly.io Upstash Redis Service Distruption",
"updated_at": "2026-05-15T12:48:59.853Z",
"url": "https://stspg.io/v8n17s5pgq0y"
},
{
"body": "On May 12th and 13th at various times, a subset of Upstash Redis instances on [Fly.io](http://Fly.io) experienced intermittent hangs and elevated error rates. The Redis process would stall inside a logging syscall \u2014 alive but not making progress \u2014 which made the issue hard to spot from our usual telemetry. After investigating with Fly's team, we identified the root cause as a bad interaction between a recent guest kernel update on Fly's newer machines and an upstream Cloud Hypervisor bug \\([cloud-hypervisor#7672](https://github.com/cloud-hypervisor/cloud-hypervisor/issues/7672)\\) affecting log writes from inside the VM. We mitigated by disabling the affected logging paths, and Fly has since rolled out a hypervisor-side patch, fully resolving the issue. No data was lost. Sorry for the disruption.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-11T18:22:29.993Z",
"resolved_inferred": false,
"started_at": "2026-05-11T15:05:43.087Z",
"state": "postmortem",
"title": "Fly.io Upstash Redis service distruption (FRA region)",
"updated_at": "2026-05-15T12:49:12.978Z",
"url": "https://stspg.io/xc9fkbk20d2w"
},
{
"body": "**Root Cause Analysis**\n\n  \nOn **April 24**, we deployed a more optimized scheduler implementation in the **US East \\(N. Virginia\\)** region.  \nOn **May 8**, a user who had active schedules deleted their account.  \nUnder normal behavior, scheduled tasks associated with a deleted account should wake up, detect that the account no longer exists, and exit after performing cleanup. Due to a bug introduced in the new scheduler implementation, this code path did not return early as intended. Execution continued and resulted in a nil pointer dereference.  \nA second issue then amplified the impact. When a panic occurs in the scheduler, it is designed to be recovered, logged, and isolated so that the process remains healthy. Because of another bug in the panic recovery path, the panic was not properly caught, which caused the worker process handling the scheduled job to terminate.  \nAfter that process exited, another worker picked up responsibility for delivering the same scheduled task. Since the same faulty execution path was still present, that worker also failed. This created a cascading failure pattern across workers attempting to process the affected schedules.  \n**Resolution**  \nWe deployed two fixes:\n\n* Added the missing early return in the deleted-account cleanup path, preventing the nil pointer dereference.\n* Corrected the panic recovery logic so that future panics are safely recovered, logged, and reported without causing worker processes to terminate.\n\nWith these changes in place, the affected execution path is now safe. Even if a future bug triggers a panic in this area, it will be isolated and reported rather than causing process-level failure.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-08T10:08:41.433Z",
"resolved_inferred": false,
"started_at": "2026-05-08T09:46:14.134Z",
"state": "postmortem",
"title": "QStash US Region Service Disruption",
"updated_at": "2026-05-12T08:25:40.262Z",
"url": "https://stspg.io/zh00zgv2ks17"
},
{
"body": "# **Incident Postmortem: Scheduled Jobs Inconsistency in US Region**\n\nOn May 1, 2026, we experienced an incident affecting a subset of schedules in the US region following a recent infrastructure update.\n\nThe issue has been resolved, and all affected schedules have been restored.\n\n## **Summary**\n\nAs part of an ongoing scalability improvement, we recently updated scheduling infrastructure in the US region to a new architecture. During this transition, a legacy execution path remained in the codebase as a fallback mechanism.\n\nOn May 1, a bug caused the system to revert to the legacy path. This resulted in inconsistent state between the old and new scheduling systems for some users.\n\n## **Impact**\n\nThe incident affected a limited number of users in the US region.\n\n**Most users were not affected**, and the vast majority of schedules continued operating normally throughout the incident.\n\nUsers who did not update schedules during the transition window continued operating normally throughout the incident.\n\n**A subset of users who created, edited, paused, or deleted schedules between April 24 and May 1 may have experienced one or more of the following:**\n\n* Schedule updates not being reflected\n* Paused schedules becoming active again\n* Deleted schedules reappearing\n* Newly created schedules not executing as expected\n\nSchedules created after the transition may have stopped executing briefly before recovery.\n\n## **Root Cause**\n\nDuring the transition, the new scheduling infrastructure became the source of truth for schedule state.\n\nDue to a bug, the system unexpectedly reverted traffic to the legacy scheduling path, which began accepting updates independently from the new system.\n\nThis caused the two systems to diverge and resulted in inconsistent schedule state for affected users.\n\n## **Resolution**\n\nAfter identifying the issue, we:\n\n1. Restored the new scheduling system as the active source of truth\n2. Reconciled data between the legacy and new systems\n3. Updated missing schedule changes back into the new infrastructure\n4. Performed conflict resolution to preserve user data and schedule continuity\n\nIn some cases, schedules that had previously been paused or deleted were restored to avoid permanent data loss.\n\n## **Preventive Measures**\n\nWe are implementing several changes to prevent similar incidents:\n\n* Removing obsolete fallback execution paths after transitions complete\n* Adding automated safeguards and alerts for unexpected system fallback behavior\n* Improving consistency validation between systems\n* Expanding rollback and reconciliation testing\n\nWe apologize for the disruption and appreciate everyone\u2019s patience while we resolved the issue.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-06T08:05:27.074Z",
"resolved_inferred": false,
"started_at": "2026-05-05T14:24:43.025Z",
"state": "postmortem",
"title": "QStash US Region: Schedule Degradation",
"updated_at": "2026-05-06T08:56:42.638Z",
"url": "https://stspg.io/qbgh8qr9p3cj"
},
{
"body": "Replication complete, incident resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-02T19:53:23.947Z",
"resolved_inferred": false,
"started_at": "2026-04-02T18:45:17.073Z",
"state": "resolved",
"title": "Fly.io Upstash Redis \u2013 iad Region Elevated Latency and Temporary Read-Only State",
"updated_at": "2026-04-02T19:53:23.965Z",
"url": "https://stspg.io/jhtjz9c6723c"
},
{
"body": "Due to a race condition in a process that attaches static IPs to nodes, some of the IPs in the dns were detached from the nodes, causing timeouts.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-03-27T15:30:00.000Z",
"resolved_inferred": false,
"started_at": "2026-03-27T15:30:00.000Z",
"state": "resolved",
"title": "Upstash Redis: GCP Global Connectivity Problems",
"updated_at": "2026-03-27T17:05:02.724Z",
"url": "https://stspg.io/wjqvlzqpdsps"
},
{
"body": "On March 6, between approximately 13:44\u201314:07 UTC, some databases experienced elevated latency and connection errors in the Tokyo \\(ap-northeast-1\\) region.\n\nThe issue was caused by a sudden spike in traffic that significantly increased network utilization and connection load on a subset of nodes.\n\nOur team mitigated the incident by scaling up capacity in the region and redistributing load across additional nodes. Service recovered once the additional capacity was brought online.\n\nResolution\n\nWe have increased the number of machines in the Tokyo region to provide additional headroom and reduce the likelihood of similar incidents during traffic spikes.\n\nNext Steps\n\nWe are continuing to review capacity safeguards and connection-handling limits to improve resilience against sudden traffic surges.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-03-06T14:49:57.180Z",
"resolved_inferred": false,
"started_at": "2026-03-06T13:50:17.434Z",
"state": "postmortem",
"title": "Region ap-northeast-1 outage on Upstash Global",
"updated_at": "2026-03-09T10:09:03.241Z",
"url": "https://stspg.io/fstdc8k6z87l"
},
{
"body": "We identified the cause of elevated latency impacting some databases in us-east-1 region between 15:30\u201315:35 UTC as a sudden surge of connection attempts that hit OS-level connection limits on our proxy layer. This resulted in slower new connection establishment and increased latency for some requests. Databases were not impacted. We are implementing additional proxy-level metrics and safeguards to detect and manage similar edge cases earlier.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-01-23T15:30:00.000Z",
"resolved_inferred": false,
"started_at": "2026-01-23T15:30:00.000Z",
"state": "resolved",
"title": "Upstash Redis: Intermittent latency on us-east-1",
"updated_at": "2026-01-23T17:05:42.691Z",
"url": "https://stspg.io/cfyy1vkxqjj6"
},
{
"body": "Issue has been identified and replicas were successfully reconnected.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-12-10T12:08:43.543Z",
"resolved_inferred": false,
"started_at": "2025-12-10T11:50:47.166Z",
"state": "resolved",
"title": "Connectivity issue impacting Regional Databases in US-East-1",
"updated_at": "2025-12-10T12:08:43.561Z",
"url": "https://stspg.io/53w7vln17ypd"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-12-05T09:18:53.546Z",
"resolved_inferred": false,
"started_at": "2025-12-05T09:03:56.241Z",
"state": "resolved",
"title": "Upstash Console and Context7 Console is Currently Experiencing issues",
"updated_at": "2025-12-05T09:18:53.564Z",
"url": "https://stspg.io/tf2f3cc3vnj4"
},
{
"body": "We identified and fixed a bug that could cause messages with Flow Control enabled to be delayed longer than their configured delay, resulting in unexpectedly long pending times.\n\nThe fix is in place and the issue should not recur. If you\u2019re still seeing unusually long-delayed messages, please contact support@upstash.com and we can help with remediation.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-12-01T07:00:00.000Z",
"resolved_inferred": false,
"started_at": "2025-12-01T07:00:00.000Z",
"state": "resolved",
"title": "QStash - Message delays for Flow Control configurations",
"updated_at": "2025-12-12T14:34:52.432Z",
"url": "https://stspg.io/5j0xwwz0gn3d"
},
{
"body": "A fix has been deployed as a workaround so that our systems are not affected from the ongoing incident of the cloud provider",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-20T18:50:38.268Z",
"resolved_inferred": false,
"started_at": "2025-10-20T14:52:07.077Z",
"state": "resolved",
"title": "Upstash Console issues",
"updated_at": "2025-10-20T18:50:38.287Z",
"url": "https://stspg.io/8twfwt7qbdyn"
},
{
"body": "As a side effect of an incident on the underlying cloud provider, Upstash Console has had availability issues between 07:00UTC and  09:23UTC.\n\nOnly Upstash Console is impacted, Upstash products remained operational.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-20T07:00:00.000Z",
"resolved_inferred": false,
"started_at": "2025-10-20T07:00:00.000Z",
"state": "resolved",
"title": "Login Issues on Upstash Console",
"updated_at": "2025-10-20T10:29:33.078Z",
"url": "https://stspg.io/blt426dr5x5b"
},
{
"body": "Between 15:46\u201315:55 UTC, some client connection attempts to databases in us-east-1 timed out due to unexpected high load on a server. The node was recovered at 15:50 UTC, and the updated DNS record propagated by 15:55 UTC. Services are operating normally.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-10T15:46:00.000Z",
"resolved_inferred": false,
"started_at": "2025-10-10T15:46:00.000Z",
"state": "resolved",
"title": "Connectivity issue on us-east-1",
"updated_at": "2025-10-10T16:49:10.168Z",
"url": "https://stspg.io/gn5kd8bwmxdp"
},
{
"body": "Impact:\nA subset of clients connecting through the eu-central-1 region experienced increased error rates and timeouts when accessing certain databases. Clients in us-west-2 were also briefly affected. The issue was limited in scope and did not impact other regions.\n\nRoot Cause:\nDuring an ongoing migration to improve database routing reliability, a configuration step was applied inconsistently across regions. \n\nResolution:\nOur monitoring alerted us within minutes, and the migration was promptly rolled back for the affected regions. Service definitions were restored, and normal database connectivity resumed by 15:08 UTC.\n\nNext Steps:\nWe are reviewing our migration process to ensure consistency across all regions and adding additional safeguards to prevent similar issues in the future.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-09T12:00:00.000Z",
"resolved_inferred": false,
"started_at": "2025-09-09T12:00:00.000Z",
"state": "resolved",
"title": "Temporary Database Routing Issue",
"updated_at": "2025-09-09T15:48:41.983Z",
"url": "https://stspg.io/rt0hggtwj47z"
},
{
"body": "Between 12:37 UTC and 12:50 UTC, an overload in the connection proxying system resulted in instability for a subset of databases.  \nThe root cause was identified as a misconfiguration in the routing rules, which caused certain requests to experience timeouts during the initial phase of a gradual deployment. Upon detection, the deployment was immediately rolled back, restoring normal service.\n\nWe are reviewing our deployment and configuration validation processes to prevent similar issues in the future.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-08-18T12:37:02.000Z",
"resolved_inferred": false,
"started_at": "2025-08-18T12:37:02.000Z",
"state": "postmortem",
"title": "Connectivity Issues in us-east-1",
"updated_at": "2025-08-18T13:30:54.868Z",
"url": "https://stspg.io/s4f8dn5zlxwb"
}
]
}