{
"vendor": "Fasterize",
"slug": "fasterize",
"platform": "statuspage",
"status_url": "https://status.fasterize.com",
"last_checked": "2026-09-16T12:28:20Z",
"last_state": "ok",
"history_backfilled": true,
"first_watched": "2026-09-04T07:06:16Z",
"incidents": [
{
"body": "Incident is closed since 11:30am but we keep on monitoring as the incident on our hosting provider is not fully resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "major",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-22T17:18:20.609+02:00",
"resolved_inferred": false,
"started_at": "2026-05-22T11:33:50.733+02:00",
"state": "resolved",
"title": "Performance degradation",
"updated_at": "2026-05-22T17:18:20.624+02:00",
"url": "https://stspg.io/pw4b7j0mjt8y"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-21T07:38:52.368+02:00",
"resolved_inferred": false,
"started_at": "2026-05-20T12:33:55.003+02:00",
"state": "resolved",
"title": "Website status may incorrectly show as \u201cPing blocked\u201d",
"updated_at": "2026-05-21T07:38:52.385+02:00",
"url": "https://stspg.io/k4vxcg467wvj"
},
{
"body": "The issue affecting flush operations through the API and dashboard has been fixed.\n\nFlush actions are now working normally again.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-05-20T10:08:25.050+02:00",
"resolved_inferred": false,
"started_at": "2026-05-20T10:00:37.917+02:00",
"state": "resolved",
"title": "Issue on the flush on the dashboard and API",
"updated_at": "2026-05-20T10:08:30.635+02:00",
"url": "https://stspg.io/c59lbj7cwl6d"
},
{
"body": "We experienced some issues with our European infrastructure. Service has been restored since 00:20. This has affected loading speeds. Some pages or websites have experienced slowdowns.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-04-10T22:30:00.000+02:00",
"resolved_inferred": false,
"started_at": "2026-04-10T22:30:00.000+02:00",
"state": "resolved",
"title": "Performance degradation",
"updated_at": "2026-04-11T00:49:51.460+02:00",
"url": "https://stspg.io/nhb0t1gvwn5t"
},
{
"body": "We have identified an issue affecting the accuracy of INP data collected between Thursday, 5 February and Monday, 9 February.\n\nDuring this period, INP metrics were incorrectly reported for Safari on iPhone when the navigation type was back-forward cache (BFCache). This behaviour is related to a known issue in WebKit, which causes inflated INP values in this specific scenario.\nMore details are available in the related Safari bug report:\nhttps://bugs.webkit.org/show_bug.cgi?id=305251\n\nThe issue has been identified and mitigated on our side. Only the INP metric is impacted; other performance metrics remain unaffected.\n\nWe apologise for any confusion this may have caused and recommend caution when analysing INP data for Safari iPhone traffic over this time window.\n\nIf you have any questions, our team remains available.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2026-02-05T13:00:00.000+01:00",
"resolved_inferred": false,
"started_at": "2026-02-05T13:00:00.000+01:00",
"state": "resolved",
"title": "INP Data Inaccuracy on Safari (iPhone)",
"updated_at": "2026-02-09T18:46:25.461+01:00",
"url": "https://stspg.io/d84bvfj0q5lj"
},
{
"body": "The incident has been resolved, everything is now operating normally",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-11-24T16:43:03.416+01:00",
"resolved_inferred": false,
"started_at": "2025-11-24T15:50:11.072+01:00",
"state": "resolved",
"title": "Fasterize Console is slow",
"updated_at": "2025-11-24T16:43:03.439+01:00",
"url": "https://stspg.io/8fcv4k703z7j"
},
{
"body": "An in-memory database cluster failure occurred leading to service unavailability across multiple Fasterize components \u2014 primarily the **Optimisation Engine** and the **API**.\n\nAt **09:48**, an in-memory database cluster failure happens after restarting multiple nodes to release a new engine version. The database cluster began an automatic failover sequence, but each time a new node was promoted as primary, it **crashed under excessive connection load**.\n\nThis cluster serves as a **cache layer** providing access to configurations. During the outage, Engine instances attempted to reconnect at a very high frequency and fell back to retrieving data directly from the main database.This fallback mechanism worked as intended until **10:32**, allowing our optimization engine to continue operating in **degraded mode**.\n\nAt **10:32**, however, the **proxy layer** of our optimisation engine became **saturated in network resources**, rendering it unavailable from the front layer.\n\nWhen the proxy layer is unreachable, the platform automatically **unplugs the CDN and sites are served directly from their origin servers**, without Fasterize optimizations.\n\nWorking with our hosting provider, we **reduced the Optimisation Engine cluster size at 11:45** to limit reconnection attempts to Redis. By **12:16**, the Redis cluster had stabilised and full service was restored.\n\nLater, at **14:15**, we detected that the **API was unable to write to the Redis cluster**. The root cause was a **security patch applied by the hosting provider**, which **restricted the use of some commands** in the cluster.The API was patched to remove usage of these commands, and full functionality was restored by **20:15**.\n\n\u200c\n\n## **Impact**\n\n* **Duration:** 10h32 \u2013 12:16 \\(outage\\), API issue until 20:15  \n* **Affected components:** Optimisation Engine, API  \n* **User impact:** Most websites temporarily served unoptimised content directly from origin; Some websites experienced unavailability due to an dysfunctional DNS fallback mechanismAPI write operations failed, blocking configuration update.\n\n## **Resolution Timeline**\n\n\u200c\n\n## Action plan\n\nShort term:\n\n* Improve **alerting and visibility** on in-memory database cluster health and failover events.\n* Review the in-memory database connection logic of the engine to avoid too many reconnection attempts in case of disconnections and avoid a case preventing the start of the process in case of an in-memory database outage.\u00a0\n* Adjust failover DNS logic to avoid redirection to the origin when the CDN/fronts are still able to accept the traffic.\n* Upscale in-memory database cluster in order to accept more connections  \n\nMedium term:\u00a0\n\n* Review and test **disaster recovery procedures** for in-memory database cache clusters, including the ability to quickly activate a passive cluster.\n\nLong term:\n\n* Re-architecture the engine to reduce the number of connections on the itn-memory database cluster",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-10-09T20:23:39.092+02:00",
"resolved_inferred": false,
"started_at": "2025-10-09T11:01:04.847+02:00",
"state": "postmortem",
"title": "Platform unavailability",
"updated_at": "2025-10-13T14:19:53.534+02:00",
"url": "https://stspg.io/rwnp4gsddztk"
},
{
"body": "A network maintenance caused a regression in the service responsible for extracting the True-Client-IP header from the X-Forwarded-For header. Around 6% of the platform trafic has been impacted. The issue has now been resolved. We apologize for the inconvenience.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "none",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-25T10:00:00.000+02:00",
"resolved_inferred": false,
"started_at": "2025-09-25T10:00:00.000+02:00",
"state": "resolved",
"title": "Some origin requests got a wrong Client IP header between 09:52 \u2013 10:34 (Paris time)",
"updated_at": "2025-09-25T11:07:19.282+02:00",
"url": "https://stspg.io/ykp8qw9r2kvm"
},
{
"body": "This incident has been resolved.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "critical",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-09T15:51:42.777+02:00",
"resolved_inferred": false,
"started_at": "2025-09-09T15:11:52.763+02:00",
"state": "resolved",
"title": "Platform has been unavailable",
"updated_at": "2025-09-09T15:51:42.791+02:00",
"url": "https://stspg.io/rm3clrb23s0j"
},
{
"body": "# **\ud83d\udee0\ufe0f Incident Description**\n\n**On Tuesday, September 9th, 2025, Fasterize carried out a scheduled maintenance to modernize part of its private network infrastructure.**\n\n**During this operation, an unexpected configuration issue temporarily disrupted communication between some components of the platform. This resulted in an increased error rate on part of the traffic between 11:43 and 11:50 \\(Paris time\\).**\n\n# **\ud83d\udcc5 Timeline**\n\n* **11:40 \u2013 Update started on the first batch of machines**  \n* **11:43 \u2013 Anomaly detected in communication between proxies and fronts**  \n* **11:50 \u2013 Issue mitigated, service returned to normal**  \n\n# **\u2705 Immediate Actions**\n\n* **Affected machines were taken out of rotation**  \n* **Healthy machines were reactivated to absorb the traffic**  \n* **Service stabilized within minutes of the anomaly**  \n\n# **\ud83d\ude80 Next Steps**\n\n* **Strengthening our validation procedures during network migrations**  \n* **Adding automated connectivity tests to detect similar issues earlier**  \n* **Improving documentation and standardization of network configurations to minimize future risks**",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-09T12:13:56.714+02:00",
"resolved_inferred": false,
"started_at": "2025-09-09T12:13:56.657+02:00",
"state": "postmortem",
"title": "Slowdowns on the platform",
"updated_at": "2025-09-10T15:25:59.016+02:00",
"url": "https://stspg.io/g6v10g7j96n8"
},
{
"body": "## **\ud83d\udee0\ufe0f Incident Report**\n\n**Date:** Tuesday, September 9, 2025\n\n**Duration:** 14:50 \u2013 18:30 \\(Paris time\\)\n\n**Impact:** ~23% of traffic\n\n### **\ud83d\udccc Summary**\n\nOn September 9, Fasterize performed a planned network maintenance to replace an obsolete private network. As part of this operation, a new private IP address was introduced for our load balancers while keeping the existing ones in place.\n\nLater that afternoon, a DDoS attack led to the saturation and restart of one of the load balancers. Following this restart, the load balancer started using the new private IP address to communicate with the backend servers. Since this IP had not yet been registered as a trusted proxy, some requests were processed incorrectly.\n\n### **\ud83d\udcc8 Impact**\n\n* ~23% of traffic was affected between **14:50 and 18:30**.  \n* Some users encountered **403 errors**.  \n* In certain cases, client IP addresses were not correctly identified in the headers.  \n\n### **\ud83d\udcc5 Timeline**\n\n* **10:25** \u2013 New private IP address added to the load balancers.  \n* **14:49** \u2013 DDoS attack causes one load balancer to restart.  \n* **14:50** \u2013 First incorrect headers observed \\(undetected at this stage\\).  \n* **16:02** \u2013 Support receives a ticket mentioning 403 errors.  \n* **17:16** \u2013 Ticket escalated to the platform team.  \n* **18:16** \u2013 Root cause identified and linked to the network update.  \n* **18:30** \u2013 New IP address added to the trusted list; incident resolved.  \n\n**Total duration of impact:** 3h40\n\n### **\ud83d\udd0d Root Cause**\n\nThe new private IP address introduced during the maintenance was not yet included in the trusted proxy configuration. When one load balancer restarted, it began using this IP, leading to incorrect handling of client headers.\n\n### **\ud83d\ude80 Next Steps**\n\n* All load balancer IPs have now been added to the trusted configuration across environments.  \n* Documentation has been updated to include this step in future network changes.  \n* Additional monitoring will be set up to detect unexpected private IPs in client headers.  \n* Escalation procedures between support and platform teams will be reviewed to ensure faster response times.",
"first_seen": "2026-09-04T07:06:16Z",
"impact": "minor",
"last_seen": "2026-09-16T12:28:20Z",
"resolved_at": "2025-09-09T02:30:00.000+02:00",
"resolved_inferred": false,
"started_at": "2025-09-09T02:30:00.000+02:00",
"state": "postmortem",
"title": "Client IP retrieval has been broken between 14:50 \u2013 18:30 (Paris time)",
"updated_at": "2025-09-10T15:28:48.391+02:00",
"url": "https://stspg.io/f8h3815tsw8s"
}
]
}