Database outage for legacy accounts
Incident Report for Customer.io Status
Resolved
Everything has been resolved. Sorry for the inconvenience.
Posted Oct 23, 2017 - 02:51 UTC
Monitoring
Good news! Our servers are now up. Our system is now working on clearing up the backlog that formed due to the outage. We will continue to monitor to make sure everything is stable.
Posted Oct 22, 2017 - 23:43 UTC
Update
Our data center is currently still working on fixing the problem. They are in the process of acquiring an additional power source so as to bring up the remainder of the equipment. We will provide another update at 0:00 UTC.
Posted Oct 22, 2017 - 23:30 UTC
Update
Our data center has been working on placing their network switches on backup power sources. About half of the switches that were impacted are now operational. We will continue to monitor their ongoing progress and provide an update at 23:30 UTC.
Posted Oct 22, 2017 - 23:03 UTC
Update
The outage is still ongoing. Our data center is in the process of swapping their switches to backup units. Some are online already, but none that host our servers as of yet. Next update at 23:00 UTC.
Posted Oct 22, 2017 - 22:32 UTC
Update
Our data center is still working on fixing the issue and some of the racks of equipment are now operational. We are continuing to monitor the situation and will post another update at 23:00 UTC.
Posted Oct 22, 2017 - 22:32 UTC
Update
We've contacted our data center and they've confirmed a networking failure that affects several racks of equipment, some of which host a number of servers in our data cluster. They're working to fix the problem. We'll post another update at 22:30 UTC.
Posted Oct 22, 2017 - 22:08 UTC
Investigating
We've identified a database outage caused by a networking issue at your data center. This outage affects only legacy accounts (if you are hosted on fly2.customer.io this outage does not affect you). We've contacted the data center to rectify the situation. We'll post another update at 22:00 UTC.
Posted Oct 22, 2017 - 21:28 UTC