The server has been going online and offline momentarily since the previous maintenance. Below you will find the data center's response to me and it is just about what I was expecting and theorizing. I apologize for the service interruptions and this will hopefully be over soon.
I have been working with our network team as well as the upstream to get you the best explanation one what occurred and the way forward. On Thursday, during a planned network maintenance window, we performed a core network capacity and performance upgrade. This was executed as planned and resulted in significantly less impact than planned for a majority of our partners.
As for the major concern, following the upgrade, we began to see what appeared to be ‘random’ flapping of our BGP sessions upstream. After investigation, we noticed that an upstream misconfiguration was causing an overload of the session engine. We then began coordinating a fix with the upstream to address the flood of data that was the root cause and a change was implemented. This change was implemented incorrectly which resulted in a loop that cause the service impact.
We have identified the root cause and a way forward. As for what that means, and what you care about, all services are curently stable, however we are planning and will send notification for, a future maintenance window that will prevent the need for this upstream reliance. This will be of minimal impact and prevent the need for any future such maintenance windows. As always, we treat our partners as we would like to be treated and take these issues very seriously. The highest levels of the organization are involved and are extremely sensitive to anything service impacting. Please do not hesitate to reach out with your concerns or questions and we’d be glad to address them.
Powered by WHMCompleteSolution
