PDA

View Full Version : Emergency Server IP Change and Hard Outage


Support
02-18-2006, 05:45 AM
Two days ago, I was informed that the stable network our server resided on was sold and moving to a new location. I had the choice to either allow the server to be moved to the new location and keep the same IP address or change our IP address to the HE.net network and keep the server at it's current location in Fremont, CA.

If I were to allow the server to be moved, it would have been moved to San Jose where there is no on-site support. I decided to change networks(ip addresses) and remain in Fremont, CA where there is on-site support.

The IP changeover process was planned to be seemless and to occur without any downtime. However, after completing the changeover today, I learned that the IP addresses assigned to our server were already in use by a different machine. This resulted in our server being unreachable until our provider issues new and "unused" IP addresses.

Upon receiving "good" IP addresses, our provider went to the datacenter and manually set our main ip so our server would be reachable via SSH interface. Our server was reachable and I could start reconfiguring with the new IP addresses but we ran into another problem with the cpanel license since our main ip address changed, cpanel was unable to verify our license. Our provider quickly issued a new license to correct that problem.

I was able to reach the server via WHM and reconfigure it with the new set of IP addresses. Everything was working fine but the sites were not displaying correctly. This was due to our nameservers pointing to an invalid machine which we had no access to. It takes about 24 hours for nameservers and ip addresses to resolve correctly worldwide. Since I did not have 24 hours to properly prepare the switch to the "good" ip addresses, we experienced some downtime with sites unreachable.

I have been constantly monitoring the system and as of about 30 minutes ago, confirmed that our sites are once again reachable.

I appologize for the downtimes. There should be no further issues regarding this IP address changeover. If any customers are experiencing any issues, please let me know immediately.