FIXED: DNS issue with server ’svensbluff’
UPDATE: 7:35 AM: Great news! :) The DNS issue on svensbluff has been resolved. No accounts had to be moved. Rebooting the server appeared to resolve the problem, which would suggest there was a tmp file corruption that needed to be flushed out of the system via a power cycle.
If you notice any irregularities with your account, please let us know. We were not made aware of the issue until 3:30 AM Central time. We feel terribly that we weren’t aware of it sooner.
Unfortunately our monitoring system did not pick up on it because it monitors by IP address, and asks each service for a response at the IP. The services were all up, even DNS, and responding … but DNS wasn’t processing requests correctly. Monitoring just asks “are you responding?” — it doesn’t ask and analyze custom requests. That’s why our monitoring system didn’t pick up on this problem. :( I am very sorry for the trouble this caused you. We are going to look into our options for monitoring, in order to be able to better detect situations like this.
We always love to hear from you. We’d rather receive a false report than no report at all. :) So if you see anything weird, please let us know by opening a support ticket right away. We want your service to be top notch, A-#1 !!!
Thank you for all the feedback as we worked on this, it was very helpful to us and helped us to iron out what was/was not going on. ##
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Good morning,
Around 3:30 AM Central time, an as-yet unidentified DNS issue caused websites on the server ’svensbluff’ to become inaccessible.
We have been trouble-shooting this issue since that time.
At this point we have been unable to find the exact cause.
We will restore DNS service to down websites as quickly as possible. We have a list of nameservers that we know are not working, and these are the nameservers that we will assure are answering authoritatively like they are supposed to be, as soon as possible.
Thank you for your patience! We will post updates here as soon as they are available.
DCSN Team
September 11th, 2008 at 7:18 am
Hi Karin,
Actually the DNS issue for us reared its ugly head about 10pm CST for us when I could no longer check email. I didnt think anything of it and went to bed
Kind regards, Mike
September 11th, 2008 at 7:26 am
Thanks Mike. :) I am suspecting this is cPanel-related, as I noticed a number of cPanel user-mapping files were also changed about 9:41 PM. (Hmmmm!)
We are going to be rebooting the server to see if that clears it up. We’ve tried *everything* else.
If it doesn’t work, all accounts will be emergency-migrated/restored to a new server. Our priority is to get sites up ASAP by whatever means is necessary. Thanks for your patience!! I know this is frustrating…….
September 11th, 2008 at 7:37 am
Hey K – At least this time I remembered to look here first before sending you a 911 text! I’m getting a little better, right? Thanks for all your hard work for us!
J