Meltdown
Sometime around 10 PM Friday the MySQL server at my hosting provider took a walk. The hosting sysop blamed it on my site and disabled the database that serves it by making the directory the MySQL files are in unreadable. MySQL didn’t seem to handle that condition well, and since MaisonBisson was still piling up queries looking for the content in the DB, things continued to go downhill.
My involvement started around 11 PM Friday night (yes, I’m that dorky). Not knowing what was going on, I started a support ticket with the host. The sysop replied fairly quickly with some MySQL logs that showed nothing, but then didn’t answer any further questions on the ticket until about 11 AM Saturday. The host’s answer was inspecific, but I replied quickly and again I got no replies.
Then, around 2 PM Saturday, the host sent me a message from their abuse department telling me they’d disabled my account (it had actually been disabled for about 15 hours or more by then) because the site was causing problems with the DB server. Apparently, because the scripts on the site were looking for the missing database content and MySQL doesn’t handle things well when the DB files are made inaccessible on the filesystem and because I was still getting traffic (though the traffic wasn’t getting content), things weren’t going well.
Again, however, my replies to support got no answer, and it was about 10 AM Sunday before I got another response from the hosting provider. But it still took until 1 PM before I could get the sysop to restore my access to the site content (under the condition that no public access be allowed on the site until they approved it).
Around 4 PM, after about 42 hours of downtime, MaisonBisson went live with a new provider.
Things I’ll admit:
- I haven’t paid serious attention to optimizing my code for heavy traffic (but I don’t really know what “heavy traffic” is)
- My site was getting a lot of page loads Friday (though it wasn’t record number), and weekend nights can be especially busy (look at the list of popular incoming searches in the sidebar during the weekend and you’ll understand)
- I don’t know how many queries per second I was calling at the time of the DB crash (or how many were running overall), or any other detail statistic of the CPU load I was generating
Lessons:
- HostGator sucks.
- Keep backups up to date (I actually had a good DB backup from early Friday, but I didn’t have filesystem backup)
- Perhaps I oughtta look at some ways to optimize all this.
- HostGator sucks.
Tips For Hosting Providers:
- Don’t treat customers like they want to crash your servers.
- Remember that customers can’t see the system performance stats you can. Involve them early — as problems are developing, but before things crash — so they can start working on solutions before anybody loses uptime.
As late as sometime mid-morning Saturday, HostGator could have kept me as a customer — and I was still considering upgrading my hosting account! — but customer service that was both slow and indifferent soured me.
11 Comments
Comments RSS TrackBack Identifier URI
Leave a comment
[...] As I was contemplating making angry calls to my hosting provider last week when they shut down MaisonBisson for a couple days, it occurred to me that I would rather make those calls via SkypeOut or some similar service that didn’t reveal my home phone number. After all, I wouldn’t want an angry sysop to take revenge by having a spare modem call me up every 27 minutes between the hours of midnight and seven AM. [...]
[...] I switched to Lunarpages last week after the fiasco with my old hosting provider. Now, because of bandwidth and CPU usage, I’m moving to a new server at Lunarpages. I wasn’t surprised about what they said when I got a message from the sysadmins about excessive CPU usage on my shared hosting account, but I was surprised with their proactive and customer friendly approach. [...]
[...] Zach pointed out that I’ve been sort of sloppy about using double-quotes where I should use single quotes and not single-quoting my array indices. All of this can add up to an unnecessary performance hit for high traffic sites. [...]
I had similar problems with HostGator, on a very, very low volume/demand site, and got a couple of really nasty emails from them when I tried to find out why they went in and hacked my file system. It pissed me off to the point that I registered hostgator-sucks.com.
After leaving HostGator, suddenly all of my domains are under attack via. ICANN complains originating from HostGator.
Evidently there are a bunch of losers associated with the day-to-day operation of HostGator, with too much time of their hands.
There will soon be a forum or blog at http://www.hostgator-sucks.com for others to share their experiences.
Brian
[tags]hostgator, hostgator sucks, hostgator support, hostgator ICANN complaints[/tags]
hi
I just had the last straw with hostgator.com Their support is to laugh at. The service was down approx 1 hour today. and has been down at least 2-3x in the last 2 months for 20 minute intervals during peak traffic. Im gonna switch ASAP. Please dont sign up with hostgator.com they plain suck
PS after numerious emails with support, i went directly to Brent the founders of hostgator for help. He told me politely to take a hike!
hostgator sucks bigtime. we also just left them after our account was shut down becuase we wanted an explanation why our email was going down for a few hours every day in the early hours of the morning….BEN at hostgator has a big attitude problem as well…
[...] I gave up on Hostgator a while ago, and I thought I’d cancelled my account until I noticed they were still charging me monthly (yeah, I should pay more attention to what’s on my CC bill). When I contacted them about it they claimed I never fully cancelled. Here’s a copy of the form I submitted: HGSales #GSW-[[private]] October 3, 2005 8:10:40 PM EDT Subject: CANCELLATION Department: Hostgator Sales Request Details: Your Email: : [[private]] Domain name: : MaisonBisson.com Your sites username: : [[private]] Your sites password: : [[private]] Why do you wish to cancel? : Poor service. Do you give us permission to terminate your entire account as soon as you submit this form? : Yes [...]
HostGator’s cease and desist letter against http://www.hostgator-sucks.com is now featured on EFF’s http://www.chillingeffects.org:
http://www.chillingeffects.org/domain/notice.cgi?NoticeID=4029
[tags]HostGator Sucks, Web, Reseller, hostgator-sucks.com[/tags]
HostGator.com is crap. Poor customer support, their reps are idiots and their prices aren’t the best either!
Don’t waste your money on HostGator!
HostGator.com sucks!
There are many hidden things in hostgator,They give you 100000…GB data space but you can not have more than 50000 files!
They abuse department shut down my sites and tell lies about my account claiming I was using more than allowed CPU.
These ideates took 2 weeks to refund my money (not a big money but I make a point to get it back from them)
samething here, tells lies about my account for using too much cpu while my website has only very few visits a day and the cpu that they showed in the email was only 6% rather than 25% or more.