NOM down for a week? What happened?

[CaptainSalty]

Questions about the outage of June 10-19

Was NOM offline for a week?
Yes, totally.

Holy crap! What happened?
The short answer about why is that the website host had a very serious software problem with the hosting program that NOM uses. It was cpanel software which is only marginally relevant to the operation of NOM, but was enough to knock us offline.

Was anyone else offline?
Oh, yes. Lots of other websites hosted on Bluehost were offline if they relied on the same software. We were just one of them.

Did we lose any posts or users?
It does not look like it. I have a full copy of the database and website files.

Couldn't you just rebuild the site?
I actually did rebuild the entire site with all posts and data on a server in my office. This was not something I could simply attach to the internet, partially because it would have disrupted the operation of Netflix by other family members. My wife and daughters are binge watching "Once Upon a Time" and it would not have been skillful disrupt that.

Is the problem fixed?
For NOM, it appears that the problem is fixed. I cannot speak for other websites on Bluehost.

What was Bluehost doing all week?!
Mainly drinking lots of caffeinated beverages and fielding support calls from customers angrier than me. Plus trying out lots of software patches to fix the exploit. Sometimes these things cannot be rushed.

What were you doing all week?
I called Bluehost every single day asking status and actually go to know one of the more helpful tech support guys. I was also exploring the finer points of server hosting including some failed forays into rebuilding NOM on a server owned by another company. It did not work for reasons that deeply annoy me. I also read a translation of the Tao Te Ching and played a bunch of video games. I did not binge watch "Once Upon a Time".

Can I log back in and start posting?
Yes, please do.

Can I ask more questions
Yes

[Journey]

Thank you very much for getting NOM back online, I’ve missed it!

[Mormorrisey]

Thanks for all your work - I really appreciate the email explanation, saved me from hyperventilating that NOM 3.0 needed to be created! Now my life has meaning again.

[DPRoberts]

Well done, Captain. Thanks for all your efforts to bring it back.

[Just This Guy]

Thanks for the status update e-mail that got sent out to everyone on Saturday. It's appreciated. It's was good to know it was a software issue and not the board just up and disappearing like the first one did.

So my thanks for your efforts to mange the site and be transparent on what is going on.

[FiveFingerMnemonic]

Thank you captain!

[wtfluff]

Thanks Cap'n.

That gnawing feeling that another piece of my digital existence had disappeared into the ether is gone. (We live in a weird digital world now, don't we? )

[Linked]

Thanks Captain! All your work is appreciated.

[slavereeno]

Thanks!

[Hagoth]

It's disturbing when things like this happen. You think they have redundancies that would prevent it but sometimes that doesn't address the problem.

I used to work for Sony Entertainment. I remember when Sony's Playstation Network went down for a month and a half. It was unbelievable. A huge international company's top priority was getting the network back on line, and it took a month and a half. I can only imagine how much revenue was lost. Sony got sued and fined from all directions. It ended with the top Sony executives publicly bowing seven times; apparently an act of deep humiliation and atonement in their culture.

[AllieOop]

Thank you Captain!

You're amazing and I appreciate all your hard work!!! It's great to be back online

[2bizE]

Thanks for getting us back up. I do appreciate your hard work on this.
I don’t understand all of the technical aspects. I can only repay by continuing with smart a$$ comments on NOM.

[Lithium Sunset]

I know this is late but thanks for the emails letting us know what was going on!