Around 18:30 CST, hackers compromised our account and made malicious modifications to several portions of the website. There is a discussion thread about it here on the 4hm forums. Below is a rolling journal of informational updates.
Summary: We're almost back to 100% again. The data on the website is stale, but it'll do until I get something fresher from the old account.
Had a mishap with the host around 9am where the database password got reset. Fixed.
Ok, everything should be fixed. Send me pm/email if you notice anything broken.
Success! I've got most of the site working now. I'm gonna call it a night.
Well, database is populated with Sept 17th data. Files are uploaded to the host. Now I'm hitting some silly little error where it can't find index.php, which is the root file of the whole website. No idea why, and it's fargin 4:30 in the AM, so I'm calling it a night.
hostrocket server still hasn't recovered yet, so I can't fetch the latest sql data from the server ANYWAY.
I am going to the Dallas Cowboys game at noon on Sunday, and I will be unavailable until 16:30 CST. Sorry guys, I tried to get us back up before dawn but it just ain't happenin.
I'm uploading the server backup to the new site. The database backup is stale (from 17th of September), so I will replace it with current data from the old server whenever I get access to it.
Domain name has been moved over to the new host. If you're seeing this, then that's a good sign of success.
I have secured an alternative means of hosting from a friend on irc. We'll be sharing server space with slackadelic.com. Tomorrow I'll begin the process of moving everything over to our new home.
Because of limited bandwidth at the new host, I'm retaining our current web host to function as a download site only.
I have secured an alternative means of hosting from a friend on irc. We'll be sharing server space with slackadelic.com. Tomorrow I'll begin the process of moving everything over to our new home.
Because of limited bandwidth at the new host, I'm retaining our current web host to function as a download site only.
Some background info:
» On Aug 27th, there was a break-in to another section of the website. This was a small testing grounds for an urelated project, but the key is that it was invisible to the outside world. There were no direct links to it from any websites, and it wasn't residing in the doc root. Which means the files were compromised from inside the server. This was later verified true.
» In response to the break-in, I requested that they move our website to a different server in their server farm. One that wasn't overrun with hackers. They did so, but screwed up a crucial detail in the move.
» On Sept 16th, hostrocket updated their php binaries and version of cpanel to improve security. Due to the previously mentioned screwup in the move, mayosource.org experienced roughly a full day's outage.
» After the outage on the 16th, I was livid and threatened to find a new host. I talked with hostrocket support, and after hearing about their security upgrades, I calmed down and decided to give them another chance.
"The admins are working on it," says the guy on the other end of the line. "It'll be a few more hours before things are back to normal." No, I don't believe this goes back to normal.
I just got off the phone with tech support, and things are worse than they seemed up front. The monkey on the phone told me that every account on the server was compromised. Every single one. I asked him to repeat that, because I didn't believe it.
They cram hundreds of accounts onto a single shared server. I don't understand how it's possible for hackers to modify files in hundreds of different accounts at the same time. Or, if this was an ongoing process, how did no one in the data center pick up on it?
Success! I'm talking to a human on the phone at tech support.
CPanel is busted. Phpmyadmin is toast. I can't configure the website. I have no way to check if our database was compromised, but I suspect it was.
I'm on the phone to tech support, no. 5 in the queue currently. Hostrocket's on hold music sucks ass.
hostrocket's faqs are non-functional and sections of their forums (shared hosting was the most active one) have been deleted. Things do not look good.
Yep, we got hacked. I'm restoring from backup as we speak, though I'm forced to do it the slow way because our host is also having serious problems. I can't currently get to the control panel for the website, and their faq is non-functional either.
Which makes me wonder if it was a coordinated attack. Our webhost, hostrocket.com has been struggling lately with a lot of hackers having inside access. Large swaths of websites have been compromised, usually all with the same MO. The common thread is that the hackers had gained access to the user that runs the apache process and modified whatever they could.
In any case, hang with me and we'll be back up as soon as I can upload and check a few things. One thing's for certain. We are done with these clowns. As of 3 hours ago I am looking for new hosting.
-- Catcher