It is currently Sun Dec 22, 2024 1:21 am

Downtime, November 2015

For forum technical issues, problems, or connectivity outages and announcements, read on! Post here if you need help using our forums or services.

Downtime, November 2015

Postby Zancarius » Wed Nov 25, 2015 7:09 pm

Apologies for the downtime. Josh may be weighing in on this as he gets time, so I'll offer a short explanation.

Essentially, a long forgotten WordPress instance was likely the source of an attack which resulted in the system the goon forums were hosted on being used as part of a botnet. We didn't have time to perform a complete forensic analysis of the virtual machine, so after some discussion we simply decided to nuke the install to avoid bringing up a system that may or may not have had a rootkit on it. We have backups (as you can see), so it's not a total catastrophe, but Josh hasn't had the time to get his instance back up and running.

Instead, we've migrated it over to my VPS instance on Linode for the time being. Some services may be down until we decide what to do (TeamSpeak, the goon main site, some of Josh's blogs), but we decided to get the forums back up and running since there's still a few of us who occasionally use them. Even if mostly for giggles. I should note that I have backups of the TeamSpeak server data, key, and logins, so you shouldn't have to worry about requesting account verification or whatever else it is that TS requires for extra privileges. Unless you didn't back up your own client key and have reinstalled Windows, that is. ;)

I'll share some post mortem data plus mitigation strategies in the coming days.

Be aware that the forums are currently running in a php-fpm chroot, which might impact functionality somewhat, particularly when uploading image attachments. If you encounter errors, don't panic. Just post what you found, and I'll be getting around to resolving it as I get time. We're going to be focusing our efforts on mitigation and, eventually, migrating everything back to Josh's server.

Now is probably a good time to review this write up, now that I think about it, because outdated software was the likely culprit in this case. And anyone can forget about an old install of vulnerable software! (Think about this the next time you get frustrated with automatic updates, because older WordPress installs never provided that option!)
I gave that lich a phylactery shard. Liches love phylactery shards.
User avatar
Zancarius
Site Admin
 
Posts: 3907
Joined: Wed Jul 05, 2006 3:06 pm
Location: New Mexico
Gender: Male

Re: Downtime, November 2015

Postby Zancarius » Wed Nov 25, 2015 7:26 pm

I should also note that while there's no obvious indications the Goon databases were accessed, the usual precautions apply: Change your passwords. If you've used the same password on the Goon forums elsewhere, you ought to change that as well.

If you've changed your password recently, it'll be encoded using bcrypt, which is a strong cipher and has no known weaknesses at this time. Older versions of phpBB from which we migrated used somewhat weaker HMAC-based ciphers (probably backed by MD5) and can be considered somewhat suspect.
I gave that lich a phylactery shard. Liches love phylactery shards.
User avatar
Zancarius
Site Admin
 
Posts: 3907
Joined: Wed Jul 05, 2006 3:06 pm
Location: New Mexico
Gender: Male


Return to Issues and Support

Who is online

Users browsing this forum: No registered users and 0 guests

cron