Tag: Spam

I’ve had it with Movable Type comment spam blitzkriegs dropping available server CPU to 0 and broadsiding web and mail services. Last night we endured a comment spam attack so severe it knocked out the mail server overnight. If you’ve followed this space for a while, you know I’ve tried virtually every trick and upgrade at my disposal to deal with the problem. But it just keeps getting worse.

A few minutes ago, I switched this weblog to a comment-registration-required system. I know this will discourage a percentage (probably a good percentage) of casual comments, and that’s a bummer. But TypeKey registration is trivially easy, and your registration will work at any TypeKey-enabled blog on the internet.

I’ve also just announced the new comment registration policy on status.birdhouse and to the owners of our four most intensive MT users.

My hatred of spammers is boundless and bottomless.

If an href tag includes the rel="nofollow" attribute, well-behaved search engines won’t follow the links they represent when spidering. So if there was a way to automatically modify the links that comment spammers leave in comments, their chief goal — raising their standings in the search engines — would be deflated.

SixApart has just released the nofolllow plugin, which scans incoming comments and adds rel="nofollow" to each embedded link automatically. Normal users are not affected — they can still click the links. But the simple presence of links to spammer’s sites will do nothing whatsoever for their GoogleRanks.

The downside, as I see it, is that for this to be effective, it must be intalled in the majority of weblogs. Spammers need to understand that their campaigns are flaccid, and that won’t be true until most of the world is using a solution like this.

Just installed nofollow at birdhouse and at the J-School.

Applying the MovableType 3.14 upgrade made a huge difference in server CPU usage when undergoing comment spam blitzkriegs, which now amount to barely a blip on the resource usage radar. Peace at last. Until…

A few days later we face a new anomaly: Someone out there has created a script that submits fake comments containing randomly generated URLs (all non-active and non-registered), randomly generated fake IPs, and randomly generated fake email addresses — they’re coming in locust clouds of one or two hundred at a time.

Because there are no recurring strings in these comment spams, blackisting them is pointless, and would only fill a blacklist database with garbage. Because the domains advertised are non-existent, I can’t correctly classify them as spam – they don’t advertise anything. Their purpose is purely vandalistic; to annoy blog owners and admins.

Even though Blacklist doesn’t catch them, they’re still held for moderation (so resource usage is nill), but you do have to take the time to batch-delete the suckers.

Posted a query to see if anyone had advice on battling this form of nihilism, but nothing useful so far. I’m quickly coming closer to the last resort: Forced registration for untrusted commenters.

Heard from a reporter at eWeek yesterday who wanted to interview me about Movable Type comment spam overloads and how they affect web hosts. Unfortunately I got the email too late and wasn’t interviewed for the story, which was published today.

Six Apart has released MT 3.14 to address a bug which was triggering rebuild behavior even in settings where it shouldn’t be necessary, such as when moderated comments are added (99% of comment spam is held as moderated by various mechanisms). We’ll be applying the patch to birdhouse blogs throughout the day.

The weblog comment spam problem has implications beyond crowded inboxes for users. Even with tools such as the incredible MT-Blacklist (which has blocked or moderated tens of thousands of comment spams on birdhouse-hosted blogs in the past few months), each request still requires a CGI process and a database request. When the spambots launch their massive onslaughts, shared hosting environments reel from the resource requirements. The problem has reached a critical threshold, and the muckety mucks at SixApart are coming out of the woods to address it head-on:

Jay Allen (author of MT-Blacklist and Product Manager at Six Apart) and Anil Dash (big cheese at SixApart) have both posted “official” positions on MT comment spam in the past few days.

So it looks like patches will be released in the next few days to address the biggest issues for web hosts. I like the fact that they’re approaching this not just as an MT problem but as an issue that affects all online discussion forums. The key to satisfying frustrated web hosts will be in creating a solution that can somehow block comment spam blitzkriegs without having to make a CGI and/or database call for every incoming request. It’s a hard problem to solve.

Update: Very good read on the many aspects and dimensions of comment spam load issues over at photodude. Throwing more hardware at the problem doesn’t make it go away (drooling over the server described there). Long comment section, also worth reading. One comment on the question of whether dynamic or statically generated sites fare better under this kind of load:

Also, last month, my husband and I shut down WordPress on the colo server we share with 3 other people, because … hits from comment spammers were making everything so slow. So we installed prerendering, which, if I’m reading this correctly, takes away the advantage of WP being dynamic(?) [right – this would make a dynamic site behave like a static site; you can’t win. -SFH].

At OJR, a fairly exhaustive piece on the history and status of weblog comment spam, including this zinger from Irish blogger Antoin O Lachtnain, laying down the gauntlet (though I doubt it had any effect):

Relevant comments are very welcome, whether you agree or disagree with what I have to say. However, advertising of goods or services is not permitted on this forum without payment of a fee. The fee per advertisement is 500 Euros, which is payable immediately by bank draft. If you post an ad but do not pay the charge immediately you have corrupted data on this Web site without my permission. As such, you are guilty of criminal damage under the Criminal Damage Act, 1991 and subject to a prison sentence of up to 10 years and a fine of up to 12,700 Euros…Please note that posting on this forum will have no effect whatsoever on the PageRank of any links that you post.