Tuesday 14 September 2004

Making the spammers behave

There's one program that changed the way spammers harvested email addresses, and I think there's an important lesson to be learned from it. When it started being a good idea to trawl the entire World Wide Web for email addresses so you could sell your over-the-counter viagra, hundreds of programs arose to do that job. There is a protocol on the web called the "robots exclusion protocol" which is designed to keep automatic programs like these from going into places they shouldn't. Naturally, the spambots ignored all "Keep Out" signs in order to get all the potential email addresses they could. This infuriated many people.

Someone, however, came up with a brilliant solution: create a kind of page that automatically produces worthless email addresses, always looks different, and links to itself hundreds of times. Then post a "Robots Keep Out" sign and let 'er rip. Legitimate web trawlers (like search engines) don't get bogged down, because they obey the sign. Rude spambots, however, get stuck forever in an endless loop which not only prevents them from moving on to harvest other addresses, but clogs up their entire result database with rubbish.

The end result? Most spambots now obey the robots exclusion protocol, and you can keep your email address from being harvested simply by posting a "Robots Keep Out" sign. The important lesson here is that when spammers break the rules for their own benefit, all you have to do is make it worth their while to obey them again.

Mokalus of Borg

PS - If only this principle was always so easy to apply.
PPS - I sometimes think we need to re-think our defensive siege strategy if we're going to win.

No comments: