{"id":816,"date":"2003-09-29T10:59:59","date_gmt":"2003-09-29T10:59:59","guid":{"rendered":"http:\/\/sunpig.com\/mt-entry-816.html"},"modified":"2006-09-23T19:30:11","modified_gmt":"2006-09-23T19:30:11","slug":"bayesian-filter-for-blog-comments","status":"publish","type":"post","link":"https:\/\/sunpig.com\/martin\/2003\/09\/29\/bayesian-filter-for-blog-comments\/","title":{"rendered":"Bayesian filter for blog comments"},"content":{"rendered":"

I don’t get much comments spam myself right now (maybe a message a week or so), but the problem is definitely getting<\/a> worse<\/a>. <\/p>\n

For Movable Type installations, there are several solutions available, such as an option to provide a “delete this comment” link<\/a> with every “new comment” email, and a combined url blocker\/comments hider technique<\/a>. Also, some people have proposed collaborative blacklists<\/a>, or collaborative authentication<\/a> for comments posters.<\/p>\n

I’m surprised that no-one seems to have suggested Bayesian filtering<\/a> for comments, though. I get about 15-20 spam messages via email every day, but the SpamBayes plugin for Outlook<\/a> routes almost all of them straight into a “Spam” folder. I never see them in my inbox. Maybe one or two message in a hundred make it through the filter, and I haven’t had any false positives for ages<\/em>. It doesn’t involve maintaining blacklists, and it’s a lot less effort than deleting every single junk message.<\/p>\n

In Movable Type, it you could have a “bayesfilter” property on the MTComments template tag: <MTComments bayesfilter=\"1\"><\/code>. All comments would have to pass through the filter, and only those that were not spam would make it on to the page.<\/p>\n

You’d need some additional mechanism to “train” the system, and somewhere to put the statistical knowledge base the filter uses to tell spam from genuine comments. Finally, you’d need a way of correcting the system after the initial training, so that any spam that does make it through can be deleted with prejudice, and so that false positives can be corrected.<\/p>\n

This would be a nice anti-spam comments system. It would involve a Movable Type plugin, and some hacking to the Movable Type application itself. Unfortunately I don’t have time to do this right now, and even if I did have time, I’ve sworn off perl. (Did you know that “perl” is an anagram of “pain”?) But I wonder if the Lazyweb<\/a> could do it for me, or if the nice people at Six Apart<\/a> would be so kind as to include this feature in MT Pro?<\/p>\n","protected":false},"excerpt":{"rendered":"

I don’t get much comments spam myself right now (maybe a message a week or so), but the problem is definitely getting<\/a> worse<\/a>. <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24],"tags":[],"class_list":["post-816","post","type-post","status-publish","format-standard","hentry","category-blogging"],"_links":{"self":[{"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/posts\/816","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/comments?post=816"}],"version-history":[{"count":0,"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/posts\/816\/revisions"}],"wp:attachment":[{"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/media?parent=816"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/categories?post=816"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sunpig.com\/martin\/wp-json\/wp\/v2\/tags?post=816"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}