Should I use delegated or native comments for my niche community site?
5 comments
Worrying about spambots and trolls is basically a premature optimization. You call this a niche site but some how it is going to be popular enough spambots and trolls will want to come. If it is truly a niche site then popularity will by definition be limited. So which is it, niche or hugely popular?
Even if it is a non-niche site, your assuming it will be popular with trolls/spambots. So as you can see your worrying about a problem you may never have. However, you are wise to think about it and be ready.
Since you said you want this to be a learning experience, go ahead and write your own commenting system. It will teach you a lot. Put up some small barriers to prevent spam. Captcha, email verification, SMS verification, etc. These are all pretty easy to do and there are services that will help you out.
Lastly, if it does become a troll hole then be prepared to migrate your data. I would say put off writing any migration scripts, but maybe look at the comment services and see how you can import comments. I've done this before on Disqus and it is not too hard.
Even if it is a non-niche site, your assuming it will be popular with trolls/spambots. So as you can see your worrying about a problem you may never have. However, you are wise to think about it and be ready.
Since you said you want this to be a learning experience, go ahead and write your own commenting system. It will teach you a lot. Put up some small barriers to prevent spam. Captcha, email verification, SMS verification, etc. These are all pretty easy to do and there are services that will help you out.
Lastly, if it does become a troll hole then be prepared to migrate your data. I would say put off writing any migration scripts, but maybe look at the comment services and see how you can import comments. I've done this before on Disqus and it is not too hard.
On the flipside if you don't want to learn and want to just finish the project fast, comment services are pretty nifty. But definitely don't just choose the service over spam concerns alone.
Don't worry about spam bots. One of my websites has more than 2500 visitors daily and I am still waiting for the first spam comment.
Spammers don't search pages by hand. Their programs look for "footprints" of the usual software (ie Wordpress) and try to automatically add comments. If you build your own commenting system from scratch and use the good practice for building (preventing XSS..), you are ok.
Or put it this way, if your website becomes so popular, that spammers will customize their programs to try to automatically post on your page, you have bigger problems :)
Or put it this way, if your website becomes so popular, that spammers will customize their programs to try to automatically post on your page, you have bigger problems :)
I run http://blogspam.net/ which does real-time spam blocking for forums/blogs/etc. On the whole you're correct, spammers go for easy targets.
But I started the project when a custom CMS of mine (more or less) started getting added to the spammer-lists. It seems like it's only a matter of time these days until people find your site and submit 10-500 comments a day. Some I can see are clearly bots, or compromised IPs, others are clearly humans (presumably paid very cheaply).
But I started the project when a custom CMS of mine (more or less) started getting added to the spammer-lists. It seems like it's only a matter of time these days until people find your site and submit 10-500 comments a day. Some I can see are clearly bots, or compromised IPs, others are clearly humans (presumably paid very cheaply).
Did you have auto-aprove comments? AA is a spammers heaven, and they will walk the extra mile if they find that kind of a CMS. Even if you have no-follow links.
Very nice initiative for the spam-fight though. How high is your success rate? Do you use any blueprints for spun comments?
I'm not sure what you mean by "auto-approve" - There are plugins for things like wordpress which will junk comments automatically, solely on the result of the test, but others are more fine-grained. (The API does allow whitelisting/blacklisting by IP, etc.)
The success rate is pretty fluid but about 80% of comments submitted were judged to be spam today. Whether there are too many false negatives/positives is hard for me to say ..
The success rate is pretty fluid but about 80% of comments submitted were judged to be spam today. Whether there are too many false negatives/positives is hard for me to say ..
Auto-approve - comments in blog show up immediately without moderation or approval.
Good luck with fighting spam. There are never enough of you!
Good luck with fighting spam. There are never enough of you!
Thanks! This helps a lot. Part of my curiosity about commenting best practice stems from an ignorance of how spambots really work.
It's nice to hear from someone in the same space.
It's nice to hear from someone in the same space.
No problem CoreSet. Look at it this way. You will learn something. Oh, and have a kill-switch readdy if Murphy comes (disable comments, non auto approve, something like that).
@stevekemp also has a valid point in another comment of his. If you write your own functionality, you store the comments. If you use 3rd party app, they store your comments. This can be the tipping point for some people.
How the spammers work. Really short and banal explanation: - dear program, find wordpress blogs (in a certain niche, or any blog - I am not picky today) - footprint of the day will be "powered by wordpress"
- thanks for the list, now check each url if that blog has a form for comments enabled
- great, now, because you were programmed to know which inputs (name, e-mail, content) (and with what names/IDs) needed to be filled, use my predefined texts and submit
- oh, and if you see a capcha, would you be so kind to OCR it and fill the right text there also? thanks
- check after one hour if the comment is visible (yes? woohoo - we found an auto-approve blog! - no? ok, check in one week)
Hmm, that gave me an idea for a honeypot webpage. One single page with footprints of many blogs and forums, some forms on it and IP logger. Every IP that would like to submit anything on this page, goes to the sh.tlist.
How the spammers work. Really short and banal explanation: - dear program, find wordpress blogs (in a certain niche, or any blog - I am not picky today) - footprint of the day will be "powered by wordpress"
- thanks for the list, now check each url if that blog has a form for comments enabled
- great, now, because you were programmed to know which inputs (name, e-mail, content) (and with what names/IDs) needed to be filled, use my predefined texts and submit
- oh, and if you see a capcha, would you be so kind to OCR it and fill the right text there also? thanks
- check after one hour if the comment is visible (yes? woohoo - we found an auto-approve blog! - no? ok, check in one week)
Hmm, that gave me an idea for a honeypot webpage. One single page with footprints of many blogs and forums, some forms on it and IP logger. Every IP that would like to submit anything on this page, goes to the sh.tlist.
IMHO, I believe you need to align your expectations about value you wanna extract from the comment section.
Do you want spread the content or you want to enrich the content? It's very important to know.
I can see you already accepted the challenge to build the thing from nothing. What worries you seems the hard and must-do decision, because if you chose a delegated system you will have a big problems to migrate the content generated through the comments.
If you want just spread the content, go with the delegated systems, FB for e.g.
If you want to create value, build your system, it will give you space to explore possibilities that underlies out of your sight.
Do you want spread the content or you want to enrich the content? It's very important to know.
I can see you already accepted the challenge to build the thing from nothing. What worries you seems the hard and must-do decision, because if you chose a delegated system you will have a big problems to migrate the content generated through the comments.
If you want just spread the content, go with the delegated systems, FB for e.g.
If you want to create value, build your system, it will give you space to explore possibilities that underlies out of your sight.
I think I understand what you mean about native comments allowing me to do more to enrich their content, but I'm curious about what you mean by "spread?"
Could you elaborate?
Could you elaborate?
What I mean about "spread"... You can use facebook comments, genius annotations or even ask for comments in leaderboards platforms like reddit or HN.
Thinking about delegated systems I can only see two purposes: a) Make your project/content be acknowledged by the people (Spread by piggybacking) or b) Cut off engineering resources regarding to the lack of value created through those comments.
At some point, we need to decide if the comments are a value to keep or to trade.
Thinking about delegated systems I can only see two purposes: a) Make your project/content be acknowledged by the people (Spread by piggybacking) or b) Cut off engineering resources regarding to the lack of value created through those comments.
At some point, we need to decide if the comments are a value to keep or to trade.
I would make your own one. Worst case is you learn from making it, then switch over to another commenting system.
I think that making one will also give you more of an insight into how the delegated ones work - that can't be a bad thing!
I think that making one will also give you more of an insight into how the delegated ones work - that can't be a bad thing!
You could compromise and setup your own external site.
e.g. I wrote https://github.com/skx/e-comments/ for a static-site that wanted comments. You could use something like that, then later bring it into the main app/site as required. The value here is that you own the data, not disqus, etc.
e.g. I wrote https://github.com/skx/e-comments/ for a static-site that wanted comments. You could use something like that, then later bring it into the main app/site as required. The value here is that you own the data, not disqus, etc.
I'm adding authentication right now and, facing the O-Auth question, wondering if I should go with delegated comments via Facebook or Disqus, or stick with my native system.
Since this is a side project (I'm a writer in a tech-ish field) I don't have much time for moderation. My biggest fear is that, left (relatively) untended, my small community site will be overrun by spambots and trolls.
A delegated system doesn't completely forestall this, but it does offer its own spam tools and (in the case of FB especially) a persistent user identity that might discourage trolling. On the other hand, I could do more with a native system, and if it turned out to be possible, I'd love to add my own simple spam/hatespeech detection as another learning exercise.
Does anyone have experience with a similar side project? Or are there any moderators who might have insight into the system?