Search Engine with User Defined Filters
3 comments
Google Search used to have "blocked sites" feature:
http://googleblog.blogspot.com/2011/03/hide-sites-to-find-mo...
They dropped it in 2013:
http://googlesystem.blogspot.com/2013/03/google-discontinues...
http://googleblog.blogspot.com/2011/03/hide-sites-to-find-mo...
They dropped it in 2013:
http://googlesystem.blogspot.com/2013/03/google-discontinues...
Yes, but blacklisting is not what I am thinking of. I mean the ability to exclude whole chunks from the crawler. Google would still crawl those sites, but the blacklist would exclude them from results.
I'm working on something similar to this in a very defined vertical. The results will be divided into website types and the user can specify which result types they want to see.
You have a specific use case you're looking for or thinking of creating one?
You have a specific use case you're looking for or thinking of creating one?
I will try to give an example. During recent Charlie Hebdo incident, I wanted to find their address. If you search for "charlie Hebdo Paris Address" the first page and many subsequent pages are all dominated by media outlets. (http://imgur.com/3nyjURA) I should go to "maps" tab to get the address
This happens very often. Media outlets dominate search results.
This happens very often. Media outlets dominate search results.
Well, say I am a programmer, and I find I can get most of what I need from the internet from Europe, North America, and Australia, for example. And, to keep the pool small and focused I turn off all media sources (news, entertainment, etc.). The crawler would use white-lists to include obvious omissions. I wonder how small you could get the pool of indexed data and still get effective search results.
I see and share your pain. No filters exactly but you can use millionshort.com to filter out most media/newspaper websites.
This is opposite of how search engines work now. They search everything and filter based on search criteria, not source criteria. Some might argue that you just add more sophisticated results filters to get the same thing, but then you have the problem of sources trying to defeat your filters all the time.
I suppose the ethical internet folks might put something together that boycotts certain segments of the net very openly, or provides the end user with the ability to tailor the input sources to suit their needs. Kind of like Search as SaaS. Does that seem viable?