How DuckDuckGo got in Time's best websites(gabrielweinberg.com)
gabrielweinberg.com
How DuckDuckGo got in Time's best websites
http://www.gabrielweinberg.com/blog/2011/08/how-duckduckgo-got-in-times-best-websites.html
51 comments
Hey Gabriel - 10 years from now where is DuckDuckGo? What's the vision?
10 years is too far out for me to speculate. In 5 years, I'd like to see us have significant market share on desktop, mobile and tablet. To get there you'll see us focus on the things we've been focusing on, i.e. working with partners and our own indexes to deliver way more and better instant answers from a variety of sources; way less spam; real privacy; and a simpler UX.
> way less spam;
That is exactly why I love DDG. I almost never get spam, but the fact that you want to make it even better is so awesome. Seriously, DDG's killer feature is a founder interacts with the users.
That is exactly why I love DDG. I almost never get spam, but the fact that you want to make it even better is so awesome. Seriously, DDG's killer feature is a founder interacts with the users.
>Seriously, DDG's killer feature is a founder interacts with the users. //
Doesn't scale well, how sustainable is this going forward.
Doesn't scale well, how sustainable is this going forward.
In other words, how much fun are you having, and how long is that going to last?
EDIT: A legitimate question, as Gabriel has cited fun as one of the perks of going DDG. How are people reading this as a negative comment/question? It was meant as a prompt to respond to the parent.
EDIT: A legitimate question, as Gabriel has cited fun as one of the perks of going DDG. How are people reading this as a negative comment/question? It was meant as a prompt to respond to the parent.
I could not imagine a better job.
[deleted]
Do you consider 'Google' competition or is your product/vision/direction so unique that they might be a complement?
It's really both. To the extent that people are switching to us from Google, that is clearly competition and our goal is to get people to use us as their primary search engine. However, a lot of what we do is very complementary to people who have a general crawl.
contacts, contacts, contacts!
I am not surprised. DuckDuckGo is a pretty good search engine. I am getting more and more dependant on their bang "!" syntax.
>I am getting more and more dependant on their bang "!" syntax. //
Doesn't that mean DDG is a pretty good proxy.
Doesn't that mean DDG is a pretty good proxy.
Exactly. I only really lob softball queries at DDG. I use ! to go to the first result on queries I know are going to have the right match first. Everything else I proxy to google through !g. DDG is great but the search quality (or perhaps the size of the data they index) is just not nearly up to par with Google.
This is a shameless plug, but using !g constantly on DDG led me to write GooseGooseGo, a trivial wrapper for DDG you can clone & run on localhost: http://goosegoosego.com
I see, it's some sort of search engine.
A majority of the search results are from Bing, blended with some other sources (Wolfram, Wikipedia, others).
From the FAQ:
http://duckduckgo.com/faq.html
Where do you get your results?
From over 30 sources, including DuckDuckBot (our own crawler), crowd-sourced sites (in our own index), Yahoo! "BOSS", "embed.ly", "WolframAlpha", "EntireWeb", "Bing" & "Blekko". For any given search, there is usually a vertical search engine out there that does a better job at answering it than a general search engine. Our long-term goal is to get you information from that best source, ideally in instant answer form.
http://duckduckgo.com/faq.html
Where do you get your results?
From over 30 sources, including DuckDuckBot (our own crawler), crowd-sourced sites (in our own index), Yahoo! "BOSS", "embed.ly", "WolframAlpha", "EntireWeb", "Bing" & "Blekko". For any given search, there is usually a vertical search engine out there that does a better job at answering it than a general search engine. Our long-term goal is to get you information from that best source, ideally in instant answer form.
[deleted]
Clean, simple, efficient, grassroots PR. And of course, you need a decent product, not some web 2.0 hyped local global widget.
We can definitely learn from Gabriel.
We can definitely learn from Gabriel.
I'm surprised the title got changed back. It was originally "How DuckDuckGo got in TIME's 50 Best Websites of 2011" and this more accurately reflects the reality. I don't think that's "gratuitous editorial spin" as warned against in the HN guidelines but, indeed, a reduction of spin.
Me too. On the blog side, I had went with the shorter title because I hate how it looks when it wraps. However, I just checked and that fits so I changed it to your title.
You mean that it fits on your choice of browser at your monitor resolution and text-size setting; right?
No, my blog is fixed width.
It's not important, but now I am confused.
In my browser, the title of the OP, namely, "How DuckDuckGo got in TIME's 50 Best Websites of 2011", takes up 3 lines because I have my browser set for really big text. If I decrease the text size one step, it goes down to taking up 2 lines.
In my browser, the title of the OP, namely, "How DuckDuckGo got in TIME's 50 Best Websites of 2011", takes up 3 lines because I have my browser set for really big text. If I decrease the text size one step, it goes down to taking up 2 lines.
I'm talking about the title on my blog site, not on this site. By fixed width, I mean that when you resize the browser it does not compress the size of that block of text, and so doesn't wrap/unwrap in that process. When I do zoom/unzoom it doesn't wrap/unwrap for me either.
I'm talking about the title on your blog site, too.
"When I do zoom/unzoom it doesn't wrap/unwrap for me"
It does for me, though. But again it's not important (to me).
"When I do zoom/unzoom it doesn't wrap/unwrap for me"
It does for me, though. But again it's not important (to me).
Peter, I have to agree. I nearly made the mistake yesterday of visiting a restaurant claiming "best overall restaurant, <cityname> dining awards" on their website. The reviews contradicted this, and it turned out they'd been making this claim since at least 2008. Much can change in 3 years in tech as restaurants. Similarly, being in the Times Top 50 is impressive, top 1000 less so, so the number is important context.
I'd love to see the traffic numbers that resulted from that article in Time, I always wonder how effective print media is at converting to visitors.
We put all of our query numbers up at http://ddg.gg/traffic.html.
Thanks for sharing! You've had nice continuous monthly growth with the direct queries. API usage seems to be hovering at 10M...any reasons for that? Was the jump in direct queries between December(2.4M) and January(5.1M) a result of the Donttrack.us site?
API usage has been mainly driven by one huge set of extensions, namely http://smarterfox.com/. So while more people start using our API, a ton would have to use it to move that needle.
The graph is annotated (though sometimes it doesn't come up). The major uptick in Jan was due to the donttrack.us site and the associated billboard and its associated press (a couple weeks after). You can see it in the daily #s very clearly if you scroll down.
The graph is annotated (though sometimes it doesn't come up). The major uptick in Jan was due to the donttrack.us site and the associated billboard and its associated press (a couple weeks after). You can see it in the daily #s very clearly if you scroll down.
[deleted]
Spot on about Twitter being underutilized. RT'ing those same journalists' stories (within reason and with your readers' interests in mind, of course) can also get you in their good graces. I was able to secure a guest posting spot with WalletPop (AOL's recently deceased personal finance blog) after the editor at the time sought me ought based on all of the RTs, FFs etc.
Soon enough, everyone will be doing this and it will no longer be a way to stand out. But the blog post is probably one of the best articles I have read on relationship building with journalists (Thank you Gabriel for continuing to be so open).
Rather utilizing Twitter to suck up to journalists hoping they notice, I note how DDG has created an online community that advocates loudly and clearly. Read: DDG has users that feel apart of the brand so much so that they engage journalists.
DDG's strong community is obvious here on HN and the internal forum is a great idea too. Credit to Gabriel for knowing not just the importance of being transparent but also HOW to express himself and his brand.
I'm only wondering how this can be applied to B2B brands where it's harder to gain traction with people that have the time to advocate.
Rather utilizing Twitter to suck up to journalists hoping they notice, I note how DDG has created an online community that advocates loudly and clearly. Read: DDG has users that feel apart of the brand so much so that they engage journalists.
DDG's strong community is obvious here on HN and the internal forum is a great idea too. Credit to Gabriel for knowing not just the importance of being transparent but also HOW to express himself and his brand.
I'm only wondering how this can be applied to B2B brands where it's harder to gain traction with people that have the time to advocate.
How exactly does DDG make money? You aren't storing search info so I can't imagine that any ads would be very targeted. Is it a "Scale then revenue" plan?
The FAQ says:
How do you make money?
A couple of ways. First, there is some minimal advertising in the right column with providers that work within our privacy principles, i.e. no targeting or sharing IP addresses to serve ads. The other way we make money is via affiliate sales, e.g. Amazon and eBay. This affiliate model also doesn't share personal information -- see our privacy policy for details.
How do you make money?
A couple of ways. First, there is some minimal advertising in the right column with providers that work within our privacy principles, i.e. no targeting or sharing IP addresses to serve ads. The other way we make money is via affiliate sales, e.g. Amazon and eBay. This affiliate model also doesn't share personal information -- see our privacy policy for details.
Is DuckDuckGo self-funded?
Yes:
DuckDuckGo was solo-founded by Gabriel Weinberg in February 2008 and is based out of Valley Forge, PA (USA). It has been self-funded to date.
https://duckduckgo.com/faq.html
DuckDuckGo was solo-founded by Gabriel Weinberg in February 2008 and is based out of Valley Forge, PA (USA). It has been self-funded to date.
https://duckduckgo.com/faq.html
That's awesome, many congrats.
Whilst Twitter is (obviously) now very well known, you are right that many people underestimate it from a communication point of view.
Whilst Twitter is (obviously) now very well known, you are right that many people underestimate it from a communication point of view.
Can you give a little insight about the algorithm ( good question - isnt it ) ?
I have a website that is no1 in Google in my niche word and has been for years - its an old html site and has an nicheword.com domain name
but your engine gives the first search result for "nicheword" - my Facebook and Flickr accounts, then other companies websites, instead of my actual website which is no1 on Google,
and my actual website is like in 10th place in your results which is pretty bad compared to Google,
looking at the DDG search results , it seems like:
1) you put more emphasis on social networds than Google ,
2) and you seem to put less emphasis on the domain address ( nicheword.com ) than Google does ,
I wonder is that logic reasonable - in terms of relevancy of the results,
3) you seem to like wordpress and other well known platforms and dislike good old .html websites ..
I have a website that is no1 in Google in my niche word and has been for years - its an old html site and has an nicheword.com domain name
but your engine gives the first search result for "nicheword" - my Facebook and Flickr accounts, then other companies websites, instead of my actual website which is no1 on Google,
and my actual website is like in 10th place in your results which is pretty bad compared to Google,
looking at the DDG search results , it seems like:
1) you put more emphasis on social networds than Google ,
2) and you seem to put less emphasis on the domain address ( nicheword.com ) than Google does ,
I wonder is that logic reasonable - in terms of relevancy of the results,
3) you seem to like wordpress and other well known platforms and dislike good old .html websites ..
I don't have an easy straightforward answer for you. We focus on a few things wrt to algo: 1) figuring out the best sources to deliver for instant answers/0-click info; 2) removing useless sites; 3) trying to detect "context" a bit from the query and make results match it as best we can, which also fits into 1).
Since this is turning into a bit of a FAQ, I'll ask something I've been wondering. DDG is dependent on its competitors. If DDG gains significant market share, what is to protect it from them just blocking it?
Obviously there is a symboitic relationship with some of its sources, but not all. How does Google/Bing benefit from it using their results?
Obviously there is a symboitic relationship with some of its sources, but not all. How does Google/Bing benefit from it using their results?
I believe DDG uses Yahoo! BOSS (http://developer.yahoo.com/search/boss/) as one of its primary indexes (check out DDG's FAQ). BOSS is a pay per use model (as due to Yahoo's deal with Microsoft, BOSS needs to charge to cover cost) - so it benefits Yahoo! directly. Also, now that Yahoo is out of search crawling/indexing business, it is not really concerned about DDG becoming a competitor - in fact it is a validation of the BOSS model which is something Yahoo will like to promote.
That's right. A couple of other data points. a) Ask/AOL also don't have crawlers and have significant share. b) There are a few places to get a decent base page rank feed now, and it appears to be increasing.
My understanding, as an outside observer, is that DDG is more about building a community (similar to HN, but probably more like metafilter). I get this idea from their offering of an XMPP service and site-specific syntax, so I may be off, but I would say they would like to cultivate a tight-nit community (I'm a scroogle user, but this sounds interesting depending on how they continue).
You are right -- we would very much like to cultivate a tight-nit community. At the same time, we'd like to go mainstream as well, and we hope the community will help us do so.
DDG also has its own crawler which iirc they plant to eventually switch over to.
DDG is very clean and spam free, but I am being pulled back to Google for the tighter integration across products (Videos, Maps, Images)
I search for "San Francisco to Los Angeles" in the url and then I click on the map tab at top left. DDG proxy is good, but it demands me to memorize the shortcuts (the shortcuts list is impressive).
Adding a contextual proxy in the results page would help i.e "Did you mean !m San Francisco to Los Angeles"
I search for "San Francisco to Los Angeles" in the url and then I click on the map tab at top left. DDG proxy is good, but it demands me to memorize the shortcuts (the shortcuts list is impressive).
Adding a contextual proxy in the results page would help i.e "Did you mean !m San Francisco to Los Angeles"
That's a really good point.
I think it's quite possible this could be leveraged to corner a niche market, though. Think of people who use Unix--they can do tons with it, but there's a ton of commands to memorize compared with what's necessary for Windows. Some people prefer having a more complicated and specialized toolset.
I think it's quite possible this could be leveraged to corner a niche market, though. Think of people who use Unix--they can do tons with it, but there's a ton of commands to memorize compared with what's necessary for Windows. Some people prefer having a more complicated and specialized toolset.
Indeed -- we should be getting a lot better at this over time. Thx for the feedback.
One thing that annoys me with search engines including DDG is where they return results for the word they reckoned I meant to enter rather than the word i did enter.
So if I enter axemple.com I get a pile of results for example.com and an option to see results instead for axample.com
In the past they would have showed me results for what I actually typed with an option to see results for the other.
So if I enter axemple.com I get a pile of results for example.com and an option to see results instead for axample.com
In the past they would have showed me results for what I actually typed with an option to see results for the other.