Swiftype (YC W12) Builds Site Search That Doesn’t Suck(techcrunch.com)
techcrunch.com
Swiftype (YC W12) Builds Site Search That Doesn’t Suck
http://techcrunch.com/2012/05/08/swiftype-launch/
73 comments
Google would like you to believe that everyone stopped innovating when they did. There's actually a thriving market for site search but very little of it is for free. Check out SLI which has been doing cool stuff in this space for years. http://www.sli-systems.com/solutions/site-search
More likely all the low hanging fruits are gone and further improvements in web search requires some serious resources.
Ivinco Website Search (http://getwebsitesearch.com/) already one year on the market in custom search.
This is so freaking awesome. I've had this problem on nearly every website and blog I've created. Also at both of my startups (Scribd and Parse -- note: Parse uses Swiftype on our documentation pages -- try it out).
I've tried google site search (which sucked and wouldn't index all the Scribd pages for example, and it wasn't good enough for the parse documentation) and I've also used lucene and sphinx (pain to set up and the marketing folks always complained about search results, speed, ordering, relevance, etc, and analytics were a pain). Swiftype is the best option by far. I love this company -- really excited they finally have launched and I can talk about them.
Think about all the sites that use google custom site search just because that's the only decent option for them up until today. Huge market here. Congrats Swiftype.
I've tried google site search (which sucked and wouldn't index all the Scribd pages for example, and it wasn't good enough for the parse documentation) and I've also used lucene and sphinx (pain to set up and the marketing folks always complained about search results, speed, ordering, relevance, etc, and analytics were a pain). Swiftype is the best option by far. I love this company -- really excited they finally have launched and I can talk about them.
Think about all the sites that use google custom site search just because that's the only decent option for them up until today. Huge market here. Congrats Swiftype.
I tested the Parse.com documentation (https://www.parse.com/docs/search#stq=xcode) and found some problems with swiftype.
When searching via Chrome, I kept getting a blank content area with no search results. I did some digging (thinking that the issue might be that I reject 3rd party cookies) and I found the problem:
https://swiftype.com/embed.js wasn't loading. I went to view this script in my browser and got an error which was preventing the script from executing: "The identity of this website has not been verified. Server's certificate is not trusted." I clicked "proceed" to view the script in the browser, and when I went back to the search page, it worked as intended.
When searching via Chrome, I kept getting a blank content area with no search results. I did some digging (thinking that the issue might be that I reject 3rd party cookies) and I found the problem:
https://swiftype.com/embed.js wasn't loading. I went to view this script in my browser and got an error which was preventing the script from executing: "The identity of this website has not been verified. Server's certificate is not trusted." I clicked "proceed" to view the script in the browser, and when I went back to the search page, it worked as intended.
On a friend's mac, opening a new page in the browser brings up the Apple homepage, and for some reason my muscle memory is such that instead of the browser's search toolbar, on that computer I sometimes click the Apple.com search field, which has a really nice expanding style effect as it gets your focus. Then I always laugh because, haha, like anyone can find anything on Apple's site.
I don't think I would try it if I wanted to get to the online store (I would Google "apple.com store") if they removed the "store" tab to make room for something else...
That is just how low my expectation of in-site search is. The status quo is abysmal!
So, I agree there is quite a bit of room for improvement here... Just remember my laughter in the 0.2 to 0.5 seconds or however long it takes for the apple.com search field to expand... :)
I don't think I would try it if I wanted to get to the online store (I would Google "apple.com store") if they removed the "store" tab to make room for something else...
That is just how low my expectation of in-site search is. The status quo is abysmal!
So, I agree there is quite a bit of room for improvement here... Just remember my laughter in the 0.2 to 0.5 seconds or however long it takes for the apple.com search field to expand... :)
Very cool. I hope they start to explore the implications of indexing a trusted source at some point. Google ignores most metadata on your pages because for all it knows you're using that metadata to game the system. With site search that's not an issue, so the search engine could look for certain metadata on the pages it crawls, from author to importance/boost to activating keywords. The interface currently allows you to do some of this (e.g. reordering), but being able to guide the indexer from within your CMS would be even nicer.
Also +1: more clarity on pricing, please.
Also +1: more clarity on pricing, please.
Spot on. This is one of the real advantages to leverage in a site search system. We have only scratched the surface of what's possible, but there is a lot more to this coming soon.
Also, not sure if you caught this on the site yet, but we do have support for custom metadata: http://swiftype.com/documentation/meta_tags
Also, not sure if you caught this on the site yet, but we do have support for custom metadata: http://swiftype.com/documentation/meta_tags
At my company, in addition to having a need to index our own content, we also have a need to index content of a few other partner organizations. Does Swiftype have functionality in place to help out in this scenario?
As an example: We are foo.org but are also partnered with bar.org and baz.org. A search for "code awareness" should return a result from either of the three websites.
As an example: We are foo.org but are also partnered with bar.org and baz.org. A search for "code awareness" should return a result from either of the three websites.
We do support multiple domains and even diverse types of sources (crawl and API in a single engine). Unfortunately we don't have it exposed in the interface just yet! We will get that turned on in the next day or so.
Ooh, thanks, I'd missed that!
I love the idea, and I think there is a huge, ignored opportunity here. But relevance is what makes or breaks search engines, and dabbling with a search engine I made for Joel on Software (like in the example video), I think you should work on tuning your scoring function.
Searching for "about me" (with quotes) returns the "Distributed Version Control is here to stay, baby" article at http://www.joelonsoftware.com/items/2010/03/17.html first, and http://www.joelonsoftware.com/AboutMe.html second. While it's pretty stupid that the "About Joel Spolsky" text is in a <div> tag versus a <hX> tag, maybe you could weight the title more, because likely people will be using Swiftype to search curated corporate sites that are typically less spammy than the general Internet.
Anyway, great job so far, and I'm very excited to see this and where this could lead!
Searching for "about me" (with quotes) returns the "Distributed Version Control is here to stay, baby" article at http://www.joelonsoftware.com/items/2010/03/17.html first, and http://www.joelonsoftware.com/AboutMe.html second. While it's pretty stupid that the "About Joel Spolsky" text is in a <div> tag versus a <hX> tag, maybe you could weight the title more, because likely people will be using Swiftype to search curated corporate sites that are typically less spammy than the general Internet.
Anyway, great job so far, and I'm very excited to see this and where this could lead!
I love the idea, and when I created a search engine for http://jamesshore.com, it complained that the web page doesn't exist. Ditto for http://www.jamesshore.com. Trying again colored the "Name" field red without any indication of what was happening.
After poking around a bit more, it looks like my search engine was created. But when I click it, I get an error message: "You have not created any DocumentTypes in James Shore yet. Please use the API to get started." This is despite selecting the "Crawler" option when creating the engine.
Also, deleting an engine asks me to confirm, but then doesn't actually appear to delete it.
I want to love this tool, so I'm looking forward to when these growing pains are resolved.
Edit: A third attempt appears to have worked. I'm guessing you're seeing a lot of load, and your code isn't handling timeout errors well?
After poking around a bit more, it looks like my search engine was created. But when I click it, I get an error message: "You have not created any DocumentTypes in James Shore yet. Please use the API to get started." This is despite selecting the "Crawler" option when creating the engine.
Also, deleting an engine asks me to confirm, but then doesn't actually appear to delete it.
I want to love this tool, so I'm looking forward to when these growing pains are resolved.
Edit: A third attempt appears to have worked. I'm guessing you're seeing a lot of load, and your code isn't handling timeout errors well?
Looks cool, but I'm a bit reluctant to add a product to my website with an unspecified pricing model. Although: "the majority of our crawler-created search engines will remain free."
We use SearchSpring. Does anyone know if this is an advantage over than? Currently, with SearchSpring we can:
- specify result rankings;
- add facets to our products and then SS allows the customer to do things like compare products, restrict to over $$, or only white, grey and red;
- does fuzzy completions etc.
I ask because while the product we use is OK, it doesn't have a great API, and I want to be able to integrate more tightly with the service and allow myself to do things like A/B tests, dynamic results, customer specific rankings etc.
- specify result rankings;
- add facets to our products and then SS allows the customer to do things like compare products, restrict to over $$, or only white, grey and red;
- does fuzzy completions etc.
I ask because while the product we use is OK, it doesn't have a great API, and I want to be able to integrate more tightly with the service and allow myself to do things like A/B tests, dynamic results, customer specific rankings etc.
The demo video is really impressive. Most websites have terrible search functionality, and google's offering is pretty sucky too. This market is absolutely huge.
Congrats, this looks fantastic!
Quick feature suggestion: Can we replace/ignore specific text in the titles? I'm indexing a site with "Page title - Site Name" as the title of each search result, and manually editing 100+ pages is no fun :)
Alternately, you could compare the window title and the h1 list (or against other page titles) and automatically adjust accordingly.
Quick feature suggestion: Can we replace/ignore specific text in the titles? I'm indexing a site with "Page title - Site Name" as the title of each search result, and manually editing 100+ pages is no fun :)
Alternately, you could compare the window title and the h1 list (or against other page titles) and automatically adjust accordingly.
Another quick suggestion: Show a tally of how many pages have been indexed in the admin panel. Cheers!
We use Swiftype at TwitchTV and it's amazing. Great technology, easy to integrate, simple APIs.
Can't recommend Swiftype enough.
Can't recommend Swiftype enough.
I'm beginning to think a good way to find startup ideas is to pick through the mess Google has left behind with its labs and different free tools out there and actually make a go of creating a business around them. The problem with all these free Google offerings is that they scare away potential startups but aren't really run like businesses so it doesn't matter if they actually solve the problem well or not. Swiftype is a great example of a startup doing pretty much exactly that. I can guarantee the average response to their idea was "well, why not just use Google search."
I can think of a few Google tools that could be done way better if a startup put a lot of effort into solving the problem better, like Google Groups which is nigh unusable due to spam and a crappy interface.
I can think of a few Google tools that could be done way better if a startup put a lot of effort into solving the problem better, like Google Groups which is nigh unusable due to spam and a crappy interface.
Very cool. The only problem I see with this is that if Swiftype is down, or slow, your website search is affected too... Depends on how you and your readers value this functionality of course. Some websites don't rely on search as badly as others do. But for basic stuff this is great.
Any chance Swiftype might get equal billing with HNSearch as the 'official' site search of HN?
Looks great, but a small suggestion is to actually implement site search on their own site, if not only for demoing purposes. I would guess that many would love to take it for a test drive right there. Apart from that, excellent job.
I am impressed guys - great job so far.
Site search was a huge distribution strategy Google used to do search. They simply worked up market until they had tons of users going to Google.com itself.
Site search was a huge distribution strategy Google used to do search. They simply worked up market until they had tons of users going to Google.com itself.
This is awesome. I wonder if they plan to use this as a vector to go after Google's market, in the long-term. Anyone at Swiftype able to comment on their long-term goals?
[deleted]
> On top of that, Swiftype also allows site owners to pin and unpin different items to the top of their search results.
Obviously a great feature for the site owner. But I'm sure if that's a useful feature for the users. At least Google's results are objective, and wouldn't give me the page generates the most revenue as the first link.
Obviously a great feature for the site owner. But I'm sure if that's a useful feature for the users. At least Google's results are objective, and wouldn't give me the page generates the most revenue as the first link.
I hear your point, but in this case, "objective" from Google means more like "random".
Google has invested heavily in relevance for their main web search, which is obviously excellent. But they've totally ignored their site search product and if you try it, you'll find that the relevance is quite poor.
Hopefully, sites will use Swiftype's controls not just to maximize short-term revenue, but to improve relevance and user experience. After all, with Google site search, Google can only use public relevance signals like inbound links to rank pages. But on your site, you have lots of internal data about which pages are most important and you can use that to steer people towards the pages they're probably looking for.
Google has invested heavily in relevance for their main web search, which is obviously excellent. But they've totally ignored their site search product and if you try it, you'll find that the relevance is quite poor.
Hopefully, sites will use Swiftype's controls not just to maximize short-term revenue, but to improve relevance and user experience. After all, with Google site search, Google can only use public relevance signals like inbound links to rank pages. But on your site, you have lots of internal data about which pages are most important and you can use that to steer people towards the pages they're probably looking for.
After clicking Create Engine I get a 404. Returning to the home screen I see the new engine listed, but when I click on it I just see the message:
You have not created any DocumentTypes in ENGINE_NAME yet. Please use the API to get started.
What am I doing wrong?
You have not created any DocumentTypes in ENGINE_NAME yet. Please use the API to get started.
What am I doing wrong?
[deleted]
Matt and Quin are two of the most talented people I have had the pleasure of working with. They have so much potential as entrepreneurs. I am proud to be Swiftype investor :) Great idea and great team.
Hopefully a Wordpress plugin is in the pipeline. This would be a no-brainer any Wordpress site if setup is as simple as installing and configuring a plugin.
There is! It is functional already, just a little more testing before we release it.
One design suggestion:
I watched the video, I'm intrigued, now I want to use Swiftype -- but there is no search form on your homepage.
Searching your own site would be easiest, but I realize there isn't much content yet, so not the best demo.
What about using the logos of sites using Swiftype to activate a search form that actually takes you to their search results? "Trusted by these companies, give Swiftype a test..."
I watched the video, I'm intrigued, now I want to use Swiftype -- but there is no search form on your homepage.
Searching your own site would be easiest, but I realize there isn't much content yet, so not the best demo.
What about using the logos of sites using Swiftype to activate a search form that actually takes you to their search results? "Trusted by these companies, give Swiftype a test..."
Both of those are great suggestions. We'll likely make those changes soon. Thanks for the thoughts!
This has been one of the top ideas in my list of startup ideas for a while. I feel a bit bummed for not doing it, but I'm glad someone else is doing it. I should really trust my instincts more and just build things I think should exist.
Edit: I'm trying it out now, but keep getting 500 errors. You're probably getting a lot of traffic right now.
Edit: I'm trying it out now, but keep getting 500 errors. You're probably getting a lot of traffic right now.
I actually did build this. With a few websites using it, but never officially launched. In hind-site I wish I had, but other things got in the way and I have no regrets.
the question is, what is at the top of your list now?
I'm really excited about this because it's a great idea that meets a huge need.
Quick feedback: while the demo video plays, the screenshot slideshow continues. In my browser, this was just beneath the bottom left of the video and was really distracting. Could you pause the slideshow when the video popup is displayed?
Quick feedback: while the demo video plays, the screenshot slideshow continues. In my browser, this was just beneath the bottom left of the video and was really distracting. Could you pause the slideshow when the video popup is displayed?
Good catch. I'll make that change right now.
How does this differ from IndexTank?
That's what I thought. IndexTank was purchased by LinkedIn and open-sourced their engine it appears.
This looks really cool, kudos to the guys that built it. One thing that's worth pointing out:
Google Site Search and Google Custom Search are two different things. The former is free and a "wrapper" as was noted, the latter is incredibly powerful and easy to implement/tune.
That said, I'd rather give my money to these guys assuming they have a sane pricing model.
This looks really cool, kudos to the guys that built it. One thing that's worth pointing out:
Google Site Search and Google Custom Search are two different things. The former is free and a "wrapper" as was noted, the latter is incredibly powerful and easy to implement/tune.
That said, I'd rather give my money to these guys assuming they have a sane pricing model.
Google Custom Search is pretty powerful indeed, but I wouldn't call it easy. The docs are pretty dense and you need to tweak all sorts of XML documents even for simple stuff like best bets.
IndexTank is/was an indexer, this is a crawler+indexer, which means it's easier to set up as you don't have to connect it to your backend.
Will there be support for location-aware searches so we can index/search by geo lat/long?
Definitely. Swiftype currently supports geo-based searching/filtering, but it appears we haven't pushed out the docs for it yet. They will be up soon!
Awesome! Can't wait to give it a spin.
Does this use ElasticSearch?
http://tapirgo.com/
For reference, that is what I'm currently using on my website, http://pepijndevos.nl
For reference, that is what I'm currently using on my website, http://pepijndevos.nl
Wow, this is really impressive! Great timing with the google site search changes.
I just wonder what the price will be like after beta is finished?
Using simple drag and drop to adjust search rank is brilliant, and the minimal design seems to follow all of Dieter Rams' 10 rules for good design.
in http://swiftype.com/engines/bca/document_types/new I click on Delete This Engine, click OK on "Are you sure? This action is permanent." and the engine is still there.
Am I missing something?
Engine Key: f1EwCDo3g9qCmJsmNnGC
We're working on a fix for this right now. Thanks for the heads up!
And just pushed the fix. Please let us know if you see it again! [email protected]
Why don't you give a quick demo of your product by providing a site search box on your own site?
Acquisition in Three, Two, One....
I LOVE YOU MATT RILEY... AND QUIN... GO TEXAS A&M
[deleted]
Search as a service, awesome.
[deleted]
Hosted site search as a market has had basically zero innovation since Google launched its site search product. That was like 6 years ago.
If you've ever tried Google site search, you know how much it sucks. It's basically just a thin wrapper around a site:yourdomain.com Google search. It gives you zero control over the index (it uses the public Google index), and it's completely inappropriate for any large-scale site.
Swiftype is a perfect fit for sites that want a high-quality, customized site-search that feels like it was built from scratch. Even if you're already using Solr on your site, Swiftype can be great for creating a lightweight search engine, like for your help/documentation section.