Ask YC: Review my startup, Userfly
51 comments
I hate to sound like wet blanket, but from my personal experience, browsing sessions by themselves aren't all that useful in learning about your users. Over the past few years, I've worked a lot with recording browsing sessions (I was the creator of Tapefailure, and now VisTrac.com, which is a work-in-progress).
The main issue is that the sessions aren't focused (you don't know about the goal of the user) and watching tens/hundreds/thousands of recordings to get an idea of what users are or aren't doing on your website is simply impractical.
What, I believe, is really important is summarized data; taking the data and boiling it down to more specific bits of information about how the users are browsing: What are they doing on the page? How long does it take them? Collectively, what is and isn't being focused on? What about how users interact with forms?
These are just some very broad examples, but there are many ways in which you can distill the recorded data, and I find those to be far more insightful than the browsing sessions themselves (look at some of what ClickTale is doing).
There is a lot that can be done in this field (otherwise I wouldn't be working in it myself), but I think recording user sessions is only the very least of it.
Despite my reservations, I'd love to have a chance to further discuss your plans. What would be the easiest way for me to get directly in touch with you? (My email is available via the "Questions? Comments? Contact." link on VisTrac.com)
The main issue is that the sessions aren't focused (you don't know about the goal of the user) and watching tens/hundreds/thousands of recordings to get an idea of what users are or aren't doing on your website is simply impractical.
What, I believe, is really important is summarized data; taking the data and boiling it down to more specific bits of information about how the users are browsing: What are they doing on the page? How long does it take them? Collectively, what is and isn't being focused on? What about how users interact with forms?
These are just some very broad examples, but there are many ways in which you can distill the recorded data, and I find those to be far more insightful than the browsing sessions themselves (look at some of what ClickTale is doing).
There is a lot that can be done in this field (otherwise I wouldn't be working in it myself), but I think recording user sessions is only the very least of it.
Despite my reservations, I'd love to have a chance to further discuss your plans. What would be the easiest way for me to get directly in touch with you? (My email is available via the "Questions? Comments? Contact." link on VisTrac.com)
I think a great service would be a simple script which for each tracked URL tells statistically the average amount of time spent on each part of site while scrolling it down, and a probability distribution of cursor position on whole page.
For example, I wonder if PG wouldn't like to see what parts of his essays readers need to read/rethink twice :-)
For example, I wonder if PG wouldn't like to see what parts of his essays readers need to read/rethink twice :-)
Maybe this would be a great upsale for clients that don't want to muddle through the videos themselves (cough, Fortune 500 marketing folks who love reports, cough) -- have the videos cranked thru Amazon's Mechanical Turk and then the Turkers can summarize events.
Very cool idea. I would definitely use something like this. I only noticed 1 small issue: it seems tabbing between input fields doesn't register. As soon as I entered text on the first input, I tabbed to the second textarea and entered text. The video didn't register the second textarea text until I had clicked one of the radio buttons (using Safari 3.1.2 on Leopard 10.5.5)
Perhaps you could also work in goals somehow. This is one of the best things about usability tests--giving users a set of goals to complete and watching how they respond. Maybe offering certain users a chance to take place in the study with X reward?
Also, CrazyEgg (another site for visualizing clicks but implemented as a heatmap) lets you run campaigns, which is extremely useful. Considering you generally want to run tests like this after you've made changes, having a set limit is a good idea (10 hours, 100 users, etc...).
One small critique, I got bored with the video and went straight for the demo. I'd suggest making the video shorter or somehow showing "the goods" up-front.
Very nice idea and good implementation.
Perhaps you could also work in goals somehow. This is one of the best things about usability tests--giving users a set of goals to complete and watching how they respond. Maybe offering certain users a chance to take place in the study with X reward?
Also, CrazyEgg (another site for visualizing clicks but implemented as a heatmap) lets you run campaigns, which is extremely useful. Considering you generally want to run tests like this after you've made changes, having a set limit is a good idea (10 hours, 100 users, etc...).
One small critique, I got bored with the video and went straight for the demo. I'd suggest making the video shorter or somehow showing "the goods" up-front.
Very nice idea and good implementation.
Thanks for the feedback. Your idea about offering certain users a chance to take place in the study is dead on to what we think is the next logical step. Connecting an actual user with actionable contact information could help companies close the loop with an actual user, and interact with them in the same way they do in paid usability studies.
Did you have a cold or something? I suggest you to clear out your nasal passages before demo'ing. Unless the sniffing sound is for effect ;) Overall, very nice work. Keep it up.
I haven't had this much fun on a demo since some random Flash app from the late 90s. Well executed cte, while some may discredit your app for providing no concrete application, I for one salute you, I would have never thought to pass a user's intuition as input. This puts the voyeurism of a Trojan virus into a web app, which is what computer illiterates love to look at. This could be a very bankable idea.
So if one of my user logins, you get their login + password? What about CC info?
This being said, it look super useful. Can we get the option to record say, a user out of 100?
This being said, it look super useful. Can we get the option to record say, a user out of 100?
To answer your first question, we don't actively capture username/passwords. However, in order to follow a user into an authenticated site, we have to either setup some type of proxy (which requires some work on the client's side), or we do some simple cookie capturing (which requires no work on the client's side) to see what the user sees. Obviously, cookies might contain sensitive information, which is why we're offering this as an optional premium service only.
As for your second question, we can certainly setup the service such that it captures a certain percentage of users, and it's certainly a route we would consider depending on the size of the client.
As for your second question, we can certainly setup the service such that it captures a certain percentage of users, and it's certainly a route we would consider depending on the size of the client.
I would have bought a subscription for this right away, as we were discussing rolling something like that on our own for a client.
However, your pricing page being a "mailto" link put me off. Tell the price right away.
Also, what are "advanced" events?
63KB is a little too heavy I'd say. Perhaps rewrite this so it doesn't use jQuery, or at least give me the option to skip it in the tracker js if I'm already using it on my site.
However, your pricing page being a "mailto" link put me off. Tell the price right away.
Also, what are "advanced" events?
63KB is a little too heavy I'd say. Perhaps rewrite this so it doesn't use jQuery, or at least give me the option to skip it in the tracker js if I'm already using it on my site.
We're still unsure of the cost to run this service as we scale up. We are looking for beta testers to try out our advanced features, so please email us, and we can get you a pro account for free, and start to iterate on the product to meet your specific needs (and lighter weight JS is definitely something we can fix for you).
we wrote something exactly like yours for corporate customers and although I love jQuery, we decided it was best to write it with just straight-up javascript. It was less code than I thought it would be considering how quick & dirty jQuery lets us do things.
Also, if you need to do something like the jQuery css selector style stuff, you might want to look at sizzle by jeresig. It clocks in at 4kb which isn't too big of an overhead.
Also, if you need to do something like the jQuery css selector style stuff, you might want to look at sizzle by jeresig. It clocks in at 4kb which isn't too big of an overhead.
The screencast is great - I might use this for a project I'm working on. The demo is very useful as well to see exactly what it does and doesn't capture (selecting text).
In the pricing table, does "users per hour" mean the number of sessions that are captured? That's what I took it to mean but I'm not sure.
You should probably put some answers to your FAQs :-)
In the pricing table, does "users per hour" mean the number of sessions that are captured? That's what I took it to mean but I'm not sure.
You should probably put some answers to your FAQs :-)
That's exactly right. One user per hour means a single session. And, that seems like a good candidate for the first FAQ :)
Thank you!
Thank you!
This is amazing! Seriously, this looks like an extremely valuable web app for people to remotely study real users.
Great idea and solid execution, but I don't like your pricing page:
http://userfly.com/pricing
What is a Basic event vs. an Advanced event? How can I choose when I don't even know what those are?
http://userfly.com/pricing
What is a Basic event vs. an Advanced event? How can I choose when I don't even know what those are?
Our pricing page sucks. Will fix asap. In the meantime.
Simple events = mouse movements, clicks, focus events, scrolling etc.
Advanced events = DOM mutation events triggered by ajax and other types of sophisticated javascript: http://en.wikipedia.org/wiki/DOM_Events
We will likely have to do customer specific fixes to get advanced event captures working well during our beta phase, which is why we want you to email us if you are at all interested.
Simple events = mouse movements, clicks, focus events, scrolling etc.
Advanced events = DOM mutation events triggered by ajax and other types of sophisticated javascript: http://en.wikipedia.org/wiki/DOM_Events
We will likely have to do customer specific fixes to get advanced event captures working well during our beta phase, which is why we want you to email us if you are at all interested.
Love the concept, could see myself using this. Might recommend a different color scheme (I associate those with Microsoft and XP) and perhaps more polish in general on the design side. Otherwise, very nice.
The service itself looks very well done. I'd be a little leery of using it on an internal admin page since you're crawling the content and I wouldn't want to leak any info to your servers.
The marketing portion of the web site looks like a work in progress. There's too many nav points (every corner). I would simplify that. The video could use some polish but it gets the point across. I would shorten the part where you're recording a session to get to the playback quicker since that's the selling point.
Overall though, very impressive work.
The marketing portion of the web site looks like a work in progress. There's too many nav points (every corner). I would simplify that. The video could use some polish but it gets the point across. I would shorten the part where you're recording a session to get to the playback quicker since that's the selling point.
Overall though, very impressive work.
Your startup idea is awesome!
A few things:
1. If you want to get more signups, make your homepage LOOK better.
2. If you could get the referral source of the user (and make it optional, cause I might want to use it on my webapp), you could get more of an idea of what that person wanted from your site (like if they got there by searching "how to write user interaction javascript", you would know their goal)
3. You need to charge for this!
4. I'll get more once I sign up and implement it on my site(s).
A few things:
1. If you want to get more signups, make your homepage LOOK better.
2. If you could get the referral source of the user (and make it optional, cause I might want to use it on my webapp), you could get more of an idea of what that person wanted from your site (like if they got there by searching "how to write user interaction javascript", you would know their goal)
3. You need to charge for this!
4. I'll get more once I sign up and implement it on my site(s).
The demo shows the actual keys being pressed down. They also say that the recording is stored on their server. Does this mean that if I used their services on my site, they'd have a recording of my user logging in? If so, how would I be comfortable with this, without knowing whether or not the recording contained the username/password of my visitor, and who can see these recordings?
I'd trust this service more, if it was a script, and the recordings were stored on my server only.
David Timar
I'd trust this service more, if it was a script, and the recordings were stored on my server only.
David Timar
Great work on a hard problem... maybe you could adapt it to make follow-me interactive tutorials... like a hostess who directs you to a site's most pertinent features.
Neat.
Might be neat to try "merging" movies for a particular page. Perhaps it's not too easy because you're modifying one copy of the DOM though. It seems like it'd be useful to watch a whole bunch of users' interactions with a particular page at the same time, and then drill down to more closely examine a few who got confused, etc. I feel like it might get a bit tedious to watch a hundreds of versions of almost the same thing otherwise.
Might be neat to try "merging" movies for a particular page. Perhaps it's not too easy because you're modifying one copy of the DOM though. It seems like it'd be useful to watch a whole bunch of users' interactions with a particular page at the same time, and then drill down to more closely examine a few who got confused, etc. I feel like it might get a bit tedious to watch a hundreds of versions of almost the same thing otherwise.
Really great job, i mean that. What were some of the challenges you faced? I've never captured a session before, sounds like an interesting problem beyond cookies.
Good site, that video is great. Got it straight away and seems a very exciting and useful tool.
Wonder if it works with Flex/Flash apps... I imagine not...
Pricing-wise, I think you should make the free version more than 1 user session and per-day rather than per-hour... 10 user sessions per day sounds and feels better than 1 user session per hour, somehow.
20 user sessions per day would sound pretty good.
Wonder if it works with Flex/Flash apps... I imagine not...
Pricing-wise, I think you should make the free version more than 1 user session and per-day rather than per-hour... 10 user sessions per day sounds and feels better than 1 user session per hour, somehow.
20 user sessions per day would sound pretty good.
The interactive demo is great, and this is certainly a valuable service if it works well (is robust on real-world complex AJAX apps, doesn't slow things down too much, can handle high volume). I've tested www.robotreplay.com which is/was a very similar service and it fell short.
I think you should be more open about your pricing - asking us to email you for an estimate is going to turn a lot of people off.
I think you should be more open about your pricing - asking us to email you for an estimate is going to turn a lot of people off.
Very cool indeed! That's a good point about admin pages. Perhaps you can have some kind of flag or config setting so that we can set which pages or directories we don't want to be captured.
I have a question about SSL sessions. The video mentioned that replays are done through you since the web page is crawled, so are you spoofing and logging in as the user in order to do the logged in pages?
I have a question about SSL sessions. The video mentioned that replays are done through you since the web page is crawled, so are you spoofing and logging in as the user in order to do the logged in pages?
I assume it doesn't work well with Flash apps, right?
I guess so. One more little reason not to develop web applications using a proprietary overgrown and buggy ad banner creation tool.
What should a person use to develop graphically-intensive web applications, then?
Quake III is quite intensive and runs on all platforms. Figure out how it's made.
Not everything needs to be cramped into a browser frame, have an annoying login screen and eat 20% of CPU while sitting idle. Oh, did I mention the hidden benefit of being useful offline?
Not everything needs to be cramped into a browser frame, have an annoying login screen and eat 20% of CPU while sitting idle. Oh, did I mention the hidden benefit of being useful offline?
I'm not really interested in deploying non-web apps at this point.
HTML Canvas element + javascript?
The experience of HTML Canvas element + javascript on IE goes something like this (according to Vladimir Vukićević, the creator of the ActiveX plugin that enables canvas graphics in IE):
"Currently, the experience is pretty crappy: you have to click through an infobar to allow installation of this component, then you have to click 'Yes' to say that you really want to run the native content, and then you have to click 'Yes' again to allow the component to interact with content on the page," he wrote in a blog entry. "In theory, with the right signatures, the right security class implementations, some eye of newt, and a pinch of garlic, it's possible to get things down to a one-time install which would make the component available everywhere."
http://arstechnica.com/news.ars/post/20080819-mozilla-drags-...
If you want to go to war against IE, feel free, but the pragmatist in me says no way.
"Currently, the experience is pretty crappy: you have to click through an infobar to allow installation of this component, then you have to click 'Yes' to say that you really want to run the native content, and then you have to click 'Yes' again to allow the component to interact with content on the page," he wrote in a blog entry. "In theory, with the right signatures, the right security class implementations, some eye of newt, and a pinch of garlic, it's possible to get things down to a one-time install which would make the component available everywhere."
http://arstechnica.com/news.ars/post/20080819-mozilla-drags-...
If you want to go to war against IE, feel free, but the pragmatist in me says no way.
This seems like it would pair really nicely with the Feedback Army site we saw recently. It might be interesting to incite Turkers to do virtual usability studies with this technology. Especially if you could give them a task to accomplish.
Thanks for the heads up. This is VERY invasive in terms of online privacy expectations. Welcome to my AdBlock list.
Very sleek implementation though. Perhaps you should consider licensing productized version of the same idea.
Very sleek implementation though. Perhaps you should consider licensing productized version of the same idea.
I'm not sure if you're trolling, but I don't see how adblocking this site gets you much. I'm sure many large sites get the same information in similar or other ways, but just don't happen to be outsourcing the gathering of that information.
Neat idea.. my only worry would be scalability on the end-user side. I.e., it might be difficult to get an aggregate understanding of user behavior without going through tons of videos.
This is really neat. Love the idea. I'm not looking for an estimate, but can you give a general idea of pricing?
Honestly, we haven't thought enough about our pricing; the cost will likely be proportional to the # of users you need to capture in a given period of time. For small sites I don't see us charging more than $10 a month. We are also playing around with the idea of licensing our software, so that companies can run the captures internally to enable ajax functionality, and keep their data private.
how does it differ from http://www.clicktale.com/?
Nice! I played around with the demo for a few seconds, and quickly saw your value proposition.
we just launched an ecommerce shop and where looking around for just this type of tool, going to dig in a bit more but surface review looks pretty solid. the free account look pretty limited, and it looks like a middle option for 2-4 users would be nice.
very cool good luck with this... might have to install it and give it a shot on the next web interface I work on.
this is really a great idea, and i can see how useful it is fro qa purposes
nice app. love the simplistic approach.
but anyone in useability will tell you that you ruin the experiment if the user knows that someone/software is looking "over their shoulder". any software installed or testing environment introduced will produce different results in users. this is why larger sites settle with coarser-grained analytics derived from tracking cookies and js beacons (which can measure abandonement). indeed cookies+beacons are a superior technique because you can randomize or exclude parts of your test body at will, or even other metrics (geolocation etc) to add/remove people to tests at will. indeed, everyone reading this has unwittingly been in a test for a major website at some point without knowing it (which means your data is actually valid!)
In short, Userfly captures actual browsing sessions from real users as if you were looking over their shoulder. What we've found is that watching real users actually use your site can be remarkably insightful, so we built Userfly to accomplish that goal.
There's a screencast that will walk you through the basic functionality. Click on the demo button above it to mess around without having to sign up for an account.
Would love your feedback on the idea and its direction!