Why did infinite scroll fail at Etsy?(danwin.com)
danwin.com
Why did infinite scroll fail at Etsy?
http://danwin.com/2013/01/infinite-scroll-fail-etsy/
62 comments
I only enjoy infinite scroll on touch screens. I think the perfect solution would be paginated results, but if the user ontouchstart pulls the page past the bottom, it would automatically load the next page (like reverse of the facebook app's pull-down for new items). It's much more convenient than trying to tap just perfectly on the "next" arrow, but paginated is just easier with a mouse.
That mindset of design I think is the next step after "responsive" design. Rather than just scaling the visual design of the page, how about truly designing the user interaction to fit multiple scenarios. If it were up to me, I'd call it Symbiotic Design (responsive 2.0)
That mindset of design I think is the next step after "responsive" design. Rather than just scaling the visual design of the page, how about truly designing the user interaction to fit multiple scenarios. If it were up to me, I'd call it Symbiotic Design (responsive 2.0)
That sounds like a great idea. I do agree, touch seems to fit infinite scroll pretty well - pagination navigation is a pain on touch devices.
"Infinite scroll" is unutterably stupid and incompetent on Twitter. The inability to skip back x pages just ensures that the vast majority of data -- including tweets and follower details -- can never be accessed. The solution is to use http://manageflitter.com/ or some other service, at least until Twitter finds yet another way to screw up even more of its supporting ecosystem....
> But the A/B tests showed various negative effects of the feature, including fewer clicks on the results and fewer items “favorited” from the infinite results page. And curiously, while users didn’t buy fewer items overall, “they just stopped using search to find these items.”
How is this a negative effect? The amount of stuff that users are buying should be pretty much the _only_ metric you care about. And they are probably not using search as much because browsing is much nicer with infinite scroll.
It sounds like they just did infinite scroll wrong.
If you read the comments page on the forum, the biggest complaint is that you lose your place in the scroll when you return to the scroll page. If they had listened to their users and made it so that you return to the spot where you clicked on the item, perhaps it would have been a success.
How is this a negative effect? The amount of stuff that users are buying should be pretty much the _only_ metric you care about. And they are probably not using search as much because browsing is much nicer with infinite scroll.
It sounds like they just did infinite scroll wrong.
If you read the comments page on the forum, the biggest complaint is that you lose your place in the scroll when you return to the scroll page. If they had listened to their users and made it so that you return to the spot where you clicked on the item, perhaps it would have been a success.
When I use these things (and even pagination sometimes), I might find something I like, but it's so easy to browse ahead that I'll skip over it in the hope that I'll find something I like even more further on.
At that point, there's no easy way to get back to what I originally liked unless I had the forethought to open it in a tab or remember it exactly.
As for the metrics, it depends on how the business wants to position itself. It's evident from the quote that they care more about just the sales and would prefer to encourage more engagement with their site.
At that point, there's no easy way to get back to what I originally liked unless I had the forethought to open it in a tab or remember it exactly.
As for the metrics, it depends on how the business wants to position itself. It's evident from the quote that they care more about just the sales and would prefer to encourage more engagement with their site.
> How is this a negative effect? The amount of stuff that users are buying should be pretty much the _only_ metric you care about.
Items ordered is most helpful for making decisions when in context. It's awfully difficult to gauge health or make decisions with in isolation. A superficial short-term return may not be as beneficial on other levels - strategically, operationally, vendor ecosystem...
The article and other commenters have pointed at some, a few more more basic things worth knowing:
- return visit behavior. If infinite scroll diminished visitors favoriting items, what's the relative conversion for visitors returning to a list of favorites vs not? (features like this, Amazon Wish List etc, usually perform pretty well).
- avg order value. Visitors are buying the same number of things, but are they making the same $$ per order? Preserving avg order value but increasing item count also increases operations overhead (which could be worth it to encourage a broader vendor pool...)
- Clicks to items helps Etsy identify interesting/trending items&sellers independent of sales (move your data knowledge upstream!). Also increases the data size on individual users to build a taste profile and surface more items of interest at an individual level
Items ordered is most helpful for making decisions when in context. It's awfully difficult to gauge health or make decisions with in isolation. A superficial short-term return may not be as beneficial on other levels - strategically, operationally, vendor ecosystem...
The article and other commenters have pointed at some, a few more more basic things worth knowing:
- return visit behavior. If infinite scroll diminished visitors favoriting items, what's the relative conversion for visitors returning to a list of favorites vs not? (features like this, Amazon Wish List etc, usually perform pretty well).
- avg order value. Visitors are buying the same number of things, but are they making the same $$ per order? Preserving avg order value but increasing item count also increases operations overhead (which could be worth it to encourage a broader vendor pool...)
- Clicks to items helps Etsy identify interesting/trending items&sellers independent of sales (move your data knowledge upstream!). Also increases the data size on individual users to build a taste profile and surface more items of interest at an individual level
> The amount of stuff that users are buying should be pretty much the _only_ metric you care about.
Generally yes, we would trade an increase purchases for anything else. The test was a negative effect because we forced people to stop using search, which was the feature we were experimenting with. Making buying less efficient, even if people want to buy so much that they eventually succeed, isn't a great result.
> And they are probably not using search as much because browsing is much nicer with infinite scroll.
No for two reasons. First, browse pages didn't have infinite scroll back when this test was done. And second, the point is that people seeing infinite scroll purchased less from search as compared to the control. This was a controlled A/B test.
As gfodor says, yes the back button did return you to your place (or it did, once we ironed out all of the problems with it in early incarnations).
Generally yes, we would trade an increase purchases for anything else. The test was a negative effect because we forced people to stop using search, which was the feature we were experimenting with. Making buying less efficient, even if people want to buy so much that they eventually succeed, isn't a great result.
> And they are probably not using search as much because browsing is much nicer with infinite scroll.
No for two reasons. First, browse pages didn't have infinite scroll back when this test was done. And second, the point is that people seeing infinite scroll purchased less from search as compared to the control. This was a controlled A/B test.
As gfodor says, yes the back button did return you to your place (or it did, once we ironed out all of the problems with it in early incarnations).
Given that I've never yet used an infinite scroll implementation that had reliably-working 'back' button behavior, I'm skeptical of the claim that yours worked. I'd put some money on the hypothesis that it worked under ideal conditions, or worked in testing, but failed an irritating fraction of the time in actual real-world usage.
EDIT: In fact, even if by some miracle it didn't, it probably still triggered a twitch of stress and irritation in users who assumed it would.
EDIT: In fact, even if by some miracle it didn't, it probably still triggered a twitch of stress and irritation in users who assumed it would.
Totally possible
I helped out with this project at Etsy and obviously our implementation intended for the back button to work as expected. Our initial rollout worked fine in most browsers but certain older versions of IE were discovered to have issues. Of course, the experiments that Dan referenced in his talk were in regards to known-good implementations that did not have issues with the back button.
Anyone who has worked on infinite scroll knows that the back button is a "interesting" technical challenge.
Anyone who has worked on infinite scroll knows that the back button is a "interesting" technical challenge.
> the back button is a "interesting" technical challenge.
Especially when some A/B testing seem to show that it is a bad UX idea (for shopping goods).
Side thoughts: I wonder if they did A/B test "My apps" page on Google play (https://play.google.com/apps), it manages to have all the wrongs of infinite scrolling (no fixed ground, no sane back button) while not being infinite scrolling.
Especially when some A/B testing seem to show that it is a bad UX idea (for shopping goods).
Side thoughts: I wonder if they did A/B test "My apps" page on Google play (https://play.google.com/apps), it manages to have all the wrongs of infinite scrolling (no fixed ground, no sane back button) while not being infinite scrolling.
I'm curious how seamless that implementation was. I've seen a few hacks that load-then-scroll which is, of course, just about the worst of all worlds.
IIRC we ended up using a cookie (in non web history api browsers) so when you hit back the page would render the entire necessary result set to get you back where you were. Ie, no Ajax load to wait for. (Don't quote me on this :))
> The amount of stuff that users are buying should be pretty much the _only_ metric you care about.
I agree if Etsy is a 100% commerce focused, but they aren't. Key to their success has been their ability to build up a community and sell stuff. Even if you weight down the impact of clicks and favourites the effect is still negative with less exploration and interaction happening. What would be the long term impact of sales of this reduction?
I agree if Etsy is a 100% commerce focused, but they aren't. Key to their success has been their ability to build up a community and sell stuff. Even if you weight down the impact of clicks and favourites the effect is still negative with less exploration and interaction happening. What would be the long term impact of sales of this reduction?
> the biggest complaint is that you lose your place in the scroll when you return to the scroll page.
It doesn't have to be that way -- http://news.ycombinator.com/item?id=2592741
It doesn't have to be that way -- http://news.ycombinator.com/item?id=2592741
Infinite scroll that remembers your place vs infinite scroll that doesn't. Now that would be an interesting test. Although I think I know which one would win.
I have this problem frequently on many sites; whether its from a UI deficiency or my own memory error. I have trained myself to always open listed/grid links in a new tab as a result.
When building sites now, I usually contemplate using target=_blank on any lists just to aid this kind of pain point.
When building sites now, I usually contemplate using target=_blank on any lists just to aid this kind of pain point.
That's a good point, about whether it was really a failure if those users still bought things...certainly it didn't seem like a benefit to search at all...but maybe most people just end up buying through other avenues anyway, such as following a shop
Also to your point, Dan talked about how they also thought that opening results in new tabs would be a sure fire winner (because power users open up multiple tabs to do comparisons), but it was also a significant negative effect. So clearly Etsy users back up when venturing into results and apparently expect to be at the exact same state they were before clicking through
Also to your point, Dan talked about how they also thought that opening results in new tabs would be a sure fire winner (because power users open up multiple tabs to do comparisons), but it was also a significant negative effect. So clearly Etsy users back up when venturing into results and apparently expect to be at the exact same state they were before clicking through
> So clearly Etsy users back up when venturing into results and apparently expect to be at the exact same state they were before clicking through
Which makes sense, given that the multiple-tabs-at-a-time is often a power-user only thing. Etsy's users are much more "typical" in their usage of the browser; similar to Tumblr (who also ditched infinite scroll).
Which makes sense, given that the multiple-tabs-at-a-time is often a power-user only thing. Etsy's users are much more "typical" in their usage of the browser; similar to Tumblr (who also ditched infinite scroll).
Contrary to Dan McKinley's half-apology as quoted in the article, infinite scroll is stupid on your website.
Infinite scroll flies in the face of the way the human brain works with groups and sets, makes it virtually impossible to usefully search within the current page using the browser's search feature, etc. And the positives are? Nothing, other than novelty.
It is one of the many recent examples of webdev/designers doing something because it is possible and trendy and new rather than because it adds any value.
Paged results with a well designed indication of where you are plus good server-side categorization and server-side search to filter results is far preferable to infinite scroll in every practical situation.
Keep the infinite scroll for those purely arty non-commercial story-telling sites, if you make me try to use an infinite scroll interface to buy stuff from you I will buy nothing.
Infinite scroll flies in the face of the way the human brain works with groups and sets, makes it virtually impossible to usefully search within the current page using the browser's search feature, etc. And the positives are? Nothing, other than novelty.
It is one of the many recent examples of webdev/designers doing something because it is possible and trendy and new rather than because it adds any value.
Paged results with a well designed indication of where you are plus good server-side categorization and server-side search to filter results is far preferable to infinite scroll in every practical situation.
Keep the infinite scroll for those purely arty non-commercial story-telling sites, if you make me try to use an infinite scroll interface to buy stuff from you I will buy nothing.
Of course your opinion may or may not be right but the point is to figure out a way to test it. The reason infinite scroll was attractive at Etsy, among other reasons, is because it is reminiscent of the experience one has at a large craft fair where you quickly scan through hundreds of items on tables looking for one that jumps out as worth zooming in on.
I don't think those experiences are at all comparable. I'm a compulsive scanner, I can usually impress people by my ability to quickly scan pages of text without really processing any of it while finding a specific word or phrase. Infinite scrolling is nothing like that experience for me. For one thing it is impossible to involve peripheral vision with any type of web page scrolling and peripheral vision is absolutely essential to that type of scanning. And moving your eyes to scan something that is fixed vs. keeping your eyes still to scan something moving are also vastly different experiences in my opinion, not really at all alike.
Sure, this is a anecdote based argument levied against the hypothesis. Its a fairly weak way to make final decisions though. Again the point was that there was intuitive reason to believe this concept may be fruitful, the problem was we devised too large of an experiment and spent too much time to get to a point to validate the core hypothesis.
To be perfectly honest opinions like yours provide guidance of what to focus on but in general this type of intuition usually turns out to only capture part of the dynamics when something is actually used by millions of people, and in many cases intuition turns out to be the polar opposite of the reality the data tells you.
To be perfectly honest opinions like yours provide guidance of what to focus on but in general this type of intuition usually turns out to only capture part of the dynamics when something is actually used by millions of people, and in many cases intuition turns out to be the polar opposite of the reality the data tells you.
It's interesting you mention your reasoning for wanting to add the functionality. I believe this falls under the "Better Than Reality: A Fundamental Internet Principle"[1]. This is such a common pitfall in the web that I feel like it needs to be reiterated over and over. That's not to say your effort wasn't worth it - you achieved your goal and that's important.
http://www.useit.com/articles/better-than-reality-a-fundamen... (note - this is from 1998 and is still relevant!)
http://www.useit.com/articles/better-than-reality-a-fundamen... (note - this is from 1998 and is still relevant!)
Infinite scroll really annoys me on most sites where you're searching to find something (vs carefully reading each and every result). It's frustrating to not really have any sort of "progress" indicator.
For example, try "scrolling" this list of acrylic sheets on inventables: https://www.inventables.com/categories/laser-cutting/acrylic...
Makes me want to find whomever implemented the system and shoot^H^H^H^H^H err explain to them how unusable it is.
For example, try "scrolling" this list of acrylic sheets on inventables: https://www.inventables.com/categories/laser-cutting/acrylic...
Makes me want to find whomever implemented the system and shoot^H^H^H^H^H err explain to them how unusable it is.
I don't understand why infinite scroll implementations don't precalculate how long the page will be when all results have loaded. This would allow the scroll bar to actually indicate how far through the results you are(I'm thinking etsy not google images where they could keep going for days...). The second issue I have with them is that they wait until you scroll to the bottom of the page before more results are loaded. Why not just keep filling the results out until they're all loaded, that way the user wouldn't experience the loading glitch whenever they scroll down. For bonus points they could allow you to scroll to any point in the results before they're loaded and have the items at that point loaded while you're watching.
Infinite scroll annoys me, period. I spend a lot of my time on 3G mobile, and data costs me money. When I scroll down to the bottom of the page, it's because I want to read what's there, not because I want to give my mobile provider more money. When we're all on flat-rate, unrestricted hyperspeed connections, maybe. (I wait with bated breath.)
Infinite scroll annoys me, period, too, but for entirely different reasons (ones that aren't going to change with improving network infrastructure, more related to ComputerGuru's reasons).
When I start browsing the sort of page that web developers are prone to applying infinite scroll to, I usually begin with a sort of semi-subconscious assumption that I'll just read down to the end of the page, or just the first N pages (for some smallish N) or something. I start doing so, but for a while don't realize that I'm never going to reach that point because the page is continually growing as I read it, and thus lose my way of quantifying (admittedly very vaguely) how much time I've wasted^W spent on it and where I should stop. My usual solution is to hit the "end" key (or its moral equivalent) and scroll up from there, ignoring whatever loads beyond that point.
Now from the developers point of view this may be a good way to get me to spend more time on their page, but if they're actually "gaming" me like that, it really just makes it enormously more obnoxious than it already was to begin with. Gaming intended or not, it strongly disinclines me to continue visiting the site in the future.
When I start browsing the sort of page that web developers are prone to applying infinite scroll to, I usually begin with a sort of semi-subconscious assumption that I'll just read down to the end of the page, or just the first N pages (for some smallish N) or something. I start doing so, but for a while don't realize that I'm never going to reach that point because the page is continually growing as I read it, and thus lose my way of quantifying (admittedly very vaguely) how much time I've wasted^W spent on it and where I should stop. My usual solution is to hit the "end" key (or its moral equivalent) and scroll up from there, ignoring whatever loads beyond that point.
Now from the developers point of view this may be a good way to get me to spend more time on their page, but if they're actually "gaming" me like that, it really just makes it enormously more obnoxious than it already was to begin with. Gaming intended or not, it strongly disinclines me to continue visiting the site in the future.
It annoys me too. In some website you can't see the footer, read it and then click its links. It keeps disappearing while I scroll down as fast as I can (keyboard or mouse).
Infinite scroll with a footer is a clear anti-pattern. If a site has both, they don't have a coherent idea of their UX.
Are there any sites that were designed from the start with infinite scroll that also have footers?
Are there any sites that were designed from the start with infinite scroll that also have footers?
> Users want more results per page.
For me, I cannot use Etsy because there are simply too many products. I have no way to narrow down the volume of products to something which anywhere near approaches my ability to make a choice.
(At least with Amazon I can filter by department, then filter by four stars and higher...)
For me, I cannot use Etsy because there are simply too many products. I have no way to narrow down the volume of products to something which anywhere near approaches my ability to make a choice.
(At least with Amazon I can filter by department, then filter by four stars and higher...)
Here's why I hate infinite scroll:
- the little scroll bar changes sizes randomly and moves places in the browser. Users can't predict this or know where it is, so they have to keep hunting for it when stuff loads in the background.
- if I browse a lot of items, scroll down a ways and then decide to scroll back up, it is much harder to find items that otherwise I would remember as being on page 3.
- The scrolling gets jagged as my browser barfs trying to shove more things in the list. In other words, even if it's faster it FEELS slower and less responsive.
- the little scroll bar changes sizes randomly and moves places in the browser. Users can't predict this or know where it is, so they have to keep hunting for it when stuff loads in the background.
- if I browse a lot of items, scroll down a ways and then decide to scroll back up, it is much harder to find items that otherwise I would remember as being on page 3.
- The scrolling gets jagged as my browser barfs trying to shove more things in the list. In other words, even if it's faster it FEELS slower and less responsive.
- also, after a few loads I am a million miles away from the top of the page which is really bad if I want to go back up to the search box or anything else at the top of the page.
I think it's a bad idea to look at what Google does with their search results and then try to apply it to product search.
Users have a very different set of goals, with Google we know that the falloff from the top search result to the bottom is huge, people expect that the relevant results are the top couple.
With product search there's different goals, for example browsing an minimal set of criteria looking for a results--you're looking a pair of jeans and starting with 36x32; or the opposite you just want to know the price of a pair of New Balance 990s in a 11 4E.
In the latter case infinite scroll or not doesn't come into play, but the browsing case it does, and from the point of view as an Etsy customer, I have the most difficulties.
One of the things I've found the most is people do better with a task if they can understand how long it's going to take. If you know there's 100 search results over 5 pages, you can decide if that's too much to go through. As an aside, this is true of a lot of things, you'll do better with pull-ups if I tell you to do 20 and give you a count, than just tell you to keep going until I say "stop".
With infinite scroll you've got no idea how much effort looking through the total search results is going to take. Etsy don't make things easy, as an example Art > Custom Portraits[1]. I have no idea how many results there are, I can scroll all the way to the bottom and find out there are at least 8 pages, but that's it.
The search results themselves are pretty snappy, so I don't see a huge advantage of infinite scrolling. I do think the search results are pretty bad and the lack of filtering is a problem. From Art > Custom Portraits I can filter on just Pets or Silhouettes or More (which I assume is everything else). It's not obvious what these links do either, they are just floating at the top.
Some better filtering, e.g. by price, would help. I'm not sure what other metadata Etsy have that would be useful for an great faceted search.
As an aside, I've been working on (product) search for over a decade and I'm close by the Etsy offices; over a coffee I'd be happy to discuss search. Part of my New Years resolution is to do a better job of networking, so I'm happy to extend this offer to any other NYC based e-commerce shops.
[1] http://www.etsy.com/browse/art/custom-portraits?ref=br_nav_n...
Users have a very different set of goals, with Google we know that the falloff from the top search result to the bottom is huge, people expect that the relevant results are the top couple.
With product search there's different goals, for example browsing an minimal set of criteria looking for a results--you're looking a pair of jeans and starting with 36x32; or the opposite you just want to know the price of a pair of New Balance 990s in a 11 4E.
In the latter case infinite scroll or not doesn't come into play, but the browsing case it does, and from the point of view as an Etsy customer, I have the most difficulties.
One of the things I've found the most is people do better with a task if they can understand how long it's going to take. If you know there's 100 search results over 5 pages, you can decide if that's too much to go through. As an aside, this is true of a lot of things, you'll do better with pull-ups if I tell you to do 20 and give you a count, than just tell you to keep going until I say "stop".
With infinite scroll you've got no idea how much effort looking through the total search results is going to take. Etsy don't make things easy, as an example Art > Custom Portraits[1]. I have no idea how many results there are, I can scroll all the way to the bottom and find out there are at least 8 pages, but that's it.
The search results themselves are pretty snappy, so I don't see a huge advantage of infinite scrolling. I do think the search results are pretty bad and the lack of filtering is a problem. From Art > Custom Portraits I can filter on just Pets or Silhouettes or More (which I assume is everything else). It's not obvious what these links do either, they are just floating at the top.
Some better filtering, e.g. by price, would help. I'm not sure what other metadata Etsy have that would be useful for an great faceted search.
As an aside, I've been working on (product) search for over a decade and I'm close by the Etsy offices; over a coffee I'd be happy to discuss search. Part of my New Years resolution is to do a better job of networking, so I'm happy to extend this offer to any other NYC based e-commerce shops.
[1] http://www.etsy.com/browse/art/custom-portraits?ref=br_nav_n...
Hi! I actually worked on that exact feature. I definitely know what you mean about lack of refinement options on that page, though you can refine (roughly) by price. Also, email's in my profile, so hit me up if you wanna swing by the office.
Just a heads up. Your email is not visible in your profile. The "email" field isn't visible to other people. If you want us to be able to see you email, you need to add it to the "about" section.
I remember hearing Aza's push for infinite scroll on search results... and that guy is wicked smart, but I feel like it only works if we assume I'm already engaging the scroll bar.
Clicking is so much faster than dragging, (barring terrible page load times)... a part of me wishes no one ever invented scrolling.
Clicking is so much faster than dragging, (barring terrible page load times)... a part of me wishes no one ever invented scrolling.
I've come across similar issues and I thought I coined the term "monolithic testing" but clearly great minds think alike.
I'm the biggest proponent of avoiding monolithic tests and having clear and testable hypotheses. I'm glad there's a high profile example to point to now. Thanks.
I'm the biggest proponent of avoiding monolithic tests and having clear and testable hypotheses. I'm glad there's a high profile example to point to now. Thanks.
I don't get all the hate towards infinite scroll. Personally, I like it!
---
Q: Hey John Doe, check out page 3 at site.com, I totally want to buy item 3 and 6.
A: Obviously by the time John Doe checks the site/link item 3 and 6 might be something entirely different. People send direct links to items.
---
Q: Oh no, I don't know where I am in the scroll list.
A: Who gives a rats ass? Your content is right there. Interesting items you found along they you open in a new tab so that you don't have to backtrack.
---
Q: But I want to quick jump to page 45?
A: Why 45? You just pulled that number from your hat (read: ass) anyway. What you really should be looking for is the search field.
---
Q: But I never get to see the footer!
A: The what...? I don't think I've looked at a site footer since '94. Navigation is at the top, and bullshit at the bottom.
---
I just don't get peoples need to "know where they are", like it made a difference?
---
Q: Hey John Doe, check out page 3 at site.com, I totally want to buy item 3 and 6.
A: Obviously by the time John Doe checks the site/link item 3 and 6 might be something entirely different. People send direct links to items.
---
Q: Oh no, I don't know where I am in the scroll list.
A: Who gives a rats ass? Your content is right there. Interesting items you found along they you open in a new tab so that you don't have to backtrack.
---
Q: But I want to quick jump to page 45?
A: Why 45? You just pulled that number from your hat (read: ass) anyway. What you really should be looking for is the search field.
---
Q: But I never get to see the footer!
A: The what...? I don't think I've looked at a site footer since '94. Navigation is at the top, and bullshit at the bottom.
---
I just don't get peoples need to "know where they are", like it made a difference?
Here's a theory:
1. People only care about the first few search results.
2. Infinite scroll exists solely to make it easier to scroll past the initial view.
3. Therefore, infinite scroll made it easier to scroll away from the results people care about.
There you go.
By the way, I don't get why Google's "instant results as you type" was cited as a reason to pursue this infinite scroll feature. Those are totally different things.
"Instant results" is for faster display of the first few results, which is great for search. "Infinite scroll" is for scrolling through long streams of information -- great for newsfeeds and timelines, but not for search results where you only care about the first few.
Google doesn't even implement infinite scroll in their results.
1. People only care about the first few search results.
2. Infinite scroll exists solely to make it easier to scroll past the initial view.
3. Therefore, infinite scroll made it easier to scroll away from the results people care about.
There you go.
By the way, I don't get why Google's "instant results as you type" was cited as a reason to pursue this infinite scroll feature. Those are totally different things.
"Instant results" is for faster display of the first few results, which is great for search. "Infinite scroll" is for scrolling through long streams of information -- great for newsfeeds and timelines, but not for search results where you only care about the first few.
Google doesn't even implement infinite scroll in their results.
This reminds me of the age-old advice of not overwhelming customers with options. I.e., if you know the customer wants your product (say, a cell phone), you're much better off giving them one or two options to choose from (think iPhone). If you overwhelm them with 14 different models each a little different in some small (or big) way, you're more likely to lose the sale altogether.
This concept plays off the simple fact people are fundamentally bad at choosing between many options.
I'm curious if this is the factor that caused negative results with the infinite scroll. I'm also curious what would happen if you started only returning 5 results/search...
This concept plays off the simple fact people are fundamentally bad at choosing between many options.
I'm curious if this is the factor that caused negative results with the infinite scroll. I'm also curious what would happen if you started only returning 5 results/search...
My guess it has something to do with the paradox of choice. Seeing more options makes it harder to decide what to look at more closely, let alone to make decision to buy.
Infinite scroll is a good solution, if you are building lobster traps - sites where you want people to spend endless amounts of time. (http://gizmodo.com/5586337/pandas-and-lobsters-why-google-ca...). It clearly works for Facebook, Twitter and Quora.
Although Etsy has some aspects of lobster trap, it still is more transactional than FB and other social time killers.
Infinite scroll is a good solution, if you are building lobster traps - sites where you want people to spend endless amounts of time. (http://gizmodo.com/5586337/pandas-and-lobsters-why-google-ca...). It clearly works for Facebook, Twitter and Quora.
Although Etsy has some aspects of lobster trap, it still is more transactional than FB and other social time killers.
Wordpressdotcom added Infinite Scroll to blogs. I turned it off but it still has it when you want to see older posts. How much I hate it! I spend more time staring at a damn spinning wheel than seeing more posts when Search won't bring up what I need (due to not recalling an exact keyword). I wonder if this was the case with Etsy too? The only Infinite Scroll I've seen that works properly has been with Twitter. Even Bloglovin (which I use for RSS) has a one-second or so stutter that can be very annoying.
Here's a dissenting opinion: I hate pagination. Especially when I need to browse through a lot of content. When I scan for something I hate being interrupted every 10 seconds by having to click a tiny pagination target, especially if that target jumps around from page to page. I understand that most infinite scrolling implementations leave out critical functionality like back button support, but that doesn't mean infinite scrolling can't be done well.
"View all". Problem solved. Or "View first X00 items", where X is digit (if you're worried about performance).
Just like many people here, I hate infinite scrolling with passion.
Just like many people here, I hate infinite scrolling with passion.
Judging by the reasoning people are providing, I only see problems with poor implementation of infinite scrolling, not infinite scrolling itself, solved by better implementation (like your "View all" suggestion).
In truth, your view all suggestion as a way to fix pagination is akin to me making a paginate button to solve infinite scrolling.
In truth, your view all suggestion as a way to fix pagination is akin to me making a paginate button to solve infinite scrolling.
We (a friend and I) built a simple application[1] that gave folks relevant clothing for the weather in their location.
We shelved the infinite scroll at the time out of haste.
It turned out to be a happy accident, as active users refreshing would generate a ton of pageviews.
Not only that, but the act of clicking refresh to "get more" is a nice way to get the user to engage.
[1] http://wevther.com
We shelved the infinite scroll at the time out of haste.
It turned out to be a happy accident, as active users refreshing would generate a ton of pageviews.
Not only that, but the act of clicking refresh to "get more" is a nice way to get the user to engage.
[1] http://wevther.com
Someone could point out that Etsy's search is relatively poor, which was only made worse by infinite scroll.
I tried to buy something that I knew was on Etsy during the holidays, and after about an hour of looking (their category breakdown reminds me of reading through the yellow pages - remember those?) I gave up and bought from Amazon instead.
I tried to buy something that I knew was on Etsy during the holidays, and after about an hour of looking (their category breakdown reminds me of reading through the yellow pages - remember those?) I gave up and bought from Amazon instead.
I wonder why this failed? I've seen infinite scroll work on some major websites, Facebook and Twitter, but then perhaps it's related to search only? FB and Twitter are loading a feed, a search page should give users what they expect to see, not randomly see more elements pop on screen, perhaps?
Seems like infinite scroll is a solution in search of a problem. Users don't want to see more results. They want to see the right results. Etsy seems to fail in this regard and has dozens of pages of irrelevant results. A better move would be to rethink search relevancy and discovery.
This is a false dilemma and while Etsy's search engine can always improve a massive amount of work has been done over the last several years to improve the relevancy of results. The vast majority of these have been small wins and the large net effect would not have been possible without the methodology Dan outlined in his talk.
Infinite scroll + History API is sexy, I'd encourage them to give it another shot, this time without breaking the back button.
Why not do both and allow users to select whichever they prefer? You could then A/B test which is the better default option.
There is cost associated with keeping code alive. It is magnified when that code will have follow-on effects for how other code (search results format, etc) gets designed.
If the code is not providing real value, it should be thrown out.
If the code is not providing real value, it should be thrown out.
Warning: long comment ahead! Read at your own risk. If you ask me for a tl;dr I'll kick you in the shins.
I'm so happy to see them measuring "median item impressions" rather than "mean item impressions." Many of the underlying variables describing consumer behavior aren't normally distributed. Talking about "average number of friends invited" when 50% of your users invite 0, 25% invite 1, 13% invite 2, etc. but 3 users invite 5,000 will necessarily lead you to propose bad product ideas and decisions.
However, I'm curious about the experimental design. For example, was this tested on new users, old users, or both? Changing a fundamental part of a site's experience like this will have some cost as users acclimate. I'd wager Etsy's audience is less technically inclined, too, so it might take them longer to acclimate.
They also commit a small fallacy when they talk about how they should have done it instead, and IMO it's a fallacy that frequent A/B testing encourages people to commit. They suggest that instead they should have determined first whether more items are better and faster items are better.
On the most surface level, perhaps there's something about more items AND faster items that outperforms either one or the other in isolation. That's easy enough to accomplish, technically. You use different statistical tests, but it's possible at the cost of perhaps a larger sample size.
On a deeper level, you're providing the users with a fairly different overall user experience. Their sense of where things are placed, what they're supposed to do when they want to "see more," how they know they have the opportunity to "see more," etc. are aspects of the infinite scroll design that aren't encapsulated in either rendering more items or rendering those items more quickly.
For example, can users bookmark specific search result pages under the current design? Can they still do the same thing under the "infinite scroll" design? I imagine there are lots of little things like this and that the UX difference alone would have a larger impact on the results than just changing the number of products per page.
To get more meaningful results from this, I'd run this experiment under the following assumptions.
1. Assume that existing users will be more impacted by this change than new users. Therefore the cost of "failure" for existing users is higher.
2. Assume that at the end of the day the #1 thing Etsy cares about is "dollar throughput" of the Etsy platform. Engagement, favoriting, searching, etc. are all positive indicators of an increased dollar velocity.
3. Assume they have information about what aspects of a users' first visit are indicators of their long-term ability to contribute to Etsy's dollar throughput.
4. Assume that eventually every user will have the same experience, new or existing alike.
So, I'd start by running the experiment with new users only. Over the course of a week or a month I'd put a % of the users who joined each day into the "infinite scroll" bucket. I'd then run the study as a longitudinal study.
Assumption (3) can guide us as to whether we need to cut off the experiment early. The length of the study would be determined by the particularities of an Etsy user's life-cycle, e.g., maybe given a cohort of users, we care about the length of time it takes 75% of the eventual purchasers to make their first purchase.
Because of assumption (4) we know that if the "infinite scroll" design is terrible for new users, we never have to bother testing it on existing users.
[1]: Non-technical users, in particular, are sensitive to sudden change. I forget where, but I read a research paper once that implied that the worst thing you can do to harm a person's user experience is change the placement of links, buttons, etc. You can change the color, text, icons, etc. but if you change the placement, they essentially have to "re-learn" the interface.
IIRC, the users were given a task (e.g., "create a document") and they measured two core variables: time to accomplish the given task and time until their "performance" at a given task was equivalent to the control user interface.
Changing the placement of a certain action in the UI had a deeper and longer-lasting impact on users' ability to perform tasks than changing anything else about the UI by a large margin.
I'm so happy to see them measuring "median item impressions" rather than "mean item impressions." Many of the underlying variables describing consumer behavior aren't normally distributed. Talking about "average number of friends invited" when 50% of your users invite 0, 25% invite 1, 13% invite 2, etc. but 3 users invite 5,000 will necessarily lead you to propose bad product ideas and decisions.
However, I'm curious about the experimental design. For example, was this tested on new users, old users, or both? Changing a fundamental part of a site's experience like this will have some cost as users acclimate. I'd wager Etsy's audience is less technically inclined, too, so it might take them longer to acclimate.
They also commit a small fallacy when they talk about how they should have done it instead, and IMO it's a fallacy that frequent A/B testing encourages people to commit. They suggest that instead they should have determined first whether more items are better and faster items are better.
On the most surface level, perhaps there's something about more items AND faster items that outperforms either one or the other in isolation. That's easy enough to accomplish, technically. You use different statistical tests, but it's possible at the cost of perhaps a larger sample size.
On a deeper level, you're providing the users with a fairly different overall user experience. Their sense of where things are placed, what they're supposed to do when they want to "see more," how they know they have the opportunity to "see more," etc. are aspects of the infinite scroll design that aren't encapsulated in either rendering more items or rendering those items more quickly.
For example, can users bookmark specific search result pages under the current design? Can they still do the same thing under the "infinite scroll" design? I imagine there are lots of little things like this and that the UX difference alone would have a larger impact on the results than just changing the number of products per page.
To get more meaningful results from this, I'd run this experiment under the following assumptions.
1. Assume that existing users will be more impacted by this change than new users. Therefore the cost of "failure" for existing users is higher.
2. Assume that at the end of the day the #1 thing Etsy cares about is "dollar throughput" of the Etsy platform. Engagement, favoriting, searching, etc. are all positive indicators of an increased dollar velocity.
3. Assume they have information about what aspects of a users' first visit are indicators of their long-term ability to contribute to Etsy's dollar throughput.
4. Assume that eventually every user will have the same experience, new or existing alike.
So, I'd start by running the experiment with new users only. Over the course of a week or a month I'd put a % of the users who joined each day into the "infinite scroll" bucket. I'd then run the study as a longitudinal study.
Assumption (3) can guide us as to whether we need to cut off the experiment early. The length of the study would be determined by the particularities of an Etsy user's life-cycle, e.g., maybe given a cohort of users, we care about the length of time it takes 75% of the eventual purchasers to make their first purchase.
Because of assumption (4) we know that if the "infinite scroll" design is terrible for new users, we never have to bother testing it on existing users.
[1]: Non-technical users, in particular, are sensitive to sudden change. I forget where, but I read a research paper once that implied that the worst thing you can do to harm a person's user experience is change the placement of links, buttons, etc. You can change the color, text, icons, etc. but if you change the placement, they essentially have to "re-learn" the interface.
IIRC, the users were given a task (e.g., "create a document") and they measured two core variables: time to accomplish the given task and time until their "performance" at a given task was equivalent to the control user interface.
Changing the placement of a certain action in the UI had a deeper and longer-lasting impact on users' ability to perform tasks than changing anything else about the UI by a large margin.
I got the impression from the article that their experiment had very little planning from the statement about it being to prove it was good and then celebrate. This probably means it was a straight X% of traffic A/B test and the results were only analysed deeply when it wasn't a positive result. This is speculation only based on the tone of the article.
This is the tongue-in-cheek slide that McKinley used:
http://image.slidesharecdn.com/continuous-experimentation-12...
I didn't rewatch the entire presentation but McKinley does discuss the user makeup in the experimental groups...yes they do account for different kinds of users, and the most drastic difference between user behavior are between sellers and non-sellers. Someone from Etsy would have to talk about how much slicing-and-dicing of the demographic that they do...but even if infinite scroll was good for some users (new users without pagination-related habits) and not for others, it's probably not a good idea to have two kinds of search experience in the hopes that the "oldies" eventually figure it out...based solely on how hard it is to implement infinite-scroll in the technical sense.
http://image.slidesharecdn.com/continuous-experimentation-12...
I didn't rewatch the entire presentation but McKinley does discuss the user makeup in the experimental groups...yes they do account for different kinds of users, and the most drastic difference between user behavior are between sellers and non-sellers. Someone from Etsy would have to talk about how much slicing-and-dicing of the demographic that they do...but even if infinite scroll was good for some users (new users without pagination-related habits) and not for others, it's probably not a good idea to have two kinds of search experience in the hopes that the "oldies" eventually figure it out...based solely on how hard it is to implement infinite-scroll in the technical sense.
It's pretty common for us to look at new vs. returning users and Etsy sellers vs. others (sellers are obviously really engaged users, and behave differently). Occasionally one group will stand out, but not in this case.
The article mentions that infinite scroll is "prominent among popular Tumblr themes, Pinterest, and of course, at Facebook and Twitter". All of these are content consumption sites, and for at least Pinterest, Facebook, and Twitter, clicking on an item in the list opens it in-page -- for Pinterest and Facebook, it's in a modal box; for Twitter, it's expanded within the stream. The content you're browsing is all there; there's no problem with clicking on an item in the list to get a more detailed view.
At Etsy, introducing infinite scroll resulted in "fewer clicks on the results and fewer items 'favorited' from the infinite results page". On an Etsy search results page [1], clicking on an item bounces you to another page. And on sites with "infinite scroll", this is typically a very uncomfortable experience, particularly when trying to get back. Depending on the implementation, you're wind up back at the beginning of the result set; even if you don't, it's usually a fairly bumpy ride, with the time taken for the page to reload its data and the jumps in scrolling as everything loads in. Even if it technically works, the kind of sensation this brings about is enough to discourage someone from actually clicking through. There's an negative association that develops with clicking on these items, the foreboding feeling that you'll end up losing your place, such that one tries to do so as little as possible -- in line with what was observed from the Etsy experiment.
On the Etsy search page, you can indeed "favorite" items without going into the item's actual page. But that's not something one is likely to do based on a tiny little thumbnail -- one would usually first click through, see a bigger picture, and possibly read the description below. It doesn't help that, on the results page, the "favorite" button is but a tiny little icon, that only appears if you directly mouse over the thumbnail; meanwhile, on an actual item page, it's right there under "Add to Cart" [2]. Perhaps users weren't even aware that you could do this from the search page.
With a site like Etsy, where lists are a means to an end -- a way to get to information on other pages -- it's no surprise that infinite scroll performs quite poorly, as opposed to content consumption sites, where browsing is a self-contained experience of its own.
(As an aside, it's fairly silly to compare infinite scroll with Google's Instant Search. Instant is well-liked because it gets you to your search results faster; this being Google, the user isn't there to hang around and enjoy the scenery, but to get to the information they're looking for as quickly as possible. And Google's results pages themselves still use pagination, despite their experimenting with infinite scrolling back in 2011 [3] -- a change that, quite clearly, didn't make the cut.)
[1] http://www.etsy.com/search?q=test&view_type=gallery&...
[2] http://www.etsy.com/listing/65114535/12-boracilicate-glass-t...
[3] http://searchenginewatch.com/article/2103479/Google-Experime...
At Etsy, introducing infinite scroll resulted in "fewer clicks on the results and fewer items 'favorited' from the infinite results page". On an Etsy search results page [1], clicking on an item bounces you to another page. And on sites with "infinite scroll", this is typically a very uncomfortable experience, particularly when trying to get back. Depending on the implementation, you're wind up back at the beginning of the result set; even if you don't, it's usually a fairly bumpy ride, with the time taken for the page to reload its data and the jumps in scrolling as everything loads in. Even if it technically works, the kind of sensation this brings about is enough to discourage someone from actually clicking through. There's an negative association that develops with clicking on these items, the foreboding feeling that you'll end up losing your place, such that one tries to do so as little as possible -- in line with what was observed from the Etsy experiment.
On the Etsy search page, you can indeed "favorite" items without going into the item's actual page. But that's not something one is likely to do based on a tiny little thumbnail -- one would usually first click through, see a bigger picture, and possibly read the description below. It doesn't help that, on the results page, the "favorite" button is but a tiny little icon, that only appears if you directly mouse over the thumbnail; meanwhile, on an actual item page, it's right there under "Add to Cart" [2]. Perhaps users weren't even aware that you could do this from the search page.
With a site like Etsy, where lists are a means to an end -- a way to get to information on other pages -- it's no surprise that infinite scroll performs quite poorly, as opposed to content consumption sites, where browsing is a self-contained experience of its own.
(As an aside, it's fairly silly to compare infinite scroll with Google's Instant Search. Instant is well-liked because it gets you to your search results faster; this being Google, the user isn't there to hang around and enjoy the scenery, but to get to the information they're looking for as quickly as possible. And Google's results pages themselves still use pagination, despite their experimenting with infinite scrolling back in 2011 [3] -- a change that, quite clearly, didn't make the cut.)
[1] http://www.etsy.com/search?q=test&view_type=gallery&...
[2] http://www.etsy.com/listing/65114535/12-boracilicate-glass-t...
[3] http://searchenginewatch.com/article/2103479/Google-Experime...
I like knowing:
* how many items are in the results
* how many pages there are (a derivative of the first, sure, but helpful to know)
* where I was if I accidentally close the tab and re-open it, or follow a link and need to hit the back button, or share my position with a friend (yes, most paginated results aren't stable in the long run, but usually are in the short run)
* easily jump to either end of the list (or, near to the end of the list, e.g. < 1, 2, 3, ... 98, 99, 100 >)
For example, I follow way too many people on twitter (1700!). I know I'm getting tweets throttled so I miss out on people I care about.
I also know that when I first picked up twitter years ago I made some noob mistakes and followed things I, well, shouldn't have (@Tide? I think a friend was working at P&G or something...).
Anyway, "People you follow" on the web UI is an infinite scroll. Even worse, it's buggy so sometimes you can trigger it to not register you're at the bottom and it won't load. AWESOME. I just want to get to the LAST PAGE. BUT I CAN'T.
And since it seems to load about 10-15 at a time, given that it takes about 1/2 a second to scroll down and wait for it to load, that means it'll take at least 60-85 seconds to reach the bottom - IF it doesn't crash (a reload takes me back to the top). Which means I've never been able to do it.
I had to pay one of those "show the folks you follow that haven't tweeted in n-months" just to try and prune the list, which helped me go from 2,000 to my present number.
Yes, this could be solved by sorting and filtering, but in the truest MVP sort of the world, why do all that extra work just for the "infinite scroll" fad? Switch back to pagination and I could accomplish all I needed and have a nice pruned list. And I bet it's less effort and has far fewer bugs than the current implementation.
I hope that in a few years enough data against infinite scroll will have cemented it as a generally accepted bad idea, only working in a few particular cases.