top | item 5017995

Why did infinite scroll fail at Etsy?

105 points| danso | 13 years ago |danwin.com | reply

62 comments

order
[+] MattRogish|13 years ago|reply
Boy, I really hate infinite scroll. I've never found it enhanced my experience over plain old paginated results for anything product related (sure, maybe it works for Google or HN home page).

I like knowing:

* how many items are in the results

* how many pages there are (a derivative of the first, sure, but helpful to know)

* where I was if I accidentally close the tab and re-open it, or follow a link and need to hit the back button, or share my position with a friend (yes, most paginated results aren't stable in the long run, but usually are in the short run)

* easily jump to either end of the list (or, near to the end of the list, e.g. < 1, 2, 3, ... 98, 99, 100 >)

For example, I follow way too many people on twitter (1700!). I know I'm getting tweets throttled so I miss out on people I care about.

I also know that when I first picked up twitter years ago I made some noob mistakes and followed things I, well, shouldn't have (@Tide? I think a friend was working at P&G or something...).

Anyway, "People you follow" on the web UI is an infinite scroll. Even worse, it's buggy so sometimes you can trigger it to not register you're at the bottom and it won't load. AWESOME. I just want to get to the LAST PAGE. BUT I CAN'T.

And since it seems to load about 10-15 at a time, given that it takes about 1/2 a second to scroll down and wait for it to load, that means it'll take at least 60-85 seconds to reach the bottom - IF it doesn't crash (a reload takes me back to the top). Which means I've never been able to do it.

I had to pay one of those "show the folks you follow that haven't tweeted in n-months" just to try and prune the list, which helped me go from 2,000 to my present number.

Yes, this could be solved by sorting and filtering, but in the truest MVP sort of the world, why do all that extra work just for the "infinite scroll" fad? Switch back to pagination and I could accomplish all I needed and have a nice pruned list. And I bet it's less effort and has far fewer bugs than the current implementation.

I hope that in a few years enough data against infinite scroll will have cemented it as a generally accepted bad idea, only working in a few particular cases.

[+] firefoxman1|13 years ago|reply
I only enjoy infinite scroll on touch screens. I think the perfect solution would be paginated results, but if the user ontouchstart pulls the page past the bottom, it would automatically load the next page (like reverse of the facebook app's pull-down for new items). It's much more convenient than trying to tap just perfectly on the "next" arrow, but paginated is just easier with a mouse.

That mindset of design I think is the next step after "responsive" design. Rather than just scaling the visual design of the page, how about truly designing the user interaction to fit multiple scenarios. If it were up to me, I'd call it Symbiotic Design (responsive 2.0)

[+] scholia|13 years ago|reply
"Infinite scroll" is unutterably stupid and incompetent on Twitter. The inability to skip back x pages just ensures that the vast majority of data -- including tweets and follower details -- can never be accessed. The solution is to use http://manageflitter.com/ or some other service, at least until Twitter finds yet another way to screw up even more of its supporting ecosystem....
[+] oconnore|13 years ago|reply
> But the A/B tests showed various negative effects of the feature, including fewer clicks on the results and fewer items “favorited” from the infinite results page. And curiously, while users didn’t buy fewer items overall, “they just stopped using search to find these items.”

How is this a negative effect? The amount of stuff that users are buying should be pretty much the _only_ metric you care about. And they are probably not using search as much because browsing is much nicer with infinite scroll.

It sounds like they just did infinite scroll wrong.

If you read the comments page on the forum, the biggest complaint is that you lose your place in the scroll when you return to the scroll page. If they had listened to their users and made it so that you return to the spot where you clicked on the item, perhaps it would have been a success.

[+] FuzzyDunlop|13 years ago|reply
When I use these things (and even pagination sometimes), I might find something I like, but it's so easy to browse ahead that I'll skip over it in the hope that I'll find something I like even more further on.

At that point, there's no easy way to get back to what I originally liked unless I had the forethought to open it in a tab or remember it exactly.

As for the metrics, it depends on how the business wants to position itself. It's evident from the quote that they care more about just the sales and would prefer to encourage more engagement with their site.

[+] harlanlewis|13 years ago|reply
> How is this a negative effect? The amount of stuff that users are buying should be pretty much the _only_ metric you care about.

Items ordered is most helpful for making decisions when in context. It's awfully difficult to gauge health or make decisions with in isolation. A superficial short-term return may not be as beneficial on other levels - strategically, operationally, vendor ecosystem...

The article and other commenters have pointed at some, a few more more basic things worth knowing:

- return visit behavior. If infinite scroll diminished visitors favoriting items, what's the relative conversion for visitors returning to a list of favorites vs not? (features like this, Amazon Wish List etc, usually perform pretty well).

- avg order value. Visitors are buying the same number of things, but are they making the same $$ per order? Preserving avg order value but increasing item count also increases operations overhead (which could be worth it to encourage a broader vendor pool...)

- Clicks to items helps Etsy identify interesting/trending items&sellers independent of sales (move your data knowledge upstream!). Also increases the data size on individual users to build a taste profile and surface more items of interest at an individual level

[+] mcfunley|13 years ago|reply
> The amount of stuff that users are buying should be pretty much the _only_ metric you care about.

Generally yes, we would trade an increase purchases for anything else. The test was a negative effect because we forced people to stop using search, which was the feature we were experimenting with. Making buying less efficient, even if people want to buy so much that they eventually succeed, isn't a great result.

> And they are probably not using search as much because browsing is much nicer with infinite scroll.

No for two reasons. First, browse pages didn't have infinite scroll back when this test was done. And second, the point is that people seeing infinite scroll purchased less from search as compared to the control. This was a controlled A/B test.

As gfodor says, yes the back button did return you to your place (or it did, once we ironed out all of the problems with it in early incarnations).

[+] gfodor|13 years ago|reply
I helped out with this project at Etsy and obviously our implementation intended for the back button to work as expected. Our initial rollout worked fine in most browsers but certain older versions of IE were discovered to have issues. Of course, the experiments that Dan referenced in his talk were in regards to known-good implementations that did not have issues with the back button.

Anyone who has worked on infinite scroll knows that the back button is a "interesting" technical challenge.

[+] kaliblack|13 years ago|reply
> The amount of stuff that users are buying should be pretty much the _only_ metric you care about.

I agree if Etsy is a 100% commerce focused, but they aren't. Key to their success has been their ability to build up a community and sell stuff. Even if you weight down the impact of clicks and favourites the effect is still negative with less exploration and interaction happening. What would be the long term impact of sales of this reduction?

[+] karolisd|13 years ago|reply
Infinite scroll that remembers your place vs infinite scroll that doesn't. Now that would be an interesting test. Although I think I know which one would win.
[+] wonderyak|13 years ago|reply
I have this problem frequently on many sites; whether its from a UI deficiency or my own memory error. I have trained myself to always open listed/grid links in a new tab as a result.

When building sites now, I usually contemplate using target=_blank on any lists just to aid this kind of pain point.

[+] danso|13 years ago|reply
That's a good point, about whether it was really a failure if those users still bought things...certainly it didn't seem like a benefit to search at all...but maybe most people just end up buying through other avenues anyway, such as following a shop

Also to your point, Dan talked about how they also thought that opening results in new tabs would be a sure fire winner (because power users open up multiple tabs to do comparisons), but it was also a significant negative effect. So clearly Etsy users back up when venturing into results and apparently expect to be at the exact same state they were before clicking through

[+] georgemcbay|13 years ago|reply
Contrary to Dan McKinley's half-apology as quoted in the article, infinite scroll is stupid on your website.

Infinite scroll flies in the face of the way the human brain works with groups and sets, makes it virtually impossible to usefully search within the current page using the browser's search feature, etc. And the positives are? Nothing, other than novelty.

It is one of the many recent examples of webdev/designers doing something because it is possible and trendy and new rather than because it adds any value.

Paged results with a well designed indication of where you are plus good server-side categorization and server-side search to filter results is far preferable to infinite scroll in every practical situation.

Keep the infinite scroll for those purely arty non-commercial story-telling sites, if you make me try to use an infinite scroll interface to buy stuff from you I will buy nothing.

[+] gfodor|13 years ago|reply
Of course your opinion may or may not be right but the point is to figure out a way to test it. The reason infinite scroll was attractive at Etsy, among other reasons, is because it is reminiscent of the experience one has at a large craft fair where you quickly scan through hundreds of items on tables looking for one that jumps out as worth zooming in on.
[+] ComputerGuru|13 years ago|reply
Infinite scroll really annoys me on most sites where you're searching to find something (vs carefully reading each and every result). It's frustrating to not really have any sort of "progress" indicator.

For example, try "scrolling" this list of acrylic sheets on inventables: https://www.inventables.com/categories/laser-cutting/acrylic...

Makes me want to find whomever implemented the system and shoot^H^H^H^H^H err explain to them how unusable it is.

[+] ollysb|13 years ago|reply
I don't understand why infinite scroll implementations don't precalculate how long the page will be when all results have loaded. This would allow the scroll bar to actually indicate how far through the results you are(I'm thinking etsy not google images where they could keep going for days...). The second issue I have with them is that they wait until you scroll to the bottom of the page before more results are loaded. Why not just keep filling the results out until they're all loaded, that way the user wouldn't experience the loading glitch whenever they scroll down. For bonus points they could allow you to scroll to any point in the results before they're loaded and have the items at that point loaded while you're watching.
[+] stan_rogers|13 years ago|reply
Infinite scroll annoys me, period. I spend a lot of my time on 3G mobile, and data costs me money. When I scroll down to the bottom of the page, it's because I want to read what's there, not because I want to give my mobile provider more money. When we're all on flat-rate, unrestricted hyperspeed connections, maybe. (I wait with bated breath.)
[+] pierrend|13 years ago|reply
It annoys me too. In some website you can't see the footer, read it and then click its links. It keeps disappearing while I scroll down as fast as I can (keyboard or mouse).
[+] nodata|13 years ago|reply
> Users want more results per page.

For me, I cannot use Etsy because there are simply too many products. I have no way to narrow down the volume of products to something which anywhere near approaches my ability to make a choice.

(At least with Amazon I can filter by department, then filter by four stars and higher...)

[+] lubujackson|13 years ago|reply
Here's why I hate infinite scroll:

- the little scroll bar changes sizes randomly and moves places in the browser. Users can't predict this or know where it is, so they have to keep hunting for it when stuff loads in the background.

- if I browse a lot of items, scroll down a ways and then decide to scroll back up, it is much harder to find items that otherwise I would remember as being on page 3.

- The scrolling gets jagged as my browser barfs trying to shove more things in the list. In other words, even if it's faster it FEELS slower and less responsive.

[+] lubujackson|13 years ago|reply
- also, after a few loads I am a million miles away from the top of the page which is really bad if I want to go back up to the search box or anything else at the top of the page.
[+] gav|13 years ago|reply
I think it's a bad idea to look at what Google does with their search results and then try to apply it to product search.

Users have a very different set of goals, with Google we know that the falloff from the top search result to the bottom is huge, people expect that the relevant results are the top couple.

With product search there's different goals, for example browsing an minimal set of criteria looking for a results--you're looking a pair of jeans and starting with 36x32; or the opposite you just want to know the price of a pair of New Balance 990s in a 11 4E.

In the latter case infinite scroll or not doesn't come into play, but the browsing case it does, and from the point of view as an Etsy customer, I have the most difficulties.

One of the things I've found the most is people do better with a task if they can understand how long it's going to take. If you know there's 100 search results over 5 pages, you can decide if that's too much to go through. As an aside, this is true of a lot of things, you'll do better with pull-ups if I tell you to do 20 and give you a count, than just tell you to keep going until I say "stop".

With infinite scroll you've got no idea how much effort looking through the total search results is going to take. Etsy don't make things easy, as an example Art > Custom Portraits[1]. I have no idea how many results there are, I can scroll all the way to the bottom and find out there are at least 8 pages, but that's it.

The search results themselves are pretty snappy, so I don't see a huge advantage of infinite scrolling. I do think the search results are pretty bad and the lack of filtering is a problem. From Art > Custom Portraits I can filter on just Pets or Silhouettes or More (which I assume is everything else). It's not obvious what these links do either, they are just floating at the top.

Some better filtering, e.g. by price, would help. I'm not sure what other metadata Etsy have that would be useful for an great faceted search.

As an aside, I've been working on (product) search for over a decade and I'm close by the Etsy offices; over a coffee I'd be happy to discuss search. Part of my New Years resolution is to do a better job of networking, so I'm happy to extend this offer to any other NYC based e-commerce shops.

[1] http://www.etsy.com/browse/art/custom-portraits?ref=br_nav_n...

[+] elimgoodman|13 years ago|reply
Hi! I actually worked on that exact feature. I definitely know what you mean about lack of refinement options on that page, though you can refine (roughly) by price. Also, email's in my profile, so hit me up if you wanna swing by the office.
[+] brownbat|13 years ago|reply
I remember hearing Aza's push for infinite scroll on search results... and that guy is wicked smart, but I feel like it only works if we assume I'm already engaging the scroll bar.

Clicking is so much faster than dragging, (barring terrible page load times)... a part of me wishes no one ever invented scrolling.

[+] karolisd|13 years ago|reply
I've come across similar issues and I thought I coined the term "monolithic testing" but clearly great minds think alike.

I'm the biggest proponent of avoiding monolithic tests and having clear and testable hypotheses. I'm glad there's a high profile example to point to now. Thanks.

[+] Spiritus|13 years ago|reply
I don't get all the hate towards infinite scroll. Personally, I like it!

---

Q: Hey John Doe, check out page 3 at site.com, I totally want to buy item 3 and 6.

A: Obviously by the time John Doe checks the site/link item 3 and 6 might be something entirely different. People send direct links to items.

---

Q: Oh no, I don't know where I am in the scroll list.

A: Who gives a rats ass? Your content is right there. Interesting items you found along they you open in a new tab so that you don't have to backtrack.

---

Q: But I want to quick jump to page 45?

A: Why 45? You just pulled that number from your hat (read: ass) anyway. What you really should be looking for is the search field.

---

Q: But I never get to see the footer!

A: The what...? I don't think I've looked at a site footer since '94. Navigation is at the top, and bullshit at the bottom.

---

I just don't get peoples need to "know where they are", like it made a difference?

[+] abalone|13 years ago|reply
Here's a theory:

1. People only care about the first few search results.

2. Infinite scroll exists solely to make it easier to scroll past the initial view.

3. Therefore, infinite scroll made it easier to scroll away from the results people care about.

There you go.

By the way, I don't get why Google's "instant results as you type" was cited as a reason to pursue this infinite scroll feature. Those are totally different things.

"Instant results" is for faster display of the first few results, which is great for search. "Infinite scroll" is for scrolling through long streams of information -- great for newsfeeds and timelines, but not for search results where you only care about the first few.

Google doesn't even implement infinite scroll in their results.

[+] tobyjsullivan|13 years ago|reply
This reminds me of the age-old advice of not overwhelming customers with options. I.e., if you know the customer wants your product (say, a cell phone), you're much better off giving them one or two options to choose from (think iPhone). If you overwhelm them with 14 different models each a little different in some small (or big) way, you're more likely to lose the sale altogether.

This concept plays off the simple fact people are fundamentally bad at choosing between many options.

I'm curious if this is the factor that caused negative results with the infinite scroll. I'm also curious what would happen if you started only returning 5 results/search...

[+] dirtyaura|13 years ago|reply
My guess it has something to do with the paradox of choice. Seeing more options makes it harder to decide what to look at more closely, let alone to make decision to buy.

Infinite scroll is a good solution, if you are building lobster traps - sites where you want people to spend endless amounts of time. (http://gizmodo.com/5586337/pandas-and-lobsters-why-google-ca...). It clearly works for Facebook, Twitter and Quora.

Although Etsy has some aspects of lobster trap, it still is more transactional than FB and other social time killers.

[+] mikecane|13 years ago|reply
Wordpressdotcom added Infinite Scroll to blogs. I turned it off but it still has it when you want to see older posts. How much I hate it! I spend more time staring at a damn spinning wheel than seeing more posts when Search won't bring up what I need (due to not recalling an exact keyword). I wonder if this was the case with Etsy too? The only Infinite Scroll I've seen that works properly has been with Twitter. Even Bloglovin (which I use for RSS) has a one-second or so stutter that can be very annoying.
[+] marknutter|13 years ago|reply
Here's a dissenting opinion: I hate pagination. Especially when I need to browse through a lot of content. When I scan for something I hate being interrupted every 10 seconds by having to click a tiny pagination target, especially if that target jumps around from page to page. I understand that most infinite scrolling implementations leave out critical functionality like back button support, but that doesn't mean infinite scrolling can't be done well.
[+] epsylon|13 years ago|reply
"View all". Problem solved. Or "View first X00 items", where X is digit (if you're worried about performance).

Just like many people here, I hate infinite scrolling with passion.

[+] pearkes|13 years ago|reply
We (a friend and I) built a simple application[1] that gave folks relevant clothing for the weather in their location.

We shelved the infinite scroll at the time out of haste.

It turned out to be a happy accident, as active users refreshing would generate a ton of pageviews.

Not only that, but the act of clicking refresh to "get more" is a nice way to get the user to engage.

[1] http://wevther.com

[+] debacle|13 years ago|reply
Someone could point out that Etsy's search is relatively poor, which was only made worse by infinite scroll.

I tried to buy something that I knew was on Etsy during the holidays, and after about an hour of looking (their category breakdown reminds me of reading through the yellow pages - remember those?) I gave up and bought from Amazon instead.

[+] jbrooksuk|13 years ago|reply
I wonder why this failed? I've seen infinite scroll work on some major websites, Facebook and Twitter, but then perhaps it's related to search only? FB and Twitter are loading a feed, a search page should give users what they expect to see, not randomly see more elements pop on screen, perhaps?
[+] chadyj|13 years ago|reply
Seems like infinite scroll is a solution in search of a problem. Users don't want to see more results. They want to see the right results. Etsy seems to fail in this regard and has dozens of pages of irrelevant results. A better move would be to rethink search relevancy and discovery.
[+] gfodor|13 years ago|reply
This is a false dilemma and while Etsy's search engine can always improve a massive amount of work has been done over the last several years to improve the relevancy of results. The vast majority of these have been small wins and the large net effect would not have been possible without the methodology Dan outlined in his talk.
[+] Benferhat|13 years ago|reply
Infinite scroll + History API is sexy, I'd encourage them to give it another shot, this time without breaking the back button.