Google Doesn't Crawl Content Behind JavaScript Load More Buttons. Infinite Scroll Not Much Better.

Google’s John Mueller has confirmed in a Webmaster Hangout that they no longer crawl content behind JavaScript “load more” buttons. Also, Mueller confirmed that they might not crawl all content loaded via the infinite scroll technique either.

While pagination is the most popular way to manage content on home, category, and category product pages, two other methods have seen increased usage over the past few years:

• JavaScript “load more” buttons - This is where, instead of pagination, you load additional content to the page dynamically via a “load more” button at the end of the page.
• Infinite Scrolling - This is where more content automatically loads as you scroll to the bottom of the page.

I suspect the increased usage is due, at least in part, to Google announcing its ability to understand websites better by executing Javascript in 2014, and crawling AJAX in 2017.

It is surprising that news of this change has not been better disseminated to the SEO and Developer communities.

Let’s look at what John Mueller had to say about this in more detail.

Google can’t crawl content behind JavaScript “Load More” buttons

Mueller has admitted that in the past, they would try and render content behind “load more” buttons, which I guess adds to the confusion.

But, in a clear statement, Mueller said that because the cost of rendering “load more” content is so expensive, they no longer do it.

I in the past, we would try to figure this out and try to trigger some of these to see what would happen, but that’s extremely expensive when it comes to rendering the whole web, so I believe we don’t do that at all.

Later in the statement, Mueller confirmed it again:

So if you’re using a load more button with JavaScript, then that’s something we probably wouldn’t use.

Google MAY NOT crawl content loaded via infinite scroll

The position with infinite scroll is a little more complicated.

Google will render your page on a very long viewport, and if the infinite scrolling increases the viewport, then they will crawl that new content. But they will only do it the one time.

If you have a lot of content that needs to load via infinite scrolling, it probably won’t be SEO-friendly.

Here is what Mueller had to say:

We do something we call frame expansion, which is we take the page that you serve, and we try to render it on an extremely long viewport, and we see if the page does anything like infinite loading where it expands the viewport.

That’s something where we would expand that new port once and see what all loads and try to index all of that, but we would not do this multiple times in a row.

Especially when it comes to infinite scrolling like the name says some websites actually do have pretty much an infinite scrolling where you could expand this a whole bunch of times, and you wouldn’t necessarily find new content you just find kind of repetitive sections across the whole website

Mueller continued that if you do intend to use infinite scrolling anyway, then you should:

Double check our documentation to make sure that you’re implementing it in a way that makes that works well for Google for indexing

You can view the whole discussion from the Webmaster Central Office-Hours Hangout below: