Does Google Crawl URLs In Structured Data?

Being able to get your links discovered, crawled and indexed by Google is important to your SEO, and can ultimately be advantageous because if one URL gets crawled, then more pages can get crawled on your website.

In an episode of SEO Office Hours, John Mueller spoke up about whether or not Google used links in structured data for crawling.

When somebody asked, whether or not Google crawls URLs located in structured data markup or if it just stores the data, Mueller answered:

“So for the most part, when we look at HTML pages, if we see something that looks like a link, we might go off and kind of like try that URL out as well.

That’s something where if we find a URL in JavaScript, we can try to pick that up and try to use it.

If we find a link in kind of a text file on a site, we can try to crawl that and use it.”

This answer was mostly about what Google could do with alternative links, which are links that aren’t traditional HTML hyperlinks with anchor texts.

According to Mueller, you should use a standard HTML hyperlink if you want something that performs like a link.

Mueller continued:

“But it’s not really a normal link.

So it’s something where I would recommend if you want Google to go off and crawl that URL, make sure that there’s a natural HTML link to that URL, with a clear anchor text as well, that you give some information about the destination page.

If you don’t want Google to crawl that specific URL, then maybe block it with robots.txt or on that page use a rel=canonical pointing to your preferred version, anything like that.

So those are kind of the directions I would go there.

I would not blindly assume that just because it’s in structured data it will not be found.

Nor would I blindly assume that just because it’s in structured data it will be found.

It might be found.

It might not be found.

I would instead focus on what you want to have happen there.

If you want to have it seen as a link, then make it a link.

If you don’t want to have it crawled or indexed, then block crawling or indexing.

That’s all totally up to you.”

Check out the answer to the question at the 23:40 minute mark in the original video:

Scott Davenport

Leave a Comment

Your email address will not be published. Required fields are marked *

Are You Ready To Thrive?

This field is for validation purposes and should be left unchanged.