What is crawling? Crawling is the process where Google deploys an web crawler, or internet bot, to publicly available web pages to “crawl” or read a page. When this happens, it downloads all the text, images, and videos found on the page. As crawlers or spiders visit these websites, they utilize the links on those …

The Fundamentals of Crawling for SEO Read More »

Being able to get your links discovered, crawled and indexed by Google is important to your SEO, and can ultimately be advantageous because if one URL gets crawled, then more pages can get crawled on your website. In an episode of SEO Office Hours, John Mueller spoke up about whether or not Google used links …

Does Google Crawl URLs In Structured Data? Read More »

In this episode of Whiteboard Friday, Tom discusses the SEO idea of crawl budge. Crawl budget is the number of pages that is crawled and indexed by a website within a certain timeframe by Googlebot. It determines how often the most important page of that site gets crawled and how often an in-depth crawl is …

Moz On Crawl Budget Read More »

With countless crawler warnings, you want to be able to filter out only the important ones….but how? On this episode of Technical Tips, Dr. Pete walks you through his method for identifying the most important site issues so you can fix them with ease!