Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • SEO Q&A
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • Case Studies
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      Discover top competitors’ winning content
      What's New

      Discover top competitors’ winning content

      Learn More
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      What is your Brand Authority?
      Moz

      What is your Brand Authority?

      Take the quiz
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • SEO Q&A

        Insights & discussions from an SEO community of 500,000+.

      Unlock flexible pricing & new endpoints
      The Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • Case Studies

        Explore how Moz drives ROI with a proven track record of success.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature: Moz Pro

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. Moz Tools
    3. Moz Pro
    4. Blog archive pages in Craw Error Report

    Unsolved Blog archive pages in Craw Error Report

    Moz Pro
    seo audit
    4
    4
    1022
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • mhenshall
      mhenshall Subscriber last edited by

      Hi there,

      I'm new to MOZ Pro and have a question. My scan shows Archive pages as having crawl issues, but this is because Yoast is set up to block robots on these pages.

      Should I be allowing search engines to crawl these pages, or am I fine to leave them as I have it set up already?

      Any advice is greatly appreciated.
      Marc

      fcevey 1 Reply Last reply Reply Quote 0
      • Johnweki
        Johnweki @fcevey last edited by

        @fcevey
        If blog archive pages are showing up in the crawl error report, it indicates that search engine bots are encountering issues while attempting to crawl and index those pages. To address this:

        Check URL Structure: Ensure that the URLs for your blog archive pages are correctly formatted and follow best practices. Avoid special characters, and use a logical and organized structure.

        Update Sitemap: Make sure that the blog archive pages are included in your website's XML sitemap. Submit the updated sitemap to search engines using their respective webmaster tools.

        Robots.txt File: Review your website's robots.txt file to ensure it's not blocking search engine bots from crawling your blog archive pages. Adjust the file if needed.

        HTTP Status Codes: Check if the archive pages return the correct HTTP status codes (e.g., 200 OK). Crawl errors might be triggered if pages return 4xx or 5xx status codes.

        Internal Linking: Ensure that there are internal links pointing to your blog archive pages. This helps search engines discover and index these pages more effectively.

        Redirects: If you've recently changed the URL structure or migrated your website, implement proper redirects from old URLs to new ones to maintain SEO authority.

        Server Issues: Investigate if there are any server-related issues causing intermittent errors when search engine bots try to access the blog archive pages.

        1 Reply Last reply Reply Quote 0
        • educationedge012
          educationedge012 last edited by

          Blog archive pages in the crawl error report indicate issues or problems encountered while indexing or accessing the archive pages of a blog or website. These errors need attention to ensure that all content remains accessible to users and search engines. ( what is project management) ( PMP Exam Prep) (study abroad)

          1 Reply Last reply Reply Quote 0
          • fcevey
            fcevey @mhenshall last edited by

            @mhenshall The decision to allow search engines to crawl archive pages in YOAST SEO or leave them as they are currently configured depends on your specific goals and needs.

            If the archive pages contain valuable and relevant content for both search engines and users, allowing them to be crawled could enhance the visibility of that content in search results. However, if the content on the archive pages is not important or is duplicated from other pages, blocking crawling could be a valid option to prevent indexing issues and improve the user experience.

            I would recommend evaluating the content on the archive pages and considering how their visibility in search engines will be affected by allowing or blocking crawling. You can use tools like Google Search Console to monitor how Google is indexing those pages and make informed decisions based on the data.

            Keep in mind that configuring YOAST SEO is a strategic decision that should align with your SEO goals and your webs```
            code_text

            Free Research Preview. ChatGPT may produce inaccurate information about people, places, or facts.  Follow us [https://posicionamiento-web-seo.com.ar/](link url)
            Johnweki 1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post

            Got a burning SEO question?

            Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


            Start my free trial


            Browse Questions

            Explore more categories

            • Moz Tools

              Chat with the community about the Moz tools.

            • SEO Tactics

              Discuss the SEO process with fellow marketers

            • Community

              Discuss industry events, jobs, and news!

            • Digital Marketing

              Chat about tactics outside of SEO

            • Research & Trends

              Dive into research and trends in the search industry.

            • Support

              Connect on product support and feature requests.

            • See all categories

            Related Questions

            • moon-boots

              SEO Drop

              seo audit seo rankings

              Over the last few months my rank has dropped by around half and for the life of me I can’t see why. There are no warnings on Google Console. Am I missing something? Website: thespacecollective.com

              Intermediate & Advanced SEO | | moon-boots
              0
            • seotoolsking

              Unsolved How to check the Domain Age?

              seo tools seo audit

              Check the Domain Age on THE SEO TOOLS KING
              Get Free seo tools only on THE SEO TOOLS KING

              Moz Tools | | seotoolsking
              4
            • valigholami1386

              site crawl

              seo seo audit

              hello . my website is https://papionshopping.ir/ but i cant make a site crawl for this . everytime i run crawling it stops and shows nothing !!!!!!!!!!!!!!!!!! plz helppp.

              Other SEO Tools | | valigholami1386
              0
            • gokimedia

              Pages with Duplicate Content Error

              duplicate content crawl errors seo audit

              Hello, the result of renewed content appeared in the scan results in my Shopify Store. But these products are unique. Why am I getting this error? Can anyone please help to explain why? screenshot-analytics.moz.com-2021.10.28-19_53_09.png

              Moz Pro | | gokimedia
              0
            • 0eup.ombitao

              How .ae and .com domain in SEO performance for UAE region?

              seo audit domain rankings

              I have a domain for my UAE based project called https://mydubaiseo.com/ and however, one of my colleagues suggested going with .ae option.
              Whether if we change the domain like as suggested get earlier results than .com domain or what?\Which domain .com or .ae ranks faster in UAE location if the SEO strategies followed in the same way?

              Technical SEO | | 0eup.ombitao
              0
            • Gordon_Hall

              On page report card - small niggle

              I've been carefully making page corrections and trying to get each age in line with SEOMoz recommendations. However, under the section marked Optional (and I realise I could just ignore it!) it tells me to "Avoid Using Meta Keywords Tag". However, none of the pages have any meta keywords in. They have a meta description, but no keywords. I have also removed any global keywords. Is the software wrong, or are there some more hidden somewhere.... website page is http://www.forktruckexpress.com/Hire/rossendale-forklift-hire.html Thanks in advance

              Moz Pro | | Gordon_Hall
              0
            • Joydeep_das

              How to increase the page authority

              Need good rank on this 2 keyword, plz help me if anyone can help. Keywords are : gift card http://www.giftbig.com/gift-cards.html gift voucher http://www.giftbig.com/gift-vouchers.html

              Moz Pro | | Joydeep_das
              0
            • mjtaylor

              Crawl Errors Confusing Me

              The SEOMoz crawl tool is telling me that I have a slew of crawl errors on the blog of one domain. All are related to the MSNbot. And related to trackbacks (which we do want to block, right?) and attachments (makes sense to block those, too) ... any idea why these are crawl issues with MSNbot and not Google? My robots.txt is here: http://www.wevegotthekeys.com/robots.txt. Thanks, MJ

              Moz Pro | | mjtaylor
              0
            Moz logo
            • Contact
            • Community
            • Free Trial
            • Terms & Privacy
            • Accessibility
            • Jobs
            • Help
            • News & Press
            • MozCon
            © 2021 - 2024 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.

            Looks like your connection to Moz was lost, please wait while we try to reconnect.