boiflndbjkenhmlbgbdnfebpooigmnkc
Robots.txt Checker is an essential tool designed to ensure the efficiency, accuracy, and validity of a website's robots.txt file. Robots.txt Checker by cmlabs is your ultimate tool for managing the essential aspects of your website's robots.txt file. Tailored for website owners and developers alike, this tool simplifies the often complex tasks associated with maintaining a healthy robots.txt configuration. With just a few clicks, you can ensure that your directives are correctly set up to guide search engine crawlers effectively. This tool can swiftly verify whether specific URLs are being appropriately blocked or allowed by your robots.txt directives. Let’s take control of your website's indexing directives. Download and try now! - This tool is available for free. - Checking Blocked URLs: Help you verify whether specific URLs on your website are blocked by the robots.txt file. - Identification of Blocking Statements: These statements are rules containing instructions for search engines not to index or access specific pages or directories on a website. - Checking Sitemap Files: The sitemap.xml file is an essential document to enhance your site's visibility in search engines. 1. Open the Robots.txt Checker You can proceed by choosing the Robots.txt Checker tool to start analyzing URLs and checking the robots.txt or sitemap.xml files within them. 2. Enter the URL To initiate the review process, simply enter the URL, as shown in the example in the blue box at the top of the tool's page. For a smooth review process, make sure the URL you enter follows the format: https://www.example.com. 3. Start the Review Process After entering the URL, you'll see several buttons, including "Check Source", selecting the bot type, and checking the URL through the "Check URL" button. Please note that you can only review URLs up to 5 times within 1 hour. 4. Analyze the Data Once the review process is complete, you'll be presented with results that show several pieces of information, including: We value your feedback! If you have any suggestions for improving Robots.txt Checker or encounter any issues while using the tool, please don't hesitate to let us know. Our support team is here to help. Reach us by email at:
Robots Exclusion Checker
Checks robots.txt, meta robots, x-robots-tag with URL alerts. Canonical warnings, HTTP header info. An SEO extension, robots tester. Robots Exclusion Checker is designed to visually indicate whether any robots exclusions are preventing your page from being crawled or indexed by Search Engines. ## The extension reports on 6 elements: 1. Robots.txt 2. Meta Robots tag 3. A.I. Bots 4. X-robots-tag 5. Rel=Canonical 6. UGC, Sponsored and Nofollow attribute values If a URL you are visiting is being affected by an "Allow” or “Disallow” within robots.txt, the extension will show you the specific rule within the extension, making it easy to copy or visit the live robots.txt. You will also be shown the full robots.txt with the specific rule highlighted (if applicable). Cool eh! Any Robots Meta tags that direct robots to “index", “noindex", “follow" or “nofollow" will flag the appropriate Red, Amber or Green icons. Directives that won’t affect Search Engine indexation, such as “nosnippet” or “noodp” will be shown but won’t be factored into the alerts. The extension makes it easy to view all directives, along with showing you any HTML meta robots tags in full that appear in the source code. Checks whether a website's robots.txt file blocks A.I. companies from accessing its content. It monitors 14 bots across 6 companies - OpenAI, Anthropic, Google, Perplexity, Meta, and Apple, covering three types of access: training data collection, search indexing, and real-time browsing. If any bot exclusions are detected, an "AI" label will appear over the Robots Exclusion Checker icon in your browser. This feature can be deactivated within settings if preferred. Spotting any robots directives in the HTTP header has been a bit of a pain in the past but no longer with this extension. Any specific exclusions will be made very visible, as well as the full HTTP Header - with the specific exclusions highlighted too! Although the canonical tag doesn’t directly impact indexation, it can still impact how your URLs behave within SERPS (Search Engine Results Pages). If the page you are viewing is Allowed to bots but a Canonical mismatch has been detected (the current URL is different to the Canonical URL) then the extension will flag an Amber icon. Canonical information is collected on every page from within the HTML and HTTP header response. - UGC, Sponsored and Nofollow A new addition to the extension gives you the option to highlight any visible links that use a "nofollow", "ugc" or "sponsored" rel attribute value. You can control which links are highlighted and set your preferred colour for each. I’d you’d prefer this is disabled, you can switch off entirely. Within settings, you can choose one of the following user-agents to simulate what each Search Engine has access to: This tool will be useful for anyone working in Search Engine Optimisation (SEO) or digital marketing, as it gives a clear visual indication if the page is being blocked by robots.txt (many existing extensions don’t flag this). Crawl or indexation issues have a direct bearing on how well your website performs in organic results, so this extension should be part of your SEO developer toolkit for Google Chrome. An alternative to some of the common robots.txt testers available online. This extension is useful for: - Faceted navigation review and optimisation (useful to see the robot control behind complex / stacked facets) - Detecting crawl or indexation issues - General SEO review and auditing within your browser ## Avoid the need for multiple SEO Extensions Within the realm of robots and indexation, there is no better extension available. In fact, by installing Robots Exclusion Checker you will avoid having to run multiple extensions within Chrome that will slow down its functionality. 1.0.3: Various bug fixes, including better handling of URLs with encoded characters. Robots.txt expansion feature to allow the viewing of extra-long rules. Now JavaScript history.pushState() compatible. 1.0.4: Various upgrades. Canonical tag detection added (HTML and HTTP Header) with Amber icon alerts. Robots.txt is now shown in full, with the appropriate rule highlighted. X-robots-tag now highlighted within full HTTP header information. Various UX improvements, such as "Copy to Clipboard” and “View Source” links. Social share icons added. 1.0.5: Forces a background HTTP header call when the extension detects a URL change but no new HTTP header info - mainly for sites heavily dependant on JavaScript. 1.0.6: Fixed an issue with the hash part of the URL when doing a canonical check. 1.0.7: Forces a background body response call in addition to HTTP headers, to ensure a non-cached view of the URL for JavaScript heavy sites. 1.0.8: Fixed an error that occurred when multiple references to the same user-agent were detected within robots.txt file. 1.0.9: Fixed an issue with the canonical mismatch alert. 1.1.0: Various UI updates, including a JavaScript alert when the extension detects a URL change with no new HTTP request. 1.1.3: Added UGC, Sponsored and Nofollow link highlighting. 1.1.4: Switched off nofollow link highlighting by default on new installs and fixed a bug related to HTTP header canonical mismatches. 1.1.6: Extension now flags 404 errors in Red. 1.1.7: Not sending cookies when making a background request to fetch a page that was navigated to with pushstate. 1.1.8: Improvements to the handling of relative vs absolute canonical URLs and unencoded URL messaging. 1.2.0.11: Updating to Google's new manifest V3 and fixing small bugs. 1.2.0.12: Added a Spanish language version and made improvements to existing translations. Linking to new website https://www.checkrobots.com 1.2.0.13: Fixed pushState navigation data extraction, resolved inconsistent icon display, and added security protections to prevent logout issues with enterprise websites. 1.3.0: Introduced A.I. bot checking to monitor robots.txt exclusion rules for 14 bots across 6 companies including OpenAI, Anthropic, Google, Perplexity, Meta and Apple, with training, search indexing and real-time browsing bots tracked. New site exclusion feature lets you skip checking for specific domains. Fixes for SPA/back-forward navigation, x-robots-tag case sensitivity for Bingbot and Yahoo and improved allow/disallow rule precedence. Redesigned settings page with collapsible sections.
SiteAnalyzer SEO Tools: Free Meta & SEO Checker
Free browser extension that helps you analyze website content, detect and fix all SEO-errors (links, images, meta, content, etc.). SiteAnalyzer SEO Tools is a free browser extension providing a set of tools for analyzing SEO indicators of web site, technical audit (links, images, meta tags, content, indexing, etc.), URL analysis and correction of SEO optimization errors. SiteAnalyzer SEO Tools is a convenient tool for webmasters and SEO specialists, including many parameters for checking the quality of the site and its pages, configured to identify the most popular SEO errors. The main features of SiteAnalyzer SEO Tools ✓ Determining the total number of site pages in the Google & Yahoo! index ✓ Checking the URL indexing status in Google and Bing ✓ Reveals the contents of Title, Description, H1-H6 tags ✓ Proper check of «Robots.txt» and «Sitemap.xml» presence ✓ Rules accounting for «Robots.txt», meta tag «Robots» and header «X-Robots-Tag» ✓ Canonical URL Checker ✓ Checking for REL «Nofollow», «Sponsored» and «UGC» for links and URLs ✓ Defining internal and external page links ✓ Semantic analysis of page content ✓ Checking pages loading speed ✓ Receiving server response codes (200, 3xx, 4xx, 5xx) ✓ Determining what CMS is used by web site ✓ Checking the registration date and the domain age ✓ Displaying all images of the page and the contents of the ALT tag ✓ Determining the IP address and country of web site ✓ Display Schema, Open Graph and Twitter markup ✓ Determining the presence of links to social networks ✓ Search for subdomains and sites on the same IP ✓ Extracting text Suggestions from Google Images ✓ Show duplicate H1-H6 headers ✓ SSL Certificate Verification ✓ DNS Domain Validation ✓ PageRank Calculation ✓ Sitemap Visualization ✓ Sitemap Clustering ✓ CSR vs SSR Site Rendering Visualization ✓ Schema.org Visualization (visualize JSON-LD schema markup) ✓ Google SERP Analysis (Organic, Advert, Local, Product, Video, Image, News, FAQ, PAA) ✓ TF-IDF Checker for Google SERP ✓ Core Web Vitals ✓ Quick Source Viewer ✓ Integration with Google Search Console ✓ Integration with SimilarWeb ✓ Page SEO Audit by 40 parameters ✓ Export reports to CSV and PNG How can the plugin help SEO specialist? ★ Checking the indexing of pages ★ Analysis of meta tags and page content ★ Search for technical bugs on the site The extension is distributed absolutely free and does not require any registration since it does not collect or send elsewhere your personal data. Updating the version An extension installed via the Chrome Online Store or through Firefox Add-ons are updated independently so no additional actions are required.
AutoTDK - SEO Search Keyword Tool
SEO Keyword Search Tool searches using long-tail keywords and provides free website traffic, Whois and SEO analysis features. SEO Search Keyword Tool & SEO Keyword Research Tool. Autotdk is a free SEO keyword research tool that helps you generate up to 1,000 low-competition, high-traffic keywords from multiple search engines using long-tail keywords. This SEO keyword tool is designed for Google SEO, content marketing, and competitor analysis, helping you quickly discover high-ranking keyword opportunities. The free SEO Keyword Research Tool allows you to search using long-tail keywords and generate low-competition keyword ideas. With Autotdk, you can access keyword search volume, keyword difficulty, and keyword suggestions for better SEO keyword research and content optimization. Autotdk SEO extension also provides powerful website analysis features, including traffic analysis, keyword analysis, backlink analysis, Whois lookup, and SEO insights to help you fully understand website performance. When visiting any website, simply click the Autotdk icon in your browser to view the following metrics: 🌍 Overview — View domain TDK information, backlink count, and overall SEO metrics 📈 Traffic — Analyze traffic trends, channels, and country distribution 📌 Keywords — Discover SEO keywords driving traffic, search volume, and difficulty 🔗 Backlinks — Analyze referring domains, backlinks, and growth trends 🤖 AI Visibility — Check AI search score and brand visibility Autotdk is a must-have SEO keyword research tool for SEO specialists, marketers, and ecommerce teams, helping you discover new SEO keyword opportunities and improve search rankings. Privacy & Security: We do not collect sensitive personal information We do not track your browsing activity All data is used only to provide our service
Meta Explorer - SEO Meta and On-Page SEO tools
Meta Tag Explorer is the best SEO tool to find SEO Meta of any website you browse Meta Explorer - The easiest and fastest way to find SEO Meta. - Find issues with your on page SEO in seconds (You can export issues as csv). - Pin the extension to keep it open across page refreshes. This comes in handy if you want to check SEO Meta of your content pages like your blog. - Works well on single page apps - Pleasant and modern UI All the existing tools need you to either click on an extension or copy paste the url in a tool to check the on page SEO. This gets very cumbersome if you have to check the on-page SEO for multiple pages, like a blog. With Meta Explorer, you don't need any clicks or copy-paste. Pin the extension & navigate to any page whose meta & on-page SEO you want to check and voila! you have the info in the sidebar. It also has a modern and pleasing UI. You won't get tired of looking at bad UI anymore. Key Features ✅ View & Copy meta tags ✅ View social tags (OG, Twitter) ✅ View header element count ✅ View header hierarchy ✅ Image file names & alt text ✅ Internal & External link count ✅ Links with Anchor text ✅ Scroll links into view ✅ Show schema markup if present ✅ Keyboard shortcut to toggle extension open/close (Cmd+Shift+E / Ctrl+Shift+E) 🐜 Issues - Check & Fix issues to improve your on page SEO. - Check Web Vitals - Export issues as CSV. 🔗 LINKS - Internal, External, No Follow link count - Links with Anchor text. - Check broken links - You can scroll the link into your view with a single click. 💬 Social - Check social preview of the site (Google, Twitter/X, Facebook, Linkedin, Slack) 🔧 Tools - Page speed insights - Ahrefs website authority checker - Partner SEO tools - Some links are affiliate links. Using them supports me to keep improving the extension. 🧐 Privacy - We don't collect any personal data or track any of your usage metrics. - We make network requests only to check broken links on your site, after you click on `Check Broken Links` button.
