Brendan Tully0 Comments
October 7, 2026
Blocking Semrush, Ahrefs and Moz from your own website sounds odd coming from a search marketing business. It is one of three Cloudflare rules commonly recommended for WordPress sites, alongside blocking XML-RPC and challenging visitors to the login page. All three cut junk traffic and server load. Before you add them, it is worth knowing what they change about how search engines and SEO tools see the site, and how to confirm nothing important was caught.
The video shows the three rules inside a live Cloudflare account, with 24 hours of results for each one. Here is how each rule interacts with search.
The rule blocks any request whose user agent contains SEMrush, Ahrefs, Open Site (Moz’s Open Site Explorer) or Dotbot. The case for it is load. These crawlers are aggressive, and on WooCommerce sites they follow add to cart and wishlist links several times per second. To the server that looks like a crowd of shoppers working on carts at once, which is a real burden on shared or low-end hosting. In the video the rule blocked 32 requests in a day, after the crawlers had already calmed down from being blocked.
The second benefit is competitive. These tools publish what they find about a site’s structure and SEO to anyone with a subscription. Blocking their crawlers limits what a competitor can learn about your site from them.
The cost is your own data. If you or your agency pay for one of these tools, its site audit for your website will stop working once the rule is live. Other features are not affected the same way: backlink data comes from crawling other people’s sites, and rank tracking reads Google’s results pages rather than your site. If you rely on a monthly audit, pause the rule while the audit runs, or skip it for the one tool you use.
As written, all three rules are safe for Google and Bing. XML-RPC is an old remote publishing interface, and wp-login.php is a login form; neither is a page search engines need to index. None of the four crawler names appears in Googlebot’s or Bingbot’s user agent. The risk comes from editing the rules later. Broadening the crawler rule to match a generic word such as “bot” or “crawler” would block Googlebot along with everything else.
After deploying, open Security Events in Cloudflare and filter by the Googlebot user agent. There should be no blocked or challenged requests from any of the three rules. A live URL Inspection test in Google Search Console on a couple of key pages confirms Google can still fetch them.
Junk traffic does not come spread across the day. The XML-RPC file can be flooded hard enough to take hosting down, and the login page on a WordPress site is a constant brute force target. When this rule was first applied to sites, the video reports between 1,000 and 10,000 login attempts a day. Every one of those requests makes PHP work.
A server busy with that traffic answers everyone more slowly, including Googlebot, and Google reduces how much it crawls when a server responds slowly or struggles. Slow responses also hurt the visitors you paid for or earned from search. Filtering at Cloudflare stops the requests before they reach the hosting, which is the advantage over a security plugin that blocks them only after WordPress has started to load.
The third rule shows a Cloudflare captcha to anyone outside the site’s home country (Australia in the video) who requests wp-login.php. For a brochure site that only affects the people who manage it. On a WooCommerce store, the video flags that real customers logging in could be challenged. If you sell internationally, that is friction placed on returning customers, so use this rule with caution on stores or exempt the countries you sell to.
All three rules run on Cloudflare’s free plan and take effect as soon as they are deployed. Of the three, the crawler block usually does the most for WooCommerce stores and the login rule the most for general sites, but only the crawler block needs a decision about your own SEO tooling.
Comments are closed.