Let us note the distinction between this declarative signal (non-binding) and the real technical blockage of Cloudflare planned at the network level from September 2026, which operates independently of robots.txt. Barry Schwartz points out the recurring gap between what publishers hope for and what crawlers actually respect. Some also see this as a predictable position from Google, which favors its own control mechanisms (Googlebot, Google-Extended).