⚡️🚨 NEW - Cloudflare says it has fended off 416 billion AI bot scraping requests in five months.
CEO warns of dramatic shift in internet business model.
Login to reply
Replies (2)
Hypocrite of the week (from the linked article): “While Cloudflare blocks almost all AI crawlers, there’s one particular bot it cannot block without affecting its customers’ online presence — Google. The search giant combined its search and AI crawler into one, meaning users who opt out of Google’s AI crawler won't be indexed in Google search results. “You can’t opt out of one without opting out of both, which is a real challenge — it’s crazy,” Prince continued. “It shouldn’t be that you can use your monopoly position of yesterday in order to leverage and have a monopoly position in the market of tomorrow.””
Isn’t that exactly what Cloudflare is doing for their customers with their new “feature” to block all AI web scrapers? They have a caching platform that dominates the internet, and by default now, blocks all AI web crawlers. What if an AI web crawler is trying to help me find the most relevant content related to my research? Or what if I am using an AI bot to find and track of all the kiddie-P hiding on various mega-platforms so I can go arrest those f-ers? Cloudflare gets to decide what I scrape now? Pot, meet kettle!