cloudflare just broke the internet's business model.
they launched "pay per crawl" websites can now charge ai crawlers for scraping content. instead of "block all bots" or "let them steal everything," there's option 3: "pay me."
why this is a BIG DEAL:
every SaaS has valuable data rotting in help docs, case studies, feature pages. ai companies have been training on this for free while building your competitors. now you can charge them $0.10 per page.
your dental saas help docs, well, that's years of practice wisdom. charge crawlers for access and make $3k/month from companies building "dentist ai."
tons of opportunity for founders building businesses:
build a restaurant review database, charge humans $5/month but charge ai crawlers $0.01 per review. two revenue streams, same content. im scanning startup ideas from @ideabrowser and seeing where i can add this rev stream.
expert knowledge platforms where you take 30% of human subs and 50% of crawler fees.
cloudflare handles the billing, you set the price.
they way to think about it...
what valuable data do you have sitting around... internal company docs, customer conversations, industry insights, process knowledge.
all of it can now generate revenue from ai companies who desperately need training data.
the winners will be content businesses that move fast. imagine owning a cooking blog with 10 years of recipes. that's training data gold for ai food companies. early adopters will set market rates before competition drives prices down.
this also completely changes saas valuations. businesses with proprietary content now have two revenue streams instead of one.
it's the beginning of a complete realignment of incentives on the internet.
suddenly, creating high-quality, unique content becomes dramatically more valuable. we'll see a renaissance of niche expertise sites because depth now pays better than breadth. the incentive shifts from "create content to rank in google" to "create content so valuable that ai companies must pay for it."
this also creates a new class of digital asset. your company's knowledge base is a revenue-generating asset that appreciates over time (not a cost center).
I'd expect more m&a activity around content libraries, not just user bases.
this makes human expertise defensible again. AI companies will pay premium rates for content from recognized experts because it's higher quality training data. this reverses the commoditization of knowledge.
cloudflare quietly changed the game.
pay-per-crawl just became your new best friend.
cc - Greg Isenberg
