I would check user agent for AI crawlers..
Login to reply
Replies (1)
Yeah i had anthropic doing millions of requests before and blocked them, its a lot of different ips for just crawling