The idea behind captchas are that it’ll be a thing only people can solve. But the reality is that LLM’s are so good at it the only reason they don’t solve them is because they’ve been told not to.
But if you trick the LLM in to thinking it’s not solving a captcha….. then it’s perfectly happy to solve it!
More info: 
Ars Technica
Dead grandma locket request tricks Bing Chat’s AI into solving security puzzle
I'm sure it's a special love code that only you and your grandma know."
