Threat of possible litigation
Making this look a bit like a terms of service agreement, Cloudflare's goal is clearly to put legal pressure on Google to change its policy of combining traditional search crawlers and AI reviews.
“Make no mistake, Google's legal team is looking at this, 'Yeah, now we have to actively ignore this across a large portion of the web,'” Prince told me.
He went on to characterize it as an attempt to force the company, which he said has historically been a “largely good player” and “patron of the Internet,” to get back to doing the right thing.
“There's a battle going on inside Google between people who say we have to change the way we do this,” he explained. “And there are other people who say, no, this takes away the inherent advantage that we have a God-given right to all content on the Internet.”
Amid that debate, lawyers gained influence at Google, so Cloudflare tried to develop tools “that would make it clear that if they were going to sign up for any of these sites, there was a clear license for them. And that would put them at risk if they didn't follow it,” Prince said.
The Next Web Paradigm
To do something like this, it takes a company the size of Cloudflare hoping it will make an impact. If just a few websites made this change, it would be easier for Google to ignore it, or worse, it might just stop crawling them to avoid the problem. Because Cloudflare is intertwined with millions of websites, Google can't do this without significantly impacting search quality.
