Yea so basically the short version is that Google is constantly constantly updating their search algorithms in response to SEO abuse. And they are rather secretive about what makes for good SEO, precisely because they do not want bad actors to game the system. So for e.g. when you hear about people being downranked and then getting stonewalled when they ask google why, that’s generally not malicious; its just that google can’t really reveal why. The same issue plagues all spam and abuse implementation — if your account gets flagged or banned, Google isn’t going to tell you why so that malicious actors cant learn the algorithm and then game it.

In some sense you can think of this as ‘Google trying to fix SEO’, but in another sense, it doesn’t really get rid of the bad behavior we’re talking about.

You could imagine various ways that the government (or Google!) could try and rein in this externality. Here are a few, none of which I will actively defend but just food for thought:

• Google could charge businesses that appear at the front of search rankings if said businesses also have adwords/adsense accounts

• Google could literally randomize the first page of results, thereby removing the incentive to game the SEO as hard as they would

• The government could introduce some kind of legal liability for excessive SEO — this would primarily result in less SEO for the most rule-abiding players (who are also least likely to be spammy), but that would then free Google up to be way more aggressive about downranking or outright ip blocking SEO spam (because they can be less worried that they are accidentally blocking good actors). Probably this would be a 1A violation, but note that this category of thing cannot be unilaterally implemented by Google.

• The government could tax SEO. I have no idea how, but again something that is unique to the government.

The reason I didn’t suggest these in the article is because for the most part, I think these tradeoffs just make the search page worse, or introduce worse incentives/outcomes in other areas, and I want to come up with solutions that solve the bad behavior but also keep the search page pretty good. Like, consider the randomizing-the-first-page-of-results ‘solution’. Is that really any better than just dealing with the SEO? Unclear. Probably on net, from a societal impact perspective, it might be. But its very fuzzy to me — Google puts a lot of effort into trying to get the first few hits to be really really accurate.