Gary Illyes’ Post

Tomorrow I'm gonna be in Sofia, Bulgaria for the SERP Conf event. If you want to ask me all your questions about Google Search, grab me for a coffee at the event venue! Should be fun after the 7th espresso. edit: typo

My mission this year is to figure out how to crawl even less, and have fewer bytes on wire. A few days ago there was a post on a Reddit community about how, in the OC's perception, Google is crawling less than previous years. In the grand scheme of things that's just not the case; we're crawling roughly as much as before, however scheduling got more intelligent and we're focusing more on URLs that more likely to deserve crawling. However, we should, in fact, crawl less. We should, for example, be more intelligent about caching and internal cache sharing among user agents, and we should have fewer bytes on wire. If you've seen an interesting IETF (or other standards body) internet draft that could help with this effort, or an actual standard I might've missed, send it my way. Decreasing crawling without sacrificing crawl-quality would benefit everyone.

Since in JavaScript 0 is in superposition (both null and not null at the same time), does that mean that JavaScript is a quantum computing programming language? 0 > null is false 0 >= null is true 0 === null is false 0 == null is false 0 < null is false 0 <= null is true Credit to random dude on TikTok whose video was scraped and I can't find his original video, but I refuse to link to the scraper. Anyway, what did I miss while on vacation?

We have a refreshed SEO Starter Guide and it's... short. This makes me not unhappy. If you're an SEO professional it probably bores you to read about the basics, but there are tons of people out there who are still just thinking about dipping their toesies in the scary waters of SEO. This guide is for them and now it's compact enough that you don't need to have a bottle of Bunnahabhain next to you to be able to read it. Yet, it still touches on the things that we think are important when it comes to search engine optimization. I don't wanna tell you how to live your life, but check out the blog post for more info on this refresh: https://lnkd.in/gqadBtes

If Lufthansa so wants to, tomorrow I'll be in Kolding, Denmark, for SEOday.dk. If you're around, drop a comment if you'd like to have coffee and talk about searchy nerd stuff.

Weekend read about de/centralization in internet infrastructure.

The Crawl Limiter tool in Search Console is going away... and that's ok. In general there were very few instances when you should've used it in the first place. I can't actually think of a single one when it would've been appropriate, but crawling emergencies is definitely not one of them. To reduce crawl rate immediately, reply with 509, 503, or 429 http status codes and Google's crawlers will slow down very soon after. And if the issue is caused by overfiltered search result pages, you should probably disallow those, which will fix the issue. https://lnkd.in/eUqa-kHU

I'm in Sydney NSW next Wednesday, 29th of November, and I have some time to kill. If you wanna meet for coffee somewhere around Hyde Park and ask me all your Search (and some ai) questions, leave a comment!

AND if you're looking for actual validation of the robotstxt rules against URLs, Will Critchlow has a tool that's using the open source parser that we put on GitHub (which is really all you need), and iirc also implements some off-parser logic. The tool: https://lnkd.in/eUy94RnB And if you want to build your own tool, here's the repo with links to other language versions too: https://lnkd.in/ee4jDkry edit: added the tool URL