I’m normally all for evocative visualizations but this one (by Tomas Pueyo who often has good visual ideas especially when anchored in real visual domains like geography) is dangerously and fatally bullshitty. Reminds me of a similar and equally terrible old diagram by a guy named Matt Might showing how an individual PhD is like a pimple on the boundary of a knowledge diagram.

In mathematical terms, the basic mistake in both cases is to uncritically imagine the already shaky idea of “knowledge” as a metric (or metrizable) space and the topology of what is “known” as some sort of oriented, compact (closed and bounded), simply connected, monotonically and noiselessly accreting region within that space, that is even approximately conveniently convex at the core.

What if the grey “tasks” circle is a fragmented mess of real tasks, bullshit rituals, counterproductive behaviors etc? What if the pink “knowledge” about it a whole different mess that’s not even registered (ie aligned) with the behaviors? What if it’s 3d+ and a tangled spaghetti mess? What if most of it is contingent on shifting evolutionary context? What if the pink stuff decays and drifts and falls apart over time with a half life? What if the unmapped “unknown” part completely subverts your “known” part with a new discovery? What if the entire pink region is mostly superstition in light of superior future knowledge? What if the known only ever forms a disconnected archipelago with no interpolation possible? What if it’s all “jagged frontier” fractally, all the way down, like a Mandelbrot set? What if model training convergence phenomena we rely on today start to break? What if knowledge is scattered across the unknown like prime numbers among the integers? What if the known is a “dense subset” of knowledge with unknowables viciously annd inseparably entangled?

This is a dozen layers of real and thorny phenomenological complexity flattened into a seductive “intuitive” 2d visual. The sort of thing that math programs typically try to beat out of you across sequences of 4-5 advanced undergrad and grad courses for a very good reason. It is highly misleading when you try to think rigorously about abstract things. And there’s good reason to think knowledge is nothing like this simple — it takes connectome type objects (brains and AI models) with billions/trillions of nodes and connections to “do” knowledge. And you think the space these objects navigate looks like simple shapes in a 2d coloring book?

Even Newton was cautious enough to limit himself to a poetic metaphor about finding pebbles on a beach by the ocean of the unknown.

The broadest set of spaces mathematicians deal with, and the objects within them, are not just dizzyingly more complex, they have practical application too, in describing real things ranging from curved space times in cosmology to weird sigma algebras in statistics. You really think the phenomenology of “intelligence” and “knowledge”, even if we ever get to usefully well-posed notions of them, is going to be modelable by something this ptolemaically simple? Math is weird, hairy, and complex and so is reality. That’s why the one is so unreasonably good at describing the other.

Unfortunately, I think a lot of AGIers actually think with such mental models. It’s not just a harmless “science communication” simplification. Not least because they started with a terrible metric space (psychometric IQ) invented for bureaucratic purposes.

You can’t just assume that the already vague and ill-posed idea of “intelligence” (even before you wildly extrapolate the dubious statistics of “g” in IQ into the cosmic “G” of “AGI”) somehow magically constitutes a beautifully well-behaved mathematical object that you can then flatten into a neat visual pointing to a couple of clean, coherent futures that, surprise, surprise, you’re already conceptually equipped to talk about.