Cognition: I am not at all clear that I follow Henry Farrell here. One could approach things from the biological-evolutionary perspective, analyzing the problem of how yeast (once one had yeast) became grass and fish and now walks on two legs like a Thrint. One could approach things from the biological-developmental perspective, analyzing the problem of how a fertilized ovum weighing 0.1 µg (once one has a fertilized ovum) turns itself from a structured chemical soup and now walks on two legs like a Thrint. In the first case one cannot but be awestruck by the difficulty of the task—and think that huge amounts of scale plus reinforcement-learning-with-Darwinian-feedback are essential. In the second case one cannot but break down and cry, as one tries to envision how chemical-gradient signals can guide cellular differentiation to structure some kind of electricity-conducting network that can have enough instincts and reflexes and then learn enough about the world out there and now walks on two legs like a Thrint. The question whether RLHF—reinforcement-learning with human feedback—can substitute for evolution across æons on the one hand and the chemical signals that guide cellular differentiation and organization on the other hand is a very knotty one. But when one reflects that it only took the people of Newfoundland 200 years to transform mutt-mixtures of collie-mastiff-water spaniel-Portuguese water dog into the St. John’s water dog, one cannot be confident that the descendants of MAMLMs may become our full partners in less time than separates us from Gilgamesh:

Henry Farrell: The Engineer as Magus: ‘If you believe that intelligence is some combination of (a) an objective function, (b) some workaday architecture (since intelligence evolves easily), and (c) shitloads of data, then indeed, scale is the means through which the ghost is compelled into the machine… build bigger, with ever more data, until true intelligence is achieved…. Amodei… anticipates it will be accomplished sooner rather than later… automated intelligence that can be delivered at scale, potentially transforming the human condition in some quite radical ways…. [But] there are reasons why neuroscientists have not converted en masse to the True Faith of Easy Intelligence Via Scaling, which have to do with the scientific knowledge they have accumulated <buttondown.com/appercep… about how existing brains actually seem to work… <