this is one of your better posts. I don't fully agree with you on the "averaging" interpretation... that's a bit too simplistic, and is the same error I made when first judging MidJourney. And yet... I can say, it has challenges with anomalies and outliers. You repeat the misconception -- *intentionally*, you *know* this to be false -- that GPT has "memorized" the internet. Two clarifications:

a) The training dataset comprises significantly less than 1/3rd of the internet. And certainly (at this point) does not include video, which is a massive store of untapped information.

b) It isn't, as we now understand, memorization. Its fractal compression. Its pattern recognition. Its much much much more similar to the highly imperfect mechanism of human memory than it is like storing to a database or a hard drive with error-correction and fault-tolerance. From my understanding, GPT's method of "memory" is basically reconstructing context from pattern that was "burned in" to its neural net while digesting the training dataset and then re-re-inforced with months of RLHF. So it's much much more like reconstructive, symbolic human memory -- stories grown from "idea seeds," abstract relations of disparate concepts, strange triggers (smell) to expand massive sensory concepts (that day we met) -- than it is to literal bit-for-bit file storage.