Make money doing the work you believe in
In Elizabeth Eisenstein’s book on print, there’s an extended argument about how print introduced “fixity” into texts, by which she meant the gradual removal of differences between manuscript versions, separation of core texts from marginalia on copies held by individuals or commentary, and conservative resolution of editorialized changes. So texts went from n 1/1 “parallel” versions to n 1/n identical serially manufactured copies of “editions.” It was like version control with merge conflicts resolved slowly, and tags. The resolution went down to the level of letter shapes, punctuation, word/line/paragraph breaks, and spellings.
Since I first read that a year ago, I’ve been convinced there’s a similar fixity process with LLMs. I thought of it as semantic fixity as opposed to versioning fixity. Stabilization of meanings. My first stab at modeling that was LLMs as index funds. Convergence and uniformity at the level of overall ideas and novelty of takes. But this is not quite complete. It’s too macro. Works partly for text but we need something more fine-grained overall, and a version that works for code and images too. And eventually robots too.
One possibility is not just semantic fixity but functional fixity, which together create interpretive fixity. This means removal of polysemy, irony, satire, and other modes of layering multiple meanings in tension with each other into a single text. The code equivalent might be things like multiple inheritance, late binding, etc. In images, duck-rabbit illusions. Robot version might be functionally unfixed behaviors like using limbs unconventionally.
The very term functional fixity is a non-example of itself. It simultaneously gestures at Eisenstein’s fixity and the psychological notion of functional fixedness (using a thing in only one prescribed way and being unable to see creative alternatives uses).
Stuff that trades ambiguity in denotation and connotation for useful suppleness in use, by way of an unstable set of meanings in equilibrium. Your mind goes “duck-rabbit-duck-rabbit-duck-rabbit.” Meaning lies in that unstable movement and indeterminacy of state.
An example. Long ago, I was chatting with an artist friend of mine about a comic we both liked called “toothpaste for dinner.” At the time he was also doing a commissioned strip gig that I had helped him get, for a professor who wanted editorial cartoons for a scholarly magazine but had no real sense of humor, so his gag ideas were tedious and not fun for my friend to illustrate. He described the feeling to me as “it’s like he wants a comic strip called toothpaste for brushing your teeth.”
The joke title toothpaste for dinner works because the weird idea of eating toothpaste as food lies incongruously in the context of the normal idea of using it for brushing. It’s only funny if both meanings are in context at once. You need 2 meanings in unstable equilibrium making your state indeterminate.
AIs sucks so far at anything involving holding unstable meanings in indeterminate micro-equilibrium. They produce output that uses every token to serve only one function, creating a certain thinness. Human output has layered thickness. AIs have indeterminacy in the production process (the next token choice has randomness like rolling a die) but not in intent or role. This is functional fixity that adds thinness and stiffness to the semantic fixity already created by the index fund effect.
I think we could actually avoid this with non-transformer frameworks. Possibly running multiple text diffusion processes in parallel and blending the results would do it. We’ve already got data parallelism, model parallelism, and pipeline parallelism at a low level. I think the need for polysemantic thickness requires a new kind of compositional parallelism. Dennett’s multiple drafts model of consciousness suggests how humans do it — by merging parallel drafts.
But there’s a chance that this is not doable and AI remains “thin and stiff” rather than becoming “thick and supple” in output.
