Peter Elbow in Writing Without Teachers: “Writing is a way to end up thinking something you couldn’t have started out thinking. Writing is, in fact, a transaction with words whereby you free yourself from what you presently think, feel, and perceive. You make available to yourself something better than what you’d be stuck with if you’d actually succeeded in making your meaning clear at the start.”

– Elbow was writing pre-internet but this seems highly relevant in the LLM context. The act of doing the writing is how you come up with the valuable thinking. To prompt a chatbot to generate a piece of writing from an idea you had is thus to get things precisely upside-down. (And many other uses of LLMs in the writing process would seem to be on similarly shaky ground.)