There is much ongoing discussion about the recent announcement by Substack to introduce a tool for AI detection to apply to posts, notes and comments.

I was initially very skeptical, since I am aware of the literature showing that these tools aren't as reliable as often claimed to. But I have used my usual approach to the unknown, and looked around, finding the post below shared directly by pangram.

As far as I understand, it's a classifier: it does not try to reverse engineering. That's a good thing.

Another good thing is that they are aware that their tool works statistically, and I think that this point should be made very clear in any Substack integration. If the tool claims that a post is AI-generated, it means that there is a X% chance that the text had been generated by AI: it says nothing about the underlying process, and overall it means that the classification might be wrong. Being wrong once in every 50 documents is not bad, but it is not even great: on average one human-generated text will be recognized as AI.

Provided that Substack explains the limitations of such tools, I welcome them. Why? Because, as I have written multiple times, it is disappointing and disheartening to see every platform, and especially this one, invaded by self-appointed experts who write about concepts and topics they don't master.

Platforms should act against demonstrably deceptive practices, but an AI detector alone cannot establish them. With a PhD in computer science, one is more likely to have the background needed to write sensibly about AI. But that is neither necessary nor sufficient: people without a PhD can also produce excellent work, and LLMs can help, provided that there is a sound method behind their use.

This method can't be copy/pasting as there's no tomorrow, without verifying sources, without checking if the content makes sense or it's a vague statement packed to look plausible.

LLMs can help non-native English speakers to fix their sentences, or to wrap up a weird paragraph into a clear one. If AI is used to improve clarity, it's more than welcome. Maybe this is what some experts name “AI-assistance”, and I think that's perfectly legit.

In fact, this points to a deeper reflection, I guess: what does it mean to be a creator in this era?

I am considering to build my own manifest and put it in the homepage of #ComplexityThoughts, for full transparency.

I think that creating is more related to the narrative strategy. One can use LLMs to brainstorm, to challenge one's perspective, to discover missing literature worth reading. Creating means identifying the relevant things and thinking how to wrap up them into a coherent picture. In science, this process is usually sustained by interactions with peers and mentors. But now let me ask: if you had the chance to ask questions to the largest librarian ever, and you know that sometimes they can be misleading, wouldn't you take the opportunity to chat? And if, within seconds, you could have the chance to brainstorm with some MSc students or PhDs, under the assumption that they can be lazy and miss some information or point to wrong studies or even invent something, wouldn't you take the opportunity to chat?

What makes you different is the method: you ask the questions, you collect the information, you verify the sources and then you connect the dots. During this process, AI can be just beneficial to deepen your thoughts and clarify the more obscure ones.

Honestly, I don't care if some figures I see in other posts are made by AI, if they are nice and provide a clear message. As my readers know, I am an avid user of notebookLM to prepare slides and infographics that can visually support my content, since sometimes an image can be worth more than one thousand words. I have asked my readers about this recent introduction, and they were enthusiastic. Similarly, my podcasts are entirely AI-generated since I don't like my voice, I have no time to record, and I have no one to bother every 2 weeks to do that with me. This fact is explicitly acknowledged in the description of every podcast.

What do you think? I'd love to hear the most diverse views on the many shades of this matter.