The app for independent voices

Chatbots are not rule-based. They're a purely statistical model. The whole algorithm is basically: given this prompt (input text), predict the top N most likely words or phrases according to this gigantic statistical model built from all the text you've ingested, then select one of those results at random and append it to the prompt.

Then take the prompt plus the newly generated text and repeat, ad nauseam, until you run out of memory and start cutting text from the top. That's it, that's all there is to it. It's just mad libs with a massive amount of data compressed by fancy statistics.

Nowhere therein is anything resembling rules (let alone consciousness).

The model itself is a big pile of numbers that represent the likehood of text B appearing, given text A precedes it, for all possible combinations of text, with lots of compression and the edges shaved off.

The actual machine learning - "AI" comes in building that model, not in applying it to generate text, which is fairly routine software (and why it can run on a relatively modest computer, as opposed to the "training" which takes many very powerful computers a very long time). The reason we use machine learning here is that it's the best - or at least the easiest-to-use - tool for developing a reasonably accurate collection of statistics across a huge pile of data and then compressing it into something "reasonably sized" - i.e. multiple terabytes instead of countless exabytes.

All this jargon - "machine learning", "artificial intelligence", "training", is just comfortable metaphors for describing how this software works. It doesn't imply that something human-like is actually going on in there.

Anyway, TL;DR: sadly your intuitions are incorrect. It's actually much, much dumber than that.

Jun 4, 2023
at
2:17 PM