The app for independent voices

a good question and the answer is purely technical, no bias or politics involved. I will use a much simpler neural network (NN) model to explain the concept - it is not really real or useful, just a didactic/rhetoric tool.

Suppose a NN to classify images into cat and non-cat. Neurons look into a group of pixels, and the first layers of neurons are as large as the image, then the next layers have less and less neurons, until we arrive at a vector of neuron activation values - think on it as a list of numbers, but there are much fewer numbers than pixels - an internal representation

Based on the values on this vector, we classify images on cat x non-cat. The NN learns statistically how to weight each neuron inputs to do that.

Now, we want to generate a fake cat image.

Generate a vector value close, but different, to those we know from true cat images.

and run the NN backwards, estimating possible values of the neurons backwards from the internal representation to the original image layers - we got another image which would be classified as a cat. Hopefully, it looks like a cat. Sometimes, it does not, the statistical learning is never perfect. When it generates something that does not look like a cat, we say the NN hallucinated.

chatGPT's NN is way more complex and convoluted than that (look for a paper called "Attention is all you need", it will give you a good idea). But the general idea of generating internal representations, and extrapolating backwards into a new original input, is there.

Jun 23, 2023
at
5:46 PM

Log in or sign up

Join the most interesting and insightful discussions.