Something generative AI is terrible at: randomizing problems.

I ask it to give me some word problems for students to practice telling the difference between circumference and area problems.

It gives me the problems in alternating order: circumference, area, circumference, area.

Kids catch on to this type of stuff quickly, they stop thinking, they don’t learn anything.

(Yes, I realize I can prompt the model to randomize the order. It is still unreliable at randomizing, and shouldn’t these things be smart enough that I don’t have to prompt for every single tiny detail I want?)