Someone did the most wonderful experiment to see if LLMs can indeed think out of distribution. Same one I and many others have been dreaming about. The results may surprise you!
michaelhla.com