Any recurrent network definitely shows oscillatory behaviour - I used to train GRUs (simpler variant of lstm) a while back for sound generation. Part of the difficulty of using them was how unstable they could be - things would work really well for a few seconds, then just degrade into noisy or weird feedback loops..showing classic chaotic behaviour. I managed to get some small ones running in realtime on fft data, with live input bled in from a microphone and the state feeding back into itself. You could drive it in and out of different stable points with different inputs, much like microphone feedback in a room if you've ever played around with that.
I think this is prevelant in almost any recurrent system, which raises the question of how our brains don't just collapse into noise or get stuck in endless loops...they're kind of finely balanced to be able to skirt around the edges of various attractors, moving from place to place.
Any recurrent network definitely shows oscillatory behaviour - I used to train GRUs (simpler variant of lstm) a while back for sound generation. Part of the difficulty of using them was how unstable they could be - things would work really well for a few seconds, then just degrade into noisy or weird feedback loops..showing classic chaotic behaviour. I managed to get some small ones running in realtime on fft data, with live input bled in from a microphone and the state feeding back into itself. You could drive it in and out of different stable points with different inputs, much like microphone feedback in a room if you've ever played around with that.
I think this is prevelant in almost any recurrent system, which raises the question of how our brains don't just collapse into noise or get stuck in endless loops...they're kind of finely balanced to be able to skirt around the edges of various attractors, moving from place to place.