This is one of the obvious use cases that jumped out back in 2022 what LLMs first made their big public appearance. The problem remains, I think, that the purpose isn't clear. 2,000 NPCs you can actually talk to, or 5,000, or 50,000 sounds exciting on paper - in reality, you're not going to want to talk to any of them.
Dialogue and conversations in fiction tend to be more driven; they're not really a communication between player and character, but between player and game writer. It's the same gap that exists with all AI writing: the writing itself can be technically accomplished, but if as a reader you're not connecting with a human writer it's hard to see the point.
The D&D/TTRPG comparison is interesting. In theory it does sound like it enables the computer to provide that fluid, improvised method of storytelling - but, again, the real pleasure of TTRPGs is the interaction with the DM's brain. The story that comes out of it is a by-product.
LLM-driven NPCs could still be useful for flavour: a crowd scene, for example, such as a busy marketplace, could be quite cool if as you're walking around you hear hundreds of different, contextually-relevant conversations. Although, at the same time...films and games have solutions for 'crowded marketplace' audio, because having all those conversations exist doesn't really add much to the experience.
The player is still going to be seeking out the primary NPCs that are written by humans and which are core to the plot, I think. And if the LLM procedural characters are just background flavour, it doesn't really seem worth the effort.
I might be stuck in old fashioned thinking here, of course. But if I could choose between a game that had 50,000 AI-powered characters or 5 hand-crafted, human-written characters...I'll always go for the latter.
I do wonder whether modern AI has more potential in applications like we saw in the Left 4 Dead games back in the day. They had automated systems to adjust the difficulty and to tweak the 'drama', so to speak. The action would ebb and flow, and it 'felt' like a classic action-horror. It meant that each playthrough would be different, and you could never fully predict what was about to happen. The was in 2008 (which makes me feel old), and there might be potential there to do some exciting new things.