MAMLMs: I take your point to be that much smaller LLMs than we already have are more than sufficient as natural-language front-ends to structured and unstructured data, and that the Royal Road is then applying those as queries to well-curated databases. That would imply that spending more money on LLMs is simply a waste of time. That is a very intriguing and, I think, quite possibly correct conclusion. A bigger and more complicated LLM would then just get us a slightly refined interpolation function from the space of training data prompts to the space of answers. And to the extent that those corpori are unreliable, you have not gotten anything extra:

> Marcelo Rinesi

> > Keep building bigger and more expensive models, but then thwack them to behave by confining them to domains—Tim Lee says coding, and mathing—where you can automate the generation of near-infinite amounts of questions with correct answers for reinforcement learning. That would be a tremendous boon for programmers and mathematical modelers. But expensive:

> I don't understand this claim. I.e. what DeepMind did for math Olympiads [ deepmind.google/discove… ] used the purely linguistic skills of a LLM to formalize problems and then applied (comparatively very lean, pun not intended) specialized engines to work on them.

> To my eyes that shows that coding and maths are areas where we can/should/will get advantages from AI by using LLMs to bridge between informal language and specialized tooling (which I think we can do with much smaller specialized models than what we already have) and then leveraging existing hardware and software tools to build non-LLM models for those domains; basically LLMs as parsers and things like AlphaZero-for-maths/-quantum chemistry/-etc as domain-specific compilers.

> I'm not saying the intellectual Jeeves isn't a good idea or business model, but that's like using electrical power *only* to use a conveyor belt to move pieces from manual workstation to manual workstation.