I'm impressed. There are real caveats in what quantifies as novel in evaluating the results of AI guided materials discovery and the author covers this issue.
The fact that AI increases novelty can be interpreted in two ways. One possibility is that the model is simply adept at generalization, exploring new parts of the materials design space. The extent to which neural networks can generalize is an open question, but recent evidence suggests that such capabilities are possible (Zhang et al., 2024). Alternatively, this finding could primarily reflect human limitations in the absence of AI, with scientists adhering even more closely to familiar templates. There are three caveats to these results. First, they do not rule out the possibility that AI increases novelty within the neighborhood of what is known but reduces the likelihood of truly revolutionary discoveries. This connects to a broader concern about the quantitative study of science: if progress is driven by outlier breakthroughs, empirical analyses may be misleading due to the inherent scarcity of right-tail discoveries (Nielsen and Qiu, 2022). Second, I do not observe the ultimate impact of the lab’s inventions. I track patent filings but lack data on citations or subsequent innovations.
h/t @calebwatney
