Regarding alternative publications, Fred Driscoll at UCSD used to ask: "Why no journal of negative results?" This was a fun thought exercise back then, but seems like a more serious thing we ought to consider in a world in which AI agents are available to assist with literature review, verification, validation, and replication.
Also Caltech as an institution is worth considering as a baseline for an alternative research organizational structure that helps reduce the insider/outsider effect. The core idea of Hale, Noyes, and Millikan was to break down the rigid silos that separated scientific disciplines at traditional universities. They believed that the most important future discoveries would lie at the boundaries between fields. The relatively small number of divisions (6) forces collaboration across disciplinary boundaries and encourages breadth over specialization. Caltech is arguably one of the most successful experiments in human progress executed in the 20th century.
Regarding the topic of the article, I found your distinction between model generation and refinment so insightful. It makes me wondeer, how do you see AI 'outsiders' influencing new model architectures versus established research teams refining them?
Thanks, just saw this! I think it’s a good question… I don’t have an immediate answer but I’m doing a little work on related questions that I’m hoping will be out in a few months.
Excellent article, though I think it understates how often technology precedes scientific understanding rather than following from it.
Steam engines preceded thermodynamics by decades. The Wright brothers succeeded through empirical testing while contemporary aerodynamic theory misled them. Transistors emerged from Bell Labs tinkering before detailed theoretical understanding. LLMs work remarkably well despite our still-limited grasp of why.
The irony is that your "outsiders"—the draper observing bacteria, the clockmaker solving longitude—weren't generating theoretical paradigm shifts. They were building things and making observations. That hands-on work *is* knowledge creation that often forces theory to catch up.
This strengthens rather than undermines your argument: we need outsiders not just for new theories, but to tinker and discover what actually works.
Thanks. I don't disagree with you; I think there is no real line between tools, discoveries and ideas, to paraphrase the fine Sydney Brenner. When I refer to paradigm shifts in the piece they are intended to imply a vision of 'models' that also encompasses technology and tools; perhaps this could have been clearer. - Alvin
Wonderful summary.
Regarding alternative publications, Fred Driscoll at UCSD used to ask: "Why no journal of negative results?" This was a fun thought exercise back then, but seems like a more serious thing we ought to consider in a world in which AI agents are available to assist with literature review, verification, validation, and replication.
Also Caltech as an institution is worth considering as a baseline for an alternative research organizational structure that helps reduce the insider/outsider effect. The core idea of Hale, Noyes, and Millikan was to break down the rigid silos that separated scientific disciplines at traditional universities. They believed that the most important future discoveries would lie at the boundaries between fields. The relatively small number of divisions (6) forces collaboration across disciplinary boundaries and encourages breadth over specialization. Caltech is arguably one of the most successful experiments in human progress executed in the 20th century.
Thanks! Agreed on the negative data, and thanks for the Caltech example, which I agree is a good case study
Regarding the topic of the article, I found your distinction between model generation and refinment so insightful. It makes me wondeer, how do you see AI 'outsiders' influencing new model architectures versus established research teams refining them?
Thanks, just saw this! I think it’s a good question… I don’t have an immediate answer but I’m doing a little work on related questions that I’m hoping will be out in a few months.
Excellent article, though I think it understates how often technology precedes scientific understanding rather than following from it.
Steam engines preceded thermodynamics by decades. The Wright brothers succeeded through empirical testing while contemporary aerodynamic theory misled them. Transistors emerged from Bell Labs tinkering before detailed theoretical understanding. LLMs work remarkably well despite our still-limited grasp of why.
The irony is that your "outsiders"—the draper observing bacteria, the clockmaker solving longitude—weren't generating theoretical paradigm shifts. They were building things and making observations. That hands-on work *is* knowledge creation that often forces theory to catch up.
This strengthens rather than undermines your argument: we need outsiders not just for new theories, but to tinker and discover what actually works.
Thanks. I don't disagree with you; I think there is no real line between tools, discoveries and ideas, to paraphrase the fine Sydney Brenner. When I refer to paradigm shifts in the piece they are intended to imply a vision of 'models' that also encompasses technology and tools; perhaps this could have been clearer. - Alvin
A superb article.
Thank you for writing this piece, I enjoyed it a lot.
YEAH OUTSIDERS!!!!
Substack is also great for exploration stage ideas