If the experiment includes only fMRI, yeah, it has a high likelihood of being worthless. Part of the reason is that all those studies were n=6, and everyone doing similar size studies should be the first up against the wall when the revolution comes.
But also, these failures are not the fault of the fMRI itself. Those nuclear spins flip w…
If the experiment includes only fMRI, yeah, it has a high likelihood of being worthless. Part of the reason is that all those studies were n=6, and everyone doing similar size studies should be the first up against the wall when the revolution comes.
But also, these failures are not the fault of the fMRI itself. Those nuclear spins flip with pretty high reproducibility. It’s the analysis of the fMRI data that’s causing the issues. It’s the interpretation.
If someone shows me a second analysis, independent of the first, that agrees, I’m totally down for the fMRI results. Even if the second technique is also mediocre reliability like the fMRI analysis, if it really is independent, the odds of concordance by chance are low. Not impossible, of course - we still have to deal with bad incentives that lead to fabrication, for example. But man, the problem is smaller.
As a corollary, if you are developing a claim that can only be validated using a single method… maybe we should ignore you in that basis alone. Too many researchers understand the techniques of data collection but not the process of science. We need to layer on a few simple rules to enforce some science on people. My vote that “every interesting claim needs validation by an independent technique” would actually get most of the job done.
If the experiment includes only fMRI, yeah, it has a high likelihood of being worthless. Part of the reason is that all those studies were n=6, and everyone doing similar size studies should be the first up against the wall when the revolution comes.
But also, these failures are not the fault of the fMRI itself. Those nuclear spins flip with pretty high reproducibility. It’s the analysis of the fMRI data that’s causing the issues. It’s the interpretation.
If someone shows me a second analysis, independent of the first, that agrees, I’m totally down for the fMRI results. Even if the second technique is also mediocre reliability like the fMRI analysis, if it really is independent, the odds of concordance by chance are low. Not impossible, of course - we still have to deal with bad incentives that lead to fabrication, for example. But man, the problem is smaller.
As a corollary, if you are developing a claim that can only be validated using a single method… maybe we should ignore you in that basis alone. Too many researchers understand the techniques of data collection but not the process of science. We need to layer on a few simple rules to enforce some science on people. My vote that “every interesting claim needs validation by an independent technique” would actually get most of the job done.