I think you've done a good job critiquing peer review as a barrier to entry and as a service to paper-readers, but that's not how it really functions in practice, for better or worse. Peer review is a system by which random, anonymous paper-writers are able to force other researchers to read their work. In other words, peer review is a s…
I think you've done a good job critiquing peer review as a barrier to entry and as a service to paper-readers, but that's not how it really functions in practice, for better or worse. Peer review is a system by which random, anonymous paper-writers are able to force other researchers to read their work. In other words, peer review is a service to paper-writers. It is especially a service to paper-writers lacking prestigious affiliations and notable previous work -- authors whose papers would go unread if they just posted on arXiv. So in this sense, peer review is a path to entry, not a barrier to entry.
This is why being a reviewer is so painful: it imposes a uniform prior for paper quality, even though we know that some researchers do much better work than others, and we would prefer to focus on reading their work. This is why even peer-review is not a guarantee of quality: the process ignores said priors, and also reduces the incentive for developing a reputation as someone who only submits quality work.
At the same time, blinded paper reviewing is probably the only way to identify good work coming from new and unknown researchers. In other words, there is a precision-recall tradeoff to peer review. If reviewers were not forced to read papers they otherwise would not, precision would increase but recall would decrease.
I think you've done a good job critiquing peer review as a barrier to entry and as a service to paper-readers, but that's not how it really functions in practice, for better or worse. Peer review is a system by which random, anonymous paper-writers are able to force other researchers to read their work. In other words, peer review is a service to paper-writers. It is especially a service to paper-writers lacking prestigious affiliations and notable previous work -- authors whose papers would go unread if they just posted on arXiv. So in this sense, peer review is a path to entry, not a barrier to entry.
This is why being a reviewer is so painful: it imposes a uniform prior for paper quality, even though we know that some researchers do much better work than others, and we would prefer to focus on reading their work. This is why even peer-review is not a guarantee of quality: the process ignores said priors, and also reduces the incentive for developing a reputation as someone who only submits quality work.
At the same time, blinded paper reviewing is probably the only way to identify good work coming from new and unknown researchers. In other words, there is a precision-recall tradeoff to peer review. If reviewers were not forced to read papers they otherwise would not, precision would increase but recall would decrease.