Research Design Peer Review Form Peer Review: Fill & Download for Free

My co-reviewer told me he was shocked by my… er… viciousness.Early on, too eager to prove that I was a thorough critical thinker, I made the mistake of treating peer review like a brutal journal club designed to tear apart the paper and propose several new and complex “future directions” experiments that could easily form the basis of a PhD project. (Needless to say, my PI did not incorporate much of my feedback into the final reviews.)Looking back, I would advise my past self to read Neil Menon's answer, Raziman T.V. (റസിമാൻ ടി.വി.)'s answer, and:It is not your job to propose ways for their Journal of Cell Biology report to become a Cell article. Do not compare them to the paper they could have but didn’t write. The novelty and significance should be appropriate to the journal and type of article, but you don’t need to demand more.(obligatory sad Lego Grad Student)Your main goal is to ensure that the claims are properly supported by the data, which is interesting, even fun. Are the experiments necessary and sufficient? Are the results interpreted correctly, or are the authors extrapolating too much? What alternative explanations could there be for the results? Did the authors draw the right conclusion from this citation? It’s a nice puzzle game in a way. Where the claims outpace the data, you can ask either for the claims to be rephrased, or for additional data or citations to be provided.You should provide constructive criticism and see yourself as a helper, not an opponent — even if that means helping them to the realization that they should spend more months on additional experiments or submit elsewhere.Interdisciplinary papers tend to have a bit of a blind spot in their secondary discipline. You may need to explain to the computational authors how to phrase their description to be of interest to biologists, or introduce the physicists to some experiments that might be common knowledge in your field. Phrase these comments constructively, not accusingly.Peer review isn’t just about looking for problems. Praise what you like about their work and what you see as their main points of novelty or impact.We’re all on the same side here. Good science should be shared in a timely manner, and people need to publish in order to progress as scientists. It’s possible to be rigorous while also being kind.

I'm by no means an expert on running a journal, and I have the utmost respect for editors experimenting with new approaches in an institution as old as academic publishing. That said, here's my take on some key areas to address.Academic publications generally go through two stages: peer review, and editing and formatting. Duration during the former is captured in the time between submission and acceptance.Source: Satoshi Village (a great exploration of publication data)Of course, if the paper is rejected it will likely go through multiple iterations with a similar duration.Duration during the latter is captured in the time between acceptance and publication:Source: Satoshi VillageIt's clear that the time spent on peer review (~150 days) accounts for the majority of the publication process, but it's important to note that there can still be substantial time involved after acceptance (~30 days). Consequently, the problem is that academic publishing combines all the major challenges of survey research and the publishing industry.Designing peer review like a surveyI say academic publishing faces the same challenges as survey research because I see both as a problem of incentivizing participation without vested interest. The incentives behind survey response have been well studied:Research suggests three main reasons: altruism (e.g., the survey furthers some purpose important to the respondent, or the respondent is fulfilling a social obligation); survey-related reasons (e.g., respondents are interested in the survey topic, or find the interviewer appealing); and egoistic reasons (e.g., I like it; the money)Source: Do incentives exert undue influence on survey participation?These are very similar to the incentives behind peer review (financial incentives aren't implemented, but there is a form of academic currency exchange with citation pushing during peer review). And like with peer review, surveys often run into the problem of non-responders: individuals from a rare population and/or individuals who are less likely to respond.One way survey researchers have tackled this problem is the use of paid research panels; however in academia this would create an undesirable paywall for faster publication times. Nature Scientific Reports actually tried an analogous approach recently with a paid fast-track option -- it was a bit of a PR disaster.I think a more appropriate direction would be to simply gather and centralize information on referees to better match reviewers with topics they're interested in (survey-related incentives). To my knowledge, currently editors heuristically search for referees based on their previous publications and review history with the journal. However, data-driven generation of ranked lists from a curated database of potential reviewers would go a long way in rapidly identifying interested referees, especially early career researchers with less renown like postdocs. Accounting for topics that were previously declined and those that were accepted/returned in a short time would also be useful for improving matches with referees. Like targeted advertising for academics.It's both promising and disillusioning that my Google Scholar updates have done a better job of identifying articles I'm interested in than the review requests I've received.Publishing challengesThe other side of the coin is the logistics of publishing. These are things like layout, copy-editing, and dressing PDFs. I consider the time spent here to be easier to resolve. For instance, Elife has a much shorter median publication time (3 days) compared to PloS journals (40-60 days) simply by allowing early access to a pre-formatted article PDF. ACS and RSC journals offer similar services. I don't really see why this shouldn't be the norm, but would love to hear the argument against early access.

It’s complicated.Part of the reason Sci-Hub exists is that the owner of many science journals, Elsevier, refuses to pay the people who publish papers in their journals, but also claim copyright on those papers, and sue the authors of the papers for doing things like putting their own research on their own Web sites.Elsevier, which made about $2 billion last year, does not pay for articles, does not pay peer reviewers, but takes research paid for by government grants and sticks it behind paywalls or in outrageously expensive journals. They’re making billions of dollars from other people’s work, which they get for free (and which is often paid for by the taxpayer), and charge predatory prices to access it.This problem has become so bad that a lot of universities are canceling their subscriptions to Elsevier journals, and last year LSU sued Elsevier for breach of contract. (The university paid Elsevier $1.65 million per year for online access to its electronic journals. Elsevier demanded another $165,000 per year in subscription fees for the university’s college of veterinary medicine, even though the $1.65 million/year covered all the university’s students, including their veterinary students.)To compare it to movie pirating:When a studio makes a movie, they pay the writers. They pay the actors. They pay the editors, the cinematographers, the directors, the effects technicians, and so on. Then they charge the audience to see the movie. If you see the movie without paying, you’re taking money the studio uses to pay its writers and actors and directors and all the other people who make the movie happen.If Elsevier made movies, they would demand that the writers, the directors, and the actors work for free. They would refuse to pay the cameramen, the editors, the set designers, and the effects technicians, because paying them would destroy the artistic integrity of the movie.The only things they would pay for are film and advertising. They would charge $800 for a movie ticket (or $36 per hour to stream a movie online), and keep all the profits. An actor or a director who talked about a movie at a convention or on a talk show would be sued. College students wouldn’t be charged to see movies, as long as the university agreed to pay $1,650,000.00 per year for a student license.