Most of these blogs are about health and evidence issues, but occassionally they will just relate to the processes of research. So this one is about responding to peer review, which most researchers find traumatic as captured in Nick Kim’s grim cartoon below.
When I’ve asked folk about this, their response to reviewer comments ranges from a disheartened sigh to thoughts of homicide or suicide. Tactics to deal with reviewers include putting them in the bottom of a drawer, asking a friend to read, or simply venting about their stupidity to a colleague. All fine, but eventually we need to do something. Maybe it can’t be converted into the equivalent of 6 foot glassy surf, but it can be made less painful.
The first issue is whether you need to respond to all comments? Given your expertise in the topic is likely to be at least as good as the reviewers, there is a reasonable chance they have some things wrong. So I generally aim to make a change for about 80% of comments. Spelling corrections and the like are easy – just say “thanks”.
Misunderstandings are harder, but may mean you could be clearer: think about rephrasing or rearranging. Valid criticisms are the hardest and most important – chew over these carefully, preferably with a colleague. Finally, if a criticism is clearly wrong and your text is already clear, then explain this and point to the text they may have missed. Now to the process.
Doing hard tasks is always easier with a friend. So here is what I suggest:
Organise an hour or so with a co-author or colleague to talk through the comments. Here is one technique (see steps below). Before the meeting edit the reviewer comments so issues are all on a separate line and numbered (only some reviewers do this). With these reformatted comments either printed out on a large sheet or on a data projector, go through the items one by one. Which of 4 categories are they in (simple correction, rephrasing, major criticism, or incorrect criticism that you have clearly explained)? Together discuss these (venting a little on the way – part of the fun 😉 and write a draft plan of the responses. Doing this with someone else eases both the emotional and intellectual burden – and almost be fun.
Circulate to all authors to agree on the plan. Then there is the hard work of making the actual changes, but at least it is now broken down into chewable pieces that you might do an hour at a time. And remember that the evidence suggest this process does improve the quality of presentation of papers, so it is probably worth the pain and effort.
The 7 STEPS
1. Organise an hour+ with co-author(s) or a colleague
2. Edit the reviewer comments so that each issue is on a separate line and numbered. Begin with any Editor’s comments.
3. What category is each (simple correction, rephrasing, major criticism, or incorrect criticism)?
4. Discuss and write a draft plan of the responses and changes.
5. Circulate to all authors to agree on the plan.
6. Make the changes (then cross check that these still match your planned changes)
7. Circulate the revised paper to all authors; resubmit; buy the champagne so you are ready for celebrations.
SUGGESTED FORMAT OF THE RESPONSES
Set out the response to reviewers in a 3 line format (can be done as a table also, but that can be messy).
X. “Reviewer comment” (numbered)
RESPONSE: We agree that …. explain
CHANGES: specify any revisions and where it is, or (occasionally) say “NO CHANGES” or “see response to Review X comment Y) If the change is short, include the full change here, e.g, “randomised trial” has been changed to “randomized controlled trial”
If the change is lengthier, then describe how and where, e.g, “2 sentences have been added to the first paragraph of the discussion”
Note: Responses can be set out as a Table, but the above is equivalent, simpler, and can be pasted into websites if needed.
Good luck. And I’d welcome any comments on ways to improve this process.
Healthy Evidence 
Excellent suggestions Paul, I will keep a copy and give them to my young colleagues. It is very typical to observe how angry and desperate a young researcher can be when getting a critique on his/her first paper for the first time. I could simply divide the “major criticism” category into two subcategories, namely “requiring some reanalysis of the data” or “not requiring reanalysis of the data.” The first category tends to elicit a specific reaction of desperation, as the researcher realizes that he or she is going to have to dig up data files and statistical commands that have been buried for at least a year. As a supervisor, I tend to deal with these remarks first with special attention, before setting other, more editorial remarks. But maybe I’m wrong, maybe it would be better to deal with the easier points first and end with the more demanding ones (should we randomize the two strategies? 😉
Friendly regards – Thierry
Thanks Thierry – the “requires new analysis” is a useful distinction. Will add that. Would also ask is that suggested new analysis correct and worthwhile?