Your reviewer and you
| 26 October, 2010 | Morgan Giddings |
9
|
|
Who is your grant reviewer, and what the heck do they really want from you?
If you’re like a lot of scientists I know, this is like the mystery of Al Capone’s vault (before it was opened up). But here’s an interesting factoid that may lead us to an answer: most people think they are smarter than average (though men tend to score higher on these types of self-assessment questions than women).
This is particularly true in science, where it does require some smarts to get a degree. We learn to think of ourselves as “being smarter” than the average, because we are. But if we reframe the question as, “Am I smarter than my average peer?” my guess is that a lot of us would still be raising our hands.
And herein lies the problem when it comes to you and your grant reviewer. The whole thing about “being smarter” is tied up with our ego.
Our ego doesn’t like being attacked. So when we get a rejection, the ego raises the red flags, and starts beating the war drums. It lowers the portcullis, saying things like: “Obviously my reviewer didn’t get it because they drew a short straw when it came to the brains department.”
It’s okay. Nearly everyone’s ego experiences the raised defenses from time to time. The trick is in whether you can step back and look at these defenses rationally, or whether you succumb to them, and let it show in the way your rewrite your proposal the next time.
And if we look at it rationally, Dr Ego’s math doesn’t work quite right. If we’re all smarter than our reviewers, then the NIH, NSF, and other funding agencies must be going out of their way to pick “less intelligent” reviewers than average. But that ain’t so: those organizations work hard to find the very best people they can.
Often, they don’t get the “famous” scientists because those famous people are too busy doing their own work and promotion to do a lot of reviewing. But I think you’d be hard pressed to equate fame with intelligence.
So Dr Ego’s equation just won’t balance. And the reason why is that we ourselves are are reviewers. Can we be smarter than ourselves? Can the statement 2 > 2 be true?
The bottom line is that I’m telling you to ignore your ego when you get angry at your reviewer. Let it boil off for a day or two, then see it for what it is—just a defensive reaction to rejection.
The key then lies in the next step. If you’re able to drop that ego and move on after your temporary setback, then the rejection holds some kind of lesson for you. If you approach it with an open mind, it will make you a better grant writer each time it happens.
It’s not always easy. Sometimes figuring out “why” a reviewer rejected you is really hard, because they’ll often cloak their true reaction in a lot of logical sounding words. We all do that, because if we said what we were really thinking, we might get in trouble with the agency (e.g. “This idiot can’t write worth a damn…” may be the thought we have, but we’ll never put that in a review).
But realize, there’s always a reason, and the clues are there for your investigation.
If you want to find out more ideas about what the reviewer really wants, grab my “Backdoor to funding report” over here.
——————-
Morgan Giddings is an Associate Prof. at the University of North Carolina Chapel Hill, focusing on proteomics, bioinformatics, and lots of other fancy sounding “ics” words. She is into figuring out how to optimize performance in a science career while maintaining a “real life” at the same time.
|
|
previous post
Very helpful comments. The narrative reads as if this is from Dr Giddings’ own personal experiences. An enjoyable read and sound advice. I suspect both male and female scientists possess similar egos that are hurt or damaged in much the same way. However, the getting over the damaged ego advice should not compensate for some frankly poor peer-reviewing that is produced by some of the funding agencies. This is where humility plays a role in the response of an applicant to a reviewer’s critique so that the reviewer’s ego is similarly not hurt by your response (unless of course reviewer’s are emotionless automotons which of course they are most certainly not). This could of course result in a knee-jerk recommendation for rejection of your grant, which would of course be both unethical and most likely professional misconduct. While certain standards are expected of the applicant, let’s not lose sight of the standards of due process and professionalism expected of the reviewer !!
One of the big issues for grants and for manuscripts as well. Remember, grants are to get cash so we can do the experiments and ultimately publish papers. Many think, similar to grant reviewers, that reviewers of their manuscripts are also a bit less intelligent, especially in poorly communicated manuscripts.
In the end, it is important to be a bit more pragmatic and save the hurt feelings for another time.
It is axiomatic that 50% of scientists are smarter than the other 50%. It is similarly axiomatic that 90% of scientists are not in the top10% in the smarts department.
I suspect the fact that Mr. Giddings hasn’t grasped the implications of this may be indicative.
Certainly, there are situations where lousy ideas and lousy writing with ridiculous conclusions get rejected. But clearly, the odds are high that the top 10% of scientists’ ideas will be rejected by those in the lower 90%. There are so many examples of this, from the rejection of H. pylori even after data was generated to bouncing the manuscript for the scanning, tunneling microscope.
Reality is, look at your stuff and be objective. If you’re right and they’re wrong, then keep going.
I review a lot of grant applications and articles for organizations across the world (I have worked in four continents, in five countries, and have ties with many more). I don’t publish a lot because I have very high standards. I get asked to review subjects in many different fields. My own work is interdisciplinary. I assume the reason I get a lot to review is that I give good reviews. I don’t pan most; in fact most I approve, but I am honest about problems and any doubts I might have. Usually my opinion is accepted, even if there is a contrary review.
What I want to say now is that I often get resubmissions, either under a similar or different grant. Most of these are vastly improved. If you get a rejection, you may have much better chances if you work hard to improve your request. This may take more than once, as I have seen.
If you think yourself that there is a problem with your proposal, then there probably is. Sometimes you can’t tell without the benefit of a negative reaction. It might turn out that if your idea is bad, you might let it go. But if not, then maybe you will be among the returns I see. I am considered pretty radical in my central profession, philosophy, so I am not unaware of the problems with bringing new ideas. The filter of review helps very much, I think, in making new ideas into even better ideas.
the H pylori issue actually illustrates the case quite nicely. if you’ve read the literature in the field as i have – i teach a course on the science of the Nobels where we work from the primary literature. so with regard to that one, if you look into it from that perspective you will quickly realize that yes, marshall and warren were on to something, and yes they were right. that’s not the point. the point is that their initial studies were flawed and the papers were in general terribly written and conceived. that prize stands out among all recent Nobels as having a very painful primary literature
i haven’t read a biography of this, but i would not at all surprised if someone hadn’t take take them aside ans said exactly that: “nobody will ever believe you even if you are correct, until you can publish one decent paper” . that was when they did the Lancet work and convinced people they were onto something.
so clarity of writing and well conceived and executed studies actually are important.
Anything being reviewed needs good content AND to be communicated clearly, and authors are usually blind to the imperfections in their brainchild, but..
Clearly many reviewers don’t have much time and skim through the material.
Some published papers really do have impossible results and/or unjustified conclusions and silly statistics that reviewers didn’t notice, even in Nature.
Destructive criticism is easier than constructive.
Some reviewers do make unjustified and tangential objections, suggesting that they hadn’t thought through their comments.
The 90% above, getting an unusual idea, are unlikely to distinguish off-the-wall genius from fantasy.
Some reviewers get an ego boost from occasionally anonymously lashing out.
Some reviewers are not expert in everything and may not see a relevance or detail that is fairly obvious to a specialist. Authors should spell it out.
Some reviewers are keen to keep their place in the field, and both put down competition and pinch ideas. An administrator in a charity once told me that “X referees all papers on organism Y” and implied that only Xs buddies were likely to be successful.
Don’t get worked up about it and expect to change their minds, because referees, like most people, especially egotistic scientists, find it very difficult to change their minds, let alone admit to an oversight or mistake.
I am writing in response to Ellen Hunt. Very nicely put. The major problem with peer-review is that no matter how much we try to improve the process, it is still a very subjective process. The handling or associate editor would in most cases, assemble a consensus opinion from the reviewers comments (2, 3 or may be 4) and detail these to the authors as a consensus rejection, major revision or minor revision/accept decision.
Essentially we are now peer-reviewing Dr (Ms) Morgan Giddings’ article and she has attracted many different viewpoints. That is the nature of subjective peer-review and critique and comment.
However, on re-reading the article, I disagree with Dr Giddings personal viewpoint that we are all, by our very nature, that egocentric that we think the reviewers are stupid and we are smarter. I think this is dumbing the point a little. One contributor above, made the point well. If you think you are right (e.g. your hypothesis) and your ideas are good and the data collated thus far supports a working hypothesis, stick to your guns, accept the [hopefully] positive comments of the reviewer and try again. Often reviewers do get it wrong, not because they are dumb, but because they have missed something or not as familiar with the field as you. In that situation, take advantage of all the positive comments the reviewer is offering, and defend [politely] the data and ideas you have worked so hard to establish.
After all, we are all human, we all have feelings and we are here to help each other. Often when I read Dr Giddings’ articles here, I sense she is too centric to the business-side and competitive dog-eat-dog nature of scientific endeavors. Perpetuating such an opinion, adds to an acceptance of such attitudes and detracts from all of our responsibilities to mentor, nurture and take more accountability for our collective pastoral responsibilities.
At times it is good to get a little bit of humor on the grant review topic. See this:
http://www.youtube.com/watch?v=Vn4e6LLc0wI&feature=player_embedded