One voice among thirty - why one person's vote matters

It's been a week since I announced that I was joining the review boards at Autism Speaks. Many of you supported me, but a few of you doubted my ability to exert much influence as the lone Aspergian on the board.

In particular, people have suggested that one person’s vote can’t make any difference to thirty scientists who share a review board with one misfit (me.) The implication is, the thirty scientists vote as they wish, and my lone vote is ignored. That perception is totally wrong, and I’d like to take a moment to show you why . . .

To understand the significance of one person’s vote, please join me as we review and rate some promising (sample) medical research. I’m describing the general process of review on the Autism Speaks boards, but the process is essentially the same for the NIH and other government agencies, and many big private foundations.

The process starts as research proposals pour into the agency. The first step is to ensure applicants met all the ground rules to qualify for the funds at hand. Agency staff does that work. This first screening may weed out half the proposals. The rejected proposals go back to the researchers with various notes and suggestions. Many researchers clean up their applications and come back again so this is really an iterative process.

Next, the proposals that made it through the initial screening reach the review board – the place I serve. Proposals are dealt out to members of the board for a first ranking. Much of the time, three reviewers read each proposal. They may be assigned randomly, or they may be dealt out by expertise. However they are allocated, if there are 30 of us on the board, and there are 100 proposals to deal with, we will each be assigned ten.

We rate the proposals we are given in several areas, like the impact on the community, how likely the work is to succeed, and whether it’s truly new research or a rehash of something already covered. Each area is scored from 1-5, or perhaps 1-7. So a proposal that I (or any of us) rated 4,4,5,5,3 in each of five areas would have a composite score of 4.2

The three initial reviewer scores are combined for a total score, and the proposals are ranked based on this first pass. At that point, staffers take the funds available for allotment and they see how far down into the ranks the money goes. For example, if we have twenty million dollars to distribute, that might be enough to fund the top third of the applications.

Given that, the agency takes all the proposals in the top third, plus a cut of the next tier, for final review. That’s where we all discuss them, and we all vote. And that’s where any one voice can really matter a lot. I’ll give you an example. Let’s say a piece of research involves social skills training, and most of the scientists gave it a 3 for importance. But I feel that it’s a really important proposal, based on my life experience, so I speak up. By doing so, I cause people around the room to rethink the proposal’s importance, and a number of people move their score from 3 to 4 or even 5. The result: that proposal’s average score rises, which moves it from “not good enough to fund” into the “recommended for funding” category.

The same thing can happen in reverse. If I said a particular proposal was irrelevant, its score might drop, and it would sink in the ranking allowing other more important work to be funded.

Any of us have the power to move ranking in this way. You might say our influence is directly proportional to how much we believe and our oratory skill.

The idea that I would be ignored or bullied – as some have suggested – is simply not realistic. Any of you who’ve met me in person know I’m six foot three and loud, so I’m hard to ignore. And these are gentle people. As Marlon Brando suggested in The Godfather, they are receptive to reasonable arguments.

The system I described varies a little bit from one agency to another but I think you can get the general idea. In the first round, the proposals are graded by teams of three, and the size of the entire board really does not matter. In the second round, a persuasive speaker can move a room of fifty people as effectively as he can move five people. So once again, the size of the board is not really indicative of any one person’s importance.

I guess being a reviewer is really what you make of it. If you’re meek as a lamb your vote won’t change anything. But if you are passionate, and articulate, you can really accomplish a lot.

And that’s what I intend to do, every time I vote.