John Goes To Washington

Last week, I went to Washington, DC to serve as a public reviewer for the National Institutes of Mental Health, on a panel that provided peer review of grant applications relating to autism research. It was a real honor to be invited, and a great responsibility. The weeks leading up to the meeting were busy for me, reading the large batch of applications.

The photo above shows me with three of my fellow reviewers. From left to right they are Anshu Barta, a pediatrician from California; Pat Schissel, an Asperger advocate from NY; me; and Aspergian author and professor Stephen Shore. We're in the lounge after a successful session chewing through applications.

As a reviewer with a personal stake in autism, I did my best to advocate the research that will provide the greatest benefits our community and our society. With all the questions we need to answer, and all the different ideas and approaches, I faced some hard decisions!

The applications covered a lot of ground. We considered applications for all manner of studies, covering statistical data, cells, animals, infants, teenagers, and adults. The researchers themselves were a diverse group, with public health specialists, psychologists, psychiatrists, neurologists, geneticists, biologists, and scientists and doctors from a host of other subspecialties.

The breadth of researchers and applications really gives one a sense of the scope of the problem we face in unravelling autism's secrets. So the issue for me was: Which efforts are likely to have the greatest public health significance and impact on the autism community?

Some research had applicability to anyone on the autism spectrum, while other work was directed certain subgroups. That made our ranking job harder, because we have so many needy groups. One thing the review process made clear to me was the tremendous diversity of the American population.

The applications I read had already been vetted by one group of scientists; the review groups like the one I participated in join the process pretty near the end of the road. My job was to provide a public perspective, and a stakeholder’s insights. For that reason, I and my fellow public review committee members were chosen for our personal connection to autism. In a sense, we provided a counterbalance to the scientists and doctors who fill the other seats on the review boards.

This particular request for applications (see http://www.nimh.nih.gov/recovery/index.shtml#autism-spectrum-disorders) has allocated to it $60 million to fund between 40-50 grants.

Now that we’ve voted, the applications will wend their way through some additional NIH processes, with the final decision on funding being made by the participating NIH institutes by the end of September. Since this money is part of the Government’s stimulus package you’ll be able to read how it got spent right next to bridge building and courthouse renovation, on www.recovery.gov. I suppose you could say this is the medical equivalent of road construction. Congress said stimulus proposals had to be shovel-ready, and in a sense, these applications are. All the research being considered is intended to deliver some kind of meaningful benefit within twenty four months.

So here’s how it happened . . .

We started at 7:50 on the morning of June 19, and finished by 5:00. Three review groups of about thirty people each tackled several hundred grant applications. Here we are, in the thick of it:

In my group there were five public reviewers, twenty-some scientific reviewers, a scientific reviewer as chair, a scientific officer from NIMH, and various staffers. The room was pretty full. The list of reviewers for each meeting is available through the NIH website, although the list of grant applications stays confidential.

Before the meeting, each application was reviewed by several scientists with expertise in the particular field. They assigned scores from 1(best) to 9 (worst). The scores of the different reviewers were averaged to form a composite score, which was like the qualifying lap time in a road race. That set the order in which we considered the applications.

That part of the job was several weeks of hard work for the initial reviewers, and even more work for the NIH staffers who had to coordinate the whole thing. Review day was the culmination of all their efforts.

We opened the meeting with the top scoring application, and proceeded down the list. For each one, the lead reviewer gave us a summary of the application, and its strengths and weaknesses as seen by the various reviewers. Then the floor was opened for questions and comments. Any of us with an opinion were free to voice it regarding the strengths or the weaknesses of the application.

Any of you who know me won't be surprised to hear I had plenty of opinions.

After all the comments were heard, the chair read us the scores assigned by the original reviewers. We were all asked to assign our own scores, which we did. In some cases I'm sure we agreed with the original scores, whereas we gave other applications significantly higher or lower ranks.

I was pleased to see disparities in the scoring from time to time, because it told me the committee members truly had their own opinions. I did my best to score them fairly, but it was a hard job
We all scored each one.

I didn't know what to expect, having never done this before, but I have to say I was pleased and I felt the process was fair. People paid attention when any of us spoke, and I'm sure our opinions did influence the voting. So our individual voices did indeed affect the outcomes. In my group, 28 of us cast votes, and it was our votes that made the final score for each application. I was pleased to note that someone spoke up for just about every application, in one way or another. In some cases they were critical, while other times they were laudatory. No one was passive.

Looking in from the outside, it seems like everyone has an opinion about what our government should fund. But when it comes time to actually choose, the job is harder than it looks. It’s like being a judge – we have a duty to be fair and evenhanded, despite our personal preferences.

It was a long day, preceded by a long two weeks getting ready. But now it's done. I'm proud to have made a contribution, and honored to have been invited.