These are some fairly generic guidelines for the review of research proposals. If you find that some portions are not relevant to the particular instance in question, use you best judgment in deciding whether or not the issues need to be discussed in your brief report.

1. Please submit your review(s) in duplicate (two copies). One copy should contain NO information that will identify you. This will be returned to the author of the proposal. The second copy is for the "granting agency" (in this case, me) and should have your name on it. Reviews need not be too lengthy. Some of the categories mentioned below may not apply, or you may have nothing substantial to say on some points. Agencies want to see succinctness. They receive plenty of information to be processed and appreciate it if reviewers do not ramble on and on in a review with a low signal-to-noise ratio. If [x] is the number written on the front of the proposal you were assigned, the title of your review should be something like:

Review of "[the title of the proposal you received to review]", proposal #[x] submitted for Economics 143, Winter 1998

2. You are not expected to do the researcher's background work if it has been insufficient. If your own expertise leads you to believe that the researcher has missed some important citations that are relevant to the proposal, however, you may point this out. In doing so, you should take care to mention these missing citations in enough detail so that the researcher can track down the paper. Many proposals are re-submitted in the next cycle of competition if the overall evaluation is encouraging.

3. Nowadays, most academic researchers are expected to have access to Econlit or analogous literature databases. Failure to have scanned Econlit or other computerized databases (if relevant) covering the existing research is not fatal, but reflects badly on the size of the investment that the researcher has been willing to make so far in the project. Degree of commitment to the topic is inferred from things like this.

4. Provide your own assessment of the extent to which the proposed research will advance the frontiers of science (in this case, economics). Indicate whether, in your opinion, the proposer's claims of the extent of his/her anticipated original contribution to the literature are clear and valid. If no clear claim is being made, this should be pointed out to the funding agency.

5. Offer the funding agency your judgment as to the social value of the research, relative to the budget being proposed (or the likely cost of the project, if no budget is discussed). Also mention the likely constituency, and whether this coincides with the researcher's claims about this constituency. (NOTE: "constituency" means the group of people interested in the results.)

6. Start by highlighting the most positive aspect of the proposal, assuming there is one. It is kindly to give positive reinforcement to the stronger aspects of a proposal before tearing into major flaws (if such flaws exist). The MAJORITY of your comments should be directed at the proposed data and the proposed model-- especially whether the specification is sensible (plausible, intuitive, or consistent with received theory). It is also kindly, after making an inventory of everything about the proposal that requires fixing, to wind up your evaluation by again briefly reinforcing the positive aspect(s) of the proposal.

7. Keep in mind what it would feel like to be on the receiving end of your comments. While most reviewing is now double-blind, nastiness in any shape can come back to haunt you. The system is hardly leak-proof. In the publishing arena, for example, editors learn that some referees can be counted upon to have impossibly high standards and to trash any paper they are sent. It is important to be as fair and objective as possible. A good proposal will identify intractabilities related to the problem at hand and will explain why a fancier model or more general stochastic specification cannot be implemented at present. The reviewer should not a priori reject a proposal because he/she would like the researcher to use an intractable approach. If the intractability dooms the project, however, rather than just compromising it to some extent, this problem should be clearly noted in the report to the agency.

8. Civility is highly desirable, without diminishing the reviewers role of pointing out, unambiguously, any deficiencies in the proposal. Failing to clearly identify where a proposal may have problems is a disservice to the agency that has commissioned your review. Reviewer's reports are not like letters of recommendation, where variations in quality must be discerned from shades of superlatives. It is crucial to convey succinctly to the funding agency your judgment about problems in the proposal, and the likely severity of those problems relative to the objectives of the proposed research.

9. Reviewers should indicate, to the best of their knowledge, whether the data (if any) to be used in the proposed research are obtainable. If the data set is widely used, and has well-known problems, the reviewer should point out these problems to the funding agency. This is especially important if the proposer has not mentioned any of the shortcomings of the data in the proposal. The PI may be unaware of such problems if they have no prior experience working with these data.

10. If the proposal in question anticipates the use of specific types of data, you should be careful as a reviewer to evaluate the likelihood of these data being available for the study within the budget period for the research. For example, if someone proposes to use data for 1960:1 through 1995:10 in their study, and you know that the government is only just now releasing 1990:3 for public use, this problem should be noted. If the proposal involves doing a survey to collect data, the budget should be sufficient to cover this. Surveys are much more expensive than most people initially imagine (if they are to be done correctly). (Note that THIS assignment did not require data availability to be verified; a good imagination would be adequate since you are not going to be called upon to actually perform the research.)

11. For future writing tasks in economics, you may want to read the lengthy how-to article by D.N. McCloskey on the art of writing in economese ("Economical Writing," Economic Inquiry; 23(2), April 1985, pages 187-222) plus follow-up commentary by Borcherding and High and a reply from McCloskey (Economic Inquiry, July 1987, pages 547-48). If you have never read these articles, or have not read them for a long time, they are worth a look. McCloskey's article has now been expanded into pamphlet form and is a required purchase in some U.S. departments of Economics. The citation is D.N. McCloskey, The writing of economics, New York: Macmillan, 1987. [ABSTRACT: Brief guide offering advice to improve clarity and readability in the writing of economics. Illustrates how the factual rules of rhetoric, style, grammar, punctuation, organization, word selection, and rewriting can contribute to the development of better writing skills for economists and social scientists. An earlier version appeared in the April 1985 issue of Economic Inquiry. McCloskey is Professor of Economics and of History, University of Iowa.]