The road to acceptance without revision: Another old journal editor/referee paper story

At one point in my life I thought it would be cool to publish a paper in Human Dimensions of Wildlife. So I sent this paper there in 2002 and received an R&R. Here is the response to the referees:

Response to Referees

The empirical models have undergone significant change. When incorporating the trout fishing, deer hunting, and wildlife watching models into the analysis we began exploring an expanded set of independent variables. This is important because controlling for all possible factors that may affect the responses is necessary when searching for differences in the 1991 and 1996 survey. Also we no longer present willingness to pay estimates for three reasons: (1) the regression models are sufficient to conduct the hypothesis test for temporal reliability, (2) brevity and (3) these estimates are presented in detail in the previous analyses of the Fishing, Hunting, and Wildlife-Associated Recreation (FHWAR) data. We have completely rewritten the paper to improve the clarity of our presentation. We address specific comments of each referee below.

Referee #1 (Comments on the HDW Review Comments Sheet)

Paragraph 1: The empirical approach has been simplified to avoid confusion. Our empirical model is the probit to maintain consistency between the previous analyses of these data. An expanded discussion of the censored probit model has been added.

Paragraph 2: We have rewritten the conclusions to increase the clarity of discussion about temporal reliability.

Paragraph 3: We have dropped discussion of the catch per trip as a potential influence on willingness to pay. We have included a discussion of potential omitted variables including demand and supply factors not included in the survey data.

Paragraph 4: The entire paper has been rewritten to improve clarity.

Referee #2 (Comments on white paper)

Paragraphs 4-6: We have focused on the analysis that adjusts the sample for the bid distribution and state of residence. We have discarded from both surveys respondents who took trips outside the state of residence. In the bass fishing, trout fishing, and deer hunting models the average (inflation adjusted) bid amount in the 1991 survey is lower than in the 1996 survey. We have randomly discarded respondents so that the average bid amounts are not statistically different across year of the survey. The adjustments to the sample are described in the “data” section of the paper. Note that the empirical results are robust to samples in which these two adjustments are not made.

Paragraph 7: The word “trends” has been purged from the paper.

Paragraph 8: We have expanded our analysis to include each of the contingent valuation questions that appear in both the 1991 and 1996 FHWAR surveys: bass and trout fishing, deer hunting, and wildlife watching.

Paragraph 9: Appendix 1 and 2 are deleted. We provide more discussion about the survey questions, including the exact text of both the 1991 and 1996 questions, in the “survey background and methods” section of the paper. We focus on the log normal functional form stating that similar results are found for other functional forms of the bid amount.

As you can see, they asked us to do a lot of work, including re-estimation with more dependent variables than just bass and trout fishing. The editor rejected the revision:

The reviewers suggested refocusing the manuscript to highlight how methodological changes in the survey over the years have potentially influenced the findings and conclusions. Both reviewers note specific recommendations in this regard. Because the topic of temporal reliability is important for surveys like FHWAR, I encourage you to revise and resubmit the paper along the lines they suggest. In particular, you will need to include the 2001 FHWAR data.

I used the 1991 and 1996 data in the paper and the editor thought it would be a good idea to update with another year’s worth of data. I didn’t think so, especially since the 2001 data asked a completely different type of willingness to pay question (open-ended). And, after reading two versions of the paper, the referees thought that the contingent valuation data wasn’t worth the trouble. Referee 1:

The strength of the paper lies not in the probit model, but in highlighting how methodology can influence the findings and the conclusions that can be drawn. A probit model might be still be used, but the variables in the model should be methodological variables (type of question asked [dichotomous choice versus iterative bid], recall period, etc.), rather than variables like PUBLIC, PRIVATE, PHOTO, etc.

Referee 2:

My view is that the authors should forget their probit analysis and concentrate on whether there appear to be “conventional” reliability and bias problems that make comparison of 1991 and 1996 likely to be a comparison dominated by bias or error rather than showing validity. …

My personal view is that there is probably enough in the current paper about changes and incompatibilities that the paper can be easily revamped along the lines suggested. In fact, at one point it seemed to me that the authors were going to study the distributions of responses for 1991 and 1996 and discover that response patterns were such that testing for temporal reliability made no sense.

In short, both referees thought a paper with a different focus would be a good idea and the editor agreed. As it was, the different paper wasn’t something that I wanted to write since it was really beyond my research interests. So, I sent the paper to JARE in 2002 (reject), the Journal of Leisure Research (JLR) in 2003 (reject) and Ecological Economics in 2005 (reject). I think it was a JLR referee who was so incensed at our analysis that he threatened to write a comment on the paper if it was published. After a cooling off period, I sent it to Applied Economics in 2005. Here is the editor’s response:

We have had difficulty in obtaining timely reports on your paper. Accordingly, I have read it myself and I have also sought the advice of an associate editor. We were both impressed with your paper and I have therefore decided to accept it without further revision for publication in Applied Economics. It is a good piece of applied work.

It was around this time that I may have decided it wasn’t such a great idea to engage with a more multi-disciplinary audience.