Most ACM SIG conferences are single-blind:
the authors don't know the reviewers (generally PC members),
but the reviewers know the author(s)' identity. Some
conferences use double-blind reviewing, in which the
identity of the authors is hidden from the reviewers,
ostensibly to improve the objectivity of the reviews.
In a few others, the identity of the authors is hidden
to the reviewers, but is revealed at the PC meeting.
ACM sponsors about 80 conferences annually;
about 20% are double-blind. In discussions with other
SIG chairs, they appear to be very satisfied with those
conferences that are double-blind. However, there are
passionate voices o n both sides of the debate.
There are several advantages to double-blind
reviewing. The primary objective is to raise the quality
of reviewing by removing, when possible, bias based
on the authors of the paper. This seems to occur in
several ways: it is easier for the reviewers to be honest
if the names of the authors are not staring them in
the face, it is easier for the program committee to
make tough decisions if they believe that the reviewers
did not know the authors, and there is some evidence
that this process is more fair to authors with obviously
female names. It also has a major perceptual advantage,
in that it makes the process seem more
objective, which encourages young authors and less-published
authors to submit papers, and reduces the perception
that there is an insiders club. It seems to raise the
quality of the conference; many conferences employing
double-blind reviewing are considered to be the top
conference in their area, e.g., CHI, DAC, ICSA, SIGCOMM
and SOSP, though it is not clear which way the causality
goes. Finally, there is evidence that prospective authors
overwhelming want double-blind reviewing.
Double-blind reviewing comes with some
disadvantages. It is an extra burden on authors, who
must revise their papers with care to hide their identity,
and on the program chair. It is often not hard for reviewers
to guess the authors, circumventing the process. Well-known
authors can use tricks to reveal themselves (such as
whitening out a bibliographic entry but leaving it in
alphabetical order). Calls for papers need to be carefully
written to preclude (as much as possible) such tricks.
Note however that even when the authors can be guessed,
the guess is not certain, and there are sometimes surprises.
Some argue that reviewing is already fair, and that
double-blind reviewing won't increase the quality of
reviewing. Some go the other direction and argue that
author identity provides valuable clues as to the quality
of the research, and that double-blind reviewing teaches
young people that only ideas matter, rather than that
persuasion and author credibility are important components
in the process of science.
After discussions with many within SIGMOD,
the majority of whom have requested double-blind reviewing,
I've decided to have the SIGMOD conference try this
for two years, after which the experiment will be evaluated,
and a decision made to continue or to revert to the
current method of single-blind reviewing. We'll start
with the SIGMOD'01 conference. It will be interesting
to see if the advantages in fact outweigh the disadvantages.
Speaking of conferences, I hope that you
have registered for the SIGMOD'00 conference, to be held
in Dallas on May 14-19, with the co-convened PODS'00
conference and several workshops before and after. When
you come, don't forget to bring a spare database textbook
to donate, helping disseminate the ideas our community
has originated and nurtured. See you in Texas!
| Rick Snodgrass |
| January, 2000 |
|
