Hannah Bast's slides on the European Symposium on Algorithms 2018 Track B experiment (two independent program committees decided on the same set of papers and then the conference accepted the union of their acceptances): http://ad-publications.informatik.uni-freiburg.de/ESA_experiment_Bast_2018.pdf
Some conclusions: the initial scoring is remarkably consistent, and per-paper discussions to reconcile differences of scoring are useful, but the final decision on which "gray zone" papers to keep is random and could be replaced by a simple threshold.
@11011110 I really appreciate the fact that people are doing this kind of experiments! ☺
Reviewing the reviewing process is great for Science! ☺
A Mastodon instance for maths people. The kind of people who make \(\pi z^2 \times a\) jokes.
\) for inline LaTeX, and
\] for display mode.