PC Blog Post - Start of Review Process
We have finally kicked off the reviewing process (January 17 to February 07)!!
As you know, we are experimenting with several conference process changes this year: (1) a COI detection system developed by Amanda Stent and Arya McCarthy, (2) a reviewer-paper matching system developed by Graham Neubig, and (3) removed the “reviewer bidding” phase, on top of significant manual effort from our Senior Area Chairs and Area Chairs in the review assignment phase. These changes were meant to improve reviewer/paper matching while improving detection of conflicts of interest as well as minimizing the weaknesses of the bidding method.
We need to first thank you all for filling out the global profile and the author/reviewer form! Your effort has made it possible to use the automated systems. It has also led to the largest ever pool of reviewer candidates!
Many of you have received paper assignments (notified through an email), and also many have not received any assignments. Paper assignments were made based on the information you provided on the reviewer form, the number of submissions in each area, the results from automated assignment systems, as well as manual adjustment from PCs/SACs/ACs. If you did not receive any assignment at this point, it is still possible you will be called upon to review at any point before final decisions are made. We would like to thank you in advance for your willingness and efforts in this review process!
We’ve listed a lot of helpful tips and resources for reviewing effectively here:
But one thing we would like to add is that always helps to start the reviews early! :)
In case you are curious about what has been involved in coming to this point, in the past month or so, the PC/SAC/AC have done the following:
- Went through all submissions to identify desk rejects. No paper was desk rejected because of the Global profile or author/review form issues.
- Assigned papers to each area. In many cases, the author specified area (during submission) was not the best home for reviewing the paper.
- Allocated reviewers to each track based on the preferences indicated on the reviewer form as well as the number of submissions to each track.
- Assigned area chairs and reviewers to each paper (this was done first by an automated method and then the SACs moved people manually)
- ACs double checked the submissions as well as the reviewer assignments.
With all of your help, we are looking forward to putting together an exciting program for ACL 2020!