ASPLOS’21 Extended Abstracts
In the past decade, our community has grown considerably both in the number of researchers and the number of institutions involved. While this is a wonderful development for our field, the corresponding growth in conference submissions is stressing our paper review processes. The increasingly larger program committees (PC) and multi-day PC meetings have not been sufficient to address the problems. PC members complain about excessive reviewing workloads and long PC meetings where they rarely talk. Authors routinely complain about low quality of reviews and inconsistent review outcomes.
We believe the key to high quality review outcomes and an overall high quality program is to involve more PC members with the review and decision for each paper. To allow for that in the presence of record submission counts (almost 500 for ASPLOS’20), we will pilot the use of extended abstracts in ASPLOS’21. Each paper submission must be accompanied by a 2-page extended abstract that summarizes the motivation, key insights, main artifacts, and important contributions (see this template for suggested structure). The extended abstracts are inspired by the familiar submission format for the yearly IEEE Micro Top Picks issue.
Here is how extended abstracts will help in the reviewing process:
- We will use the extended abstracts in the first round of reviewing (R1). We expect each abstract to be reviewed by 4 or more PC members (plus additional external reviewers) and each PC member will review 4x to 5x more abstracts than papers in a traditional R1. We hope that more PC reviews per submission and better calibrated PC members will lead to better R1 decisions. Contrast this to recent ASPLOS conferences, where some papers were rejected in R1 with 1 or 2 PC reviews.
- We will use the extended abstracts to increase the number of informed participants in the PC meeting. Ahead of the PC meeting, we will ask PC members to read a few additional extended abstracts and 1-page review summaries for submissions in their area. This will allow us to quickly bring further expertise into the discussion for challenging submissions. We will also allow any PC member to access extended abstract and review summaries. That way, we hope to avoid what is common in PC meetings, where a disagreement by the 3 PC reviewers is resolved with a vote by uninformed and uncalibrated PC members.
The PC members may suggest other uses for the extended abstracts throughout the review process.
We are excited to measure and understand how extended abstracts can improve our reviewing process. Along with other experiments, such as multiple submission deadlines throughout the year, they can help us achieve high quality paper review and selection as our community grows.