All submissions are preliminary screened by the Program Chair for adherence to the paper preparation guidelines and double-blind submission guidelines.
Afterwards, Program Committee (PC) members bid for papers, indicating their ability and willingness to review individual submissions. Then, the Program Chair assigns papers to reviewers based on the submitted bids. The goal of this stage is to ensure that each manuscript is assigned to reviewers with sufficient expertise to properly evaluate it, while balancing the reviewing load among PC members. Each paper receives at least 4 reviews, and it is scored based on the evaluation criteria and overall merit.
Paper evaluation is primarily based on the submitted manuscript and according to the published evaluation criteria. Technical content is limited to a maximum of 20 pages to ensure consistent rules for authors, fit into the confines suitable for a conference contribution, and limit the required reviewing effort. Authors may also submit supplemental materials, for example in the form of an extended appendix or technical report. Reviewers can consult such materials, especially where it may help clarify technical correctness. However, the main manuscript must be self-contained, e.g., contributions and motivations of the work should not be delegated to supplemental materials, and reviewers should not be expected to read tens of additional pages of supplemental materials to judge the work.
In some rare cases, the program committee might lack sufficient expert members to evaluate a submission. In this case the Program Chair, in consultation with the committee, can invite an external reviewer to provide feedback on the paper. The external reviewer provides a written review according to the published evaluation criteria.
After reviews are completed, PC members have the opportunity to look at other reviews for their assigned papers and discuss them on the reviewers’ forum. The Program Chair selects a meta-reviewer for each submission, who is responsible for leading the discussion.
Acceptance decisions are taken during the ECRTS PC meeting, which is held virtually shortly before the notification deadline. Decisions are taken by the PC committee as a whole, and not just by the reviewers of each individual paper. Nonetheless, the committee may decide to reject without discussion papers whose merit scores fall below a certain threshold. For papers discussed during the PC meeting, the meta-reviewer provides a summary of the discussion to the authors. In some cases, the committee may allow the authors to exceed the page limit, within reason, by suggesting to incorporate additional materials in the paper, e.g., from submitted supplemental materials that the committee deems important to clarify the contribution of the paper (provided that such materials has been properly reviewed together with the main manuscript). This is communicated in the meta-review. Notifications to the authors include all reviews received by their paper (including, if any, by external reviewers) with the accompanying scores.
The PC committee may decide to conditionally accept (shepherd) a paper. In this case, the notification email contains a set of conditions that must be met for the paper to be accepted, and a timeline for the shepherding process. The authors must then prepare a manuscript revision. The revision must satisfy the stated conditions, otherwise the paper will not be accepted. In some cases, even with concise writing, addressing the conditions might require additional manuscript pages; in this case, the committee may allow the authors to exceed the page limit within reason.
Conflicts of Interest (COIs)
During bidding, PC members declare COIs based on a list comprising all authors of submitted papers. While authors are also asked to declare COIs, this is mainly done to prevent errors (the Program Chair will check for discrepancies between the authors and PC members submitted COIs), and primary responsibility for avoiding conflicts rests with the reviewers. In case feedback is seeked from an external reviewer, the Program Chair will use their best judgment to avoid a COI while preserving anonymity according to the double-blind process.
PC members are not allowed to review nor see any of the reviews or the reviewers’ identities for papers they are in conflict with. During the PC meeting, they do not attend the discussion for COI papers nor are parties to the acceptance decision.
The review process for papers where the Program Chair is in conflict is handled by an alternate chair, nominated among the senior members of the committee. The Program Chair is allowed to see COIs declarations and submitted bids, but the alternate chair performs reviewers assignments for such papers, so that the Program Chair is not aware of the identities of the reviewers. The Program Chair leaves the PC meeting during the discussion of the COI papers, which is supervised by the alternate chair.
The Program Chair is not allowed to submit papers to the conference. All other organizers, including the General Chair and PC members, are allowed to do so.
Outstanding Paper and Best Paper Awards
During the PC meeting, committee members can nominate papers of outstanding quality to be considered for an award. Following the meeting, the Program Chair (or alternate chair in case the Program Chair is in COI with a nominated paper) selects a small sub-committee of expert reviewers, ensuring that members are not in COI with the nominated papers and have sufficient expertise to compare their relative merit. The sub-committee is responsible for assigning Outstanding Paper Awards (typically, 3 such awards are presented every year); and to select the Best Paper among those. The selection is performed based on the camera-ready version of the nominated papers; for this reason, the selection is single-blind, i.e., members of the sub-committee will see the authors’ identities.
Best Presentation Award
Conference attendees will vote on each presentation to determine the Best Presentation Award.