Write a Blog >>
ESEC/FSE 2022
Mon 14 - Fri 18 November 2022 Singapore
Tue 15 Nov 2022 11:00 - 11:15 at SRC LT 51 - Empirical II Chair(s): Lwin Khin Shar

The most popular code review tools (e.g., Gerrit and GitHub) present the files to review sorted in alphabetical order. Could this choice or, more generally, the relative position in which a file is presented bias the outcome of code reviews? We investigate this hypothesis by triangulating complementary evidence in a two-step study.
First, we observe developers’ code review activity. We analyze the review comments pertaining to 219,476 Pull Requests (PRs) from 138 popular Java projects on GitHub. We found files shown earlier in a PR to receive more comments than files shown later, also when controlling for possible confounding factors: e.g., the presence of discussion threads or the lines added in a file. Second, we measure the impact of file position on defect finding in code review. Recruit- ing 106 participants, we conduct an online controlled experiment in which we measure participants’ performance in detecting two unrelated defects seeded into two different files. Participants are assigned to one of two treatments in which the position of the defective files is switched. For one type of defect, participants are not affected by its file’s position; for the other, they have 64% lower odds to identify it when its file is last as opposed to first. Overall, our findings provide evidence that the relative position in which files are presented has an impact on code reviews’ outcome; we discuss these results and implications for tool design and code review.

Tue 15 Nov

Displayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change

10:45 - 12:30
Empirical IIIndustry Paper / Research Papers / Demonstrations at SRC LT 51
Chair(s): Lwin Khin Shar Singapore Management University
10:45
15m
Talk
Using Nudges to Accelerate Code Reviews at ScaleDistinguished Paper Award
Research Papers
Qianhua Shan Meta, David Sukhdeo Meta, Qianying Huang Meta, Seth Rogers Meta, Lawrence Chen Meta, Elise Paradis Meta, Peter Rigby Concordia University; Meta, Nachiappan Nagappan Meta
DOI
11:00
15m
Talk
First Come First Served: The Impact of File Position on Code ReviewDistinguished Paper Award
Research Papers
Enrico Fregnan University of Zurich, Larissa Braz University of Zurich, Marco D'Ambros USI Lugano, Gül Calikli University of Glasgow, Alberto Bacchelli University of Zurich
DOI Pre-print Media Attached
11:15
15m
Talk
Code, Quality, and Process Metrics in Graduated and Retired ASFI Projects
Research Papers
Ştefan Stănciulescu University of California at Davis, Likang Yin University of California at Davis, Vladimir Filkov University of California at Davis
DOI Pre-print Media Attached
11:30
15m
Talk
Are Elevator Software Robust against Uncertainties? Results and Experiences from an Industrial Case Study
Industry Paper
Liping Han Nanjing University of Aeronautics and Astronautics; Simula Research Laboratory, Tao Yue Simula Research Laboratory, Shaukat Ali Simula Research Laboratory, Aitor Arrieta Mondragon University, Maite Arratibel Orona
DOI
11:45
15m
Talk
CommentFinder: A Simpler, Faster, More Accurate Code Review Comments Recommendation
Research Papers
Yang Hong Monash University, Chakkrit Tantithamthavorn Monash University, Patanamon Thongtanunam University of Melbourne, Aldeida Aleti Monash University
DOI
12:00
7m
Talk
TAPHSIR: Towards AnaPHoric Ambiguity Detection and ReSolution In Requirements
Demonstrations
Saad Ezzini University of Luxembourg, Sallam Abualhaija University of Luxembourg, Chetan Arora Deakin University, Mehrdad Sabetzadeh University of Ottawa
12:08
7m
Talk
COREQQA - A COmpliance REQuirements Understanding using Question Answering Tool
Demonstrations
Sallam Abualhaija University of Luxembourg, Chetan Arora Deakin University, Lionel Briand University of Luxembourg; University of Ottawa