Write a Blog >>
ESEC/FSE 2022
Mon 14 - Fri 18 November 2022 Singapore
Mon 14 Nov 2022 11:45 - 12:00 at SRC LT 51 - Empirical I Chair(s): Lingxiao Jiang

In modern code reviews, many artifacts play roles in knowledge- sharing and documentation: summaries, test plans, and comments, etc. Improving developer tools and facilitating better code reviews require an understanding of the quality of pull requests and their artifacts. This is difficult to measure, however, because they are often free-form natural language and unstructured text data. In this paper, we focus on measuring the quality of test plans at Meta. Test plans are used as a communication mechanism between the author of a pull request and its reviewers, serving as walkthroughs to help confirm that the changed code is behaving as expected. We collected developer opinions on over 650 test plans from more than 500 Meta developers, then introduced a transformer-based model to leverage the success of natural language processing (NLP) tech- niques in the code review domain. In our study, we show that the learned model is able to capture the sentiment of developers and reflect a correlation of test plan quality with review engagement and reversions: compared to a decision tree model, our proposed transformer-based model achieves a 7% higher F1-score. Finally, we present a case study of how such a metric may be useful in experiments to inform improvements in developer tools and experiences.

Mon 14 Nov

Displayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change

11:00 - 12:30
Empirical IResearch Papers / Industry Paper at SRC LT 51
Chair(s): Lingxiao Jiang Singapore Management University
11:00
15m
Talk
What Improves Developer Productivity at Google? Code Quality
Industry Paper
Lan Cheng Google, Emerson Murphy-Hill Google, Mark Canning Google, Ciera Jaspan Google, Collin Green Google, Andrea Knight Google, Nan Zhang Google, Liz Kammer Google
DOI
11:15
15m
Talk
Understanding Why We Cannot Model How Long a Code Review Will Take: An Industrial Case Study
Industry Paper
Lawrence Chen Meta, Peter Rigby Concordia University; Meta, Nachiappan Nagappan Facebook
DOI
11:30
15m
Talk
Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization
Research Papers
Lin Shi ISCAS, Fangwen Mu Institute of Software Chinese Academy of Sciences, Xiao Chen Institute of Software at Chinese Academy of Sciences, Song Wang York University, Junjie Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Ye Yang Stevens Institute of Technology, Ge Li Peking University, Xin Xia Huawei, Qing Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences
DOI Pre-print
11:45
15m
Talk
Leveraging Test Plan Quality to Improve Code Review Efficacy
Industry Paper
Lawrence Chen Meta, Rui Abreu Meta Platforms, Tobi Akomolede Meta Platforms, Peter Rigby Concordia University; Meta, Satish Chandra Meta Platforms, Nachiappan Nagappan Facebook
DOI