Write a Blog >>
ESEC/FSE 2022
Mon 14 - Fri 18 November 2022 Singapore
Tue 15 Nov 2022 10:45 - 11:00 at SRC Auditorium 2 - Machine Learning II Chair(s): Atif Memon

Deep learning (DL) has been widely applied to many domains. Unique challenges in engineering DL systems are posed by the programming paradigm shift from traditional systems to DL systems, and performance is one of the challenges. Performance problems (PPs) in DL systems can cause severe consequences such as excessive resource consumption and financial loss. While bugs in DL systems have been extensively investigated, PPs in DL systems have hardly been explored. To bridge this gap, we present the first comprehensive study to i) characterize symptoms, root causes, and introducing and exposing stages of PPs in DL systems developed in \textsc{TensorFLow} and \textsc{Keras}, with 224 PPs collected from 210 StackOverflow posts, and to ii) assess the capability of existing performance analysis approaches in tackling PPs, with a constructed benchmark of 58 PPs in DL systems. Our findings shed light on the implications on developing high-performance DL systems, and detecting and localizing PPs in DL systems. To demonstrate the usefulness of our findings, we develop a static checker DeepPerf to detect three types of PPs. It has detected 488 new PPs in 130 GitHub projects. 105 and 27 PPs have been confirmed and fixed.

Tue 15 Nov

Displayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change

10:45 - 12:15
10:45
15m
Talk
Understanding Performance Problems in Deep Learning Systems
Research Papers
Junming Cao Fudan University, Bihuan Chen Fudan University, Chao Sun Fudan University, Longjie Hu Fudan University, Shuaihong Wu Fudan University, Xin Peng Fudan University
DOI
11:00
15m
Talk
API Recommendation for Machine Learning Libraries: How Far Are We?
Research Papers
Moshi Wei York University, Yuchao Huang Institute of Software at Chinese Academy of Sciences, Junjie Wang Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Jiho Shin York University, Nima Shiri Harzevili York University, Song Wang York University
DOI Pre-print
11:15
15m
Talk
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence
Research Papers
Chaozheng Wang Harbin Institute of Technology, Yuanhang Yang Harbin Institute of Technology, Cuiyun Gao Harbin Institute of Technology, Yun Peng Chinese University of Hong Kong, Hongyu Zhang University of Newcastle, Michael Lyu Chinese University of Hong Kong
DOI
11:30
15m
Talk
Improving ML-Based Information Retrieval Software with User-Driven Functional Testing and Defect Class Analysis
Industry Paper
Junjie Zhu Apple, Teng Long Apple, Wei Wang Apple, Atif Memon Apple
DOI
11:45
15m
Talk
Discrepancies among Pre-trained Deep Neural Networks: A New Threat to Model Zoo Reliability
Ideas, Visions and Reflections
Diego Montes Purdue University, Pongpatapee Peerapatanapokin Purdue University, Jeff Schultz Purdue University, Chengjun Guo Purdue University, Wenxin Jiang Purdue University, James C. Davis Purdue University
DOI