Testing of Autonomous Driving Systems: Where Are We and Where Should We Go?
Autonomous driving has shown great potential to reform modern transportation. Yet its reliability and safety have drawn a lot of attention and concerns. Compared with traditional software systems, autonomous driving systems (ADSs) often use deep neural networks in tandem with logic-based modules. This new paradigm poses unique challenges for software testing. Despite the recent development of new ADS testing techniques, it is not clear to what extent those techniques have addressed the needs of ADS practitioners. To fill this gap, we present the first comprehensive study to identify the current practices and needs of ADS testing. We conducted semi-structured interviews with developers from 10 autonomous driving companies and surveyed 100 developers who have worked on autonomous driving systems. A systematic analysis of the interview and survey data revealed 7 common practices and 4 emerging needs of autonomous driving testing. Through a comprehensive literature review, we developed a taxonomy of existing ADS testing techniques and analyzed the gap between ADS research and practitioners’ needs. Finally, we proposed several future directions for SE researchers, such as developing test reduction techniques to accelerate simulation-based ADS testing.
Mon 14 NovDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
11:00 - 12:30 | |||
11:00 15mTalk | Testing of Autonomous Driving Systems: Where Are We and Where Should We Go? Research Papers Guannan Lou Macquarie University, Yao Deng Macquarie University, Xi Zheng Macquarie University, Mengshi Zhang Meta, Tianyi Zhang Purdue University DOI | ||
11:15 15mTalk | Fuzzing Deep-Learning Libraries via Automated Relational API Inference Research Papers Yinlin Deng University of Illinois at Urbana-Champaign, Chenyuan Yang University of Illinois at Urbana-Champaign, Anjiang Wei Stanford University, Lingming Zhang University of Illinois at Urbana-Champaign DOI | ||
11:30 15mTalk | Perfect Is the Enemy of Test Oracle Research Papers Ali Reza Ibrahimzada University of Illinois Urbana-Champaign, Yigit Varli Middle East Technical University, Dilara Tekinoglu University of Massachusetts at Amherst, Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign DOI Pre-print Media Attached | ||
11:45 15mTalk | Scenario-Based Test Reduction and Prioritization for Multi-Module Autonomous Driving Systems Research Papers Yao Deng Macquarie University, Xi Zheng Macquarie University, Mengshi Zhang Meta, Guannan Lou Macquarie University, Tianyi Zhang Purdue University DOI |