Measuring design compliance using neural language models – an automotive case study
As the modern vehicle becomes more software-defined, it is beginning to take significant effort to avoid serious regression in software design. This is because automotive software architects rely largely upon manual review of code to spot deviations from specified design principles. Such an approach is both inefficient and prone to error. In recent days, neural language models pre-trained on source code are beginning to be used for automating a variety of programming tasks. In this work, we extend the application of such a Programming Language Model (PLM) to automate the assessment of design compliance. Using a PLM, we construct a system that assesses whether a set of query programs comply with Controller-Handler, a design pattern specified to ensure hardware abstraction in automotive control software. The assessment is based upon measuring whether the geometrical arrangement of query program embeddings, extracted from the PLM, aligns with that of a set of known implementations of the pattern. The level of alignment is then transformed into an interpretable measure of compliance. Using a controlled experiment, we demonstrate that our technique determines compliance with a precision of 92%. Also, using expert review to calibrate the automated assessment, we introduce a protocol to determine the nature of the violation, helping eventual refactoring. Results from this work indicate that neural language models can provide valuable assistance to human architects in assessing and fixing violations in automotive software design.
Fri 18 NovDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
16:00 - 17:30 | |||
16:00 20mResearch paper | Measuring design compliance using neural language models – an automotive case study PROMISE Dhasarathy Parthasarathy Volvo AB, Cecilia Ekelin Volvo AB, Anjali Karri Volvo AB, Jiapeng Sun Volvo AB, Panagiotis Moraitis Volvo AB | ||
16:20 20mResearch paper | API + Code = Better Code Summary? - Insights from an Exploratory Study PROMISE | ||
16:40 20mResearch paper | LOGI: An Empirical Model of Heat-Induced Disk Drive Data Loss and its Implications for Data Recovery PROMISE Hammad Ahmad University of Michigan, Colton Holoday , Ian Bertram University of Michigan, Kevin Angstadt , Zohreh Sharafi Polytechnique Montréal, Westley Weimer University of Michigan | ||
17:00 10mDay closing | Farewell PROMISE 2022 PROMISE Shane McIntosh University of Waterloo, Weiyi Shang Concordia University, Gema Rodríguez-Pérez University of British Columbia (UBC) |