Write a Blog >>
ESEC/FSE 2022
Mon 14 - Fri 18 November 2022 Singapore
Mon 14 Nov 2022 11:30 - 11:45 at SRC Auditorium 2 - Machine Learning I Chair(s): Shin Yoo

Pre-trained Generative Language models (e.g., PLBART, CodeT5, SPT-Code) for source code yielded strong results on several tasks in the past few years, including code generation and translation. These models have adopted varying pre-training objectives to learn statistics of code construction from very large-scale corpora in a self-supervised fashion; the success of pre-trained models largely hinges on these pre-training objectives. This paper proposes a new pre-training objective, “Naturalizing” of source code, exploiting code’s bimodal, dual-channel (formal & natural channels) nature. Unlike natural language, code’s bimodal, dual-channel nature allows us to generate semantically equivalent code at scale. We introduce six classes of semantic preserving transformations to introduce unnatural forms of code, and then force our model to produce more natural original programs written by developers. Learning to generate equivalent, but more natural code, at scale, over large corpora of open-source code, without explicit manual supervision, helps the model learn to both ingest & generate code. We fine-tune our model in three generative Software Engineering tasks: code generation, code translation, and code refinement with limited human-curated labeled data and achieve state-of-the-art performance rivaling CodeT5. We show that our pre-trained model is especially competitive at zero-shot and few-shot learning, and better at learning code properties (e.g., syntax, data flow)

Mon 14 Nov

Displayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change

11:00 - 12:30
Machine Learning IIndustry Paper / Research Papers at SRC Auditorium 2
Chair(s): Shin Yoo KAIST
11:00
15m
Talk
Adaptive Fairness Improvement Based on Causality Analysis
Research Papers
Mengdi Zhang Singapore Management University, Jun Sun Singapore Management University
DOI
11:15
15m
Talk
Nalanda: A Socio-technical Graph Platform for Building Software Analytics Tools at Enterprise Scale
Industry Paper
Chandra Maddila Microsoft Research, Suhas Shanbhogue Microsoft Research, Apoorva Agrawal Microsoft Research, Thomas Zimmermann Microsoft Research, Chetan Bansal Microsoft, Nicole Forsgren Microsoft Research, Divyanshu Agrawal Microsoft Research, Kim Herzig Microsoft, Arie van Deursen Delft University of Technology
DOI Pre-print
11:30
15m
Talk
NatGen: Generative Pre-training by “Naturalizing” Source Code
Research Papers
Saikat Chakraborty Microsoft Research, Toufique Ahmed University of California at Davis, Yangruibo Ding Columbia University, Prem Devanbu University of California at Davis, Baishakhi Ray Columbia University
DOI Pre-print Media Attached
11:45
15m
Talk
Uncertainty-Aware Transfer Learning to Evolve Digital Twins for Industrial Elevators
Industry Paper
Xu Qinghua Simula Research Laboratory; University of Oslo, Shaukat Ali Simula Research Laboratory, Tao Yue Simula Research Laboratory, Maite Arratibel Orona
DOI
12:00
15m
Talk
All You Need Is Logs: Improving Code Completion by Learning from Anonymous IDE Usage Logs
Industry Paper
Vitaliy Bibaev JetBrains, Alexey Kalina JetBrains, Vadim Lomshakov JetBrains, Yaroslav Golubev JetBrains Research, Alexander Bezzubov JetBrains, Nikita Povarov JetBrains, Timofey Bryksin JetBrains Research
DOI Pre-print