Researchers have recently achieved significant advances in deep learning techniques,
which in turn has substantially advanced other research disciplines, such as natural …

In the last decade, deep supervised learning has had tremendous success. However, its
flaws, such as its dependency on manual and costly annotations on large datasets and …

Code completion, which aims to predict the following code token (s) according to the code
context, can improve the productivity of software development. Recent work has proved that …

Machine learning and its promising branch deep learning have shown success in a wide
range of application domains. Recently, much effort has been expended on applying deep …

Code representation learning, which aims to encode the semantics of source code into
distributed vectors, plays an important role in recent deep-learning-based models for code …

With the great success of pre-trained models, the pretrain-then-finetune paradigm has been
widely adopted on downstream tasks for source code understanding. However, compared to …

Large-scale pre-trained models such as CodeBERT, GraphCodeBERT have earned
widespread attention from both academia and industry. Attributed to the superior ability in …

Learning-based program repair has achieved good results in a recent series of papers. Yet,
we observe that the related work fails to repair some bugs because of a lack of knowledge …

In this work we systematically review the recent advancements in code processing with
language models, covering 50+ models, 30+ evaluation tasks, 170+ datasets, and 700 …

Most existing pre-trained language models for source code focus on learning the static code
text, typically augmented with static code structures (abstract syntax tree, dependency …