Research at the intersection of machine learning, programming languages, and software
engineering has recently taken important steps in proposing learnable probabilistic models …

Graphs are ubiquitous in encoding relational information of real-world objects in many
domains. Graph generation, whose purpose is to generate new graphs from a distribution …

Benchmark datasets have a significant impact on accelerating research in programming
language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster …

In this paper, we introduce a new task for code completion that focuses on handling long
code input and propose a sparse Transformer model, called LongCoder, to address this …

Large Language Models (LLMs) have greatly advanced code auto-completion systems, with
a potential for substantial productivity enhancements for developers. However, current …

Large language models (LLMs) have demonstrated impressive capabilities across various
natural language processing (NLP) tasks, such as machine translation, question answering …

We present a neural model for representing snippets of code as continuous distributed
vectors (``code embeddings''). The main idea is to represent a code snippet as a single fixed …

Several advances in deep learning have been successfully applied to the software
development process. Of recent interest is the use of neural language models to build tools …

The ability to generate natural language sequences from source code snippets has a variety
of applications such as code summarization, documentation, and retrieval. Sequence-to …

In this work we systematically review the recent advancements in software engineering with
language models, covering 70+ models, 40+ evaluation tasks, 180+ datasets, and 900 …