Abstract Many tasks in Natural Language Processing (NLP) and Computer Vision (CV) offer
evidence that humans disagree, from objective tasks such as part-of-speech tagging to more …

We introduce scientific claim verification, a new task to select abstracts from the research
literature containing evidence that SUPPORTS or REFUTES a given scientific claim, and to …

Crowdsourcing has been the prevalent paradigm for creating natural language
understanding datasets in recent years. A common crowdsourcing practice is to recruit a …

Statistical significance testing is a standard statistical tool designed to ensure that
experimental results are not coincidental. In this opinion/theoretical paper we discuss the …

We show that constituency parsing benefits from unsupervised pre-training across a variety
of languages and a range of pre-training conditions. We first compare the benefits of no pre …

Supervised learning assumes that a ground truth label exists. However, the reliability of this
ground truth depends on human annotators, who often disagree. Prior work has shown that …

A common form of sarcasm on Twitter consists of a positive sentiment contrasted with a
negative situation. For example, many sarcastic tweets include a positive sentiment, such as …

Lacking standardized extrinsic evaluation methods for vector representations of words, the
NLP community has relied heavily on word similarity tasks as a proxy for intrinsic evaluation …

Events and entities are closely related; entities are often actors or participants in events and
events without entities are uncommon. The interpretation of events and entities is highly …

Writing tests is a time-consuming yet essential task during software development. We
propose to leverage recent advances in deep learning for text and code generation to assist …