AI is widely thought to be poised to transform business, yet current perceptions of the scope
of this transformation may be myopic. Recent progress in natural language processing …

The emergence of large language models (LLMs) has marked a significant breakthrough in
natural language processing (NLP), fueling a paradigm shift in information acquisition …

We find that existing language modeling datasets contain many near-duplicate examples
and long repetitive substrings. As a result, over 1% of the unprompted output of language …

Metrics for set similarity are a core aspect of several data mining tasks. To remove duplicate
results in a Web search, for example, a common approach looks at the Jaccard index …

Identifying near duplicates within large, noisy text corpora has a myriad of applications that
range from de-duplicating training datasets, reducing privacy risk, and evaluating test set …

In entity resolution, blocking pre-partitions data for further processing by more expensive
methods. Two entity mentions are in the same block if they share identical or related …

The amount of data stored in data repositories increases every year. This makes it
challenging to link records between different datasets across companies and even …

Bibliographic databases are used to measure the performance of researchers, universities
and research institutions. Thus, high data quality is required and data duplication is avoided …

State-of-the-art neural language models are capable of generating incredibly fluent English
text. This success provides opportunities for novel forms of interaction, where human writers …

Financial institutions rely on data for many operations, including a need to drive efficiency,
enhance services and prevent financial crime. Data sharing across an organisation or …