Textually pretrained speech language models

M Hassid, T Remez, TA Nguyen, I Gat… - Advances in …, 2023 - proceedings.neurips.cc
Speech language models (SpeechLMs) process and generate acoustic data only, without
textual supervision. In this work, we propose TWIST, a method for training SpeechLMs using …

Revisiting token drop** strategy in efficient bert pretraining

Q Zhong, L Ding, J Liu, X Liu, M Zhang, B Du… - ar** is a recently-proposed strategy to speed up the pretraining of masked
language models, such as BERT, by skip** the computation of a subset of the input tokens …

[PDF][PDF] How Good Is It? Evaluating the Efficacy of Common versus Domain-Specific Prompts on Foundational Large Language Models

OE Amujo, SJ Yang - arxiv preprint arxiv:2407.11006, 2024 - researchgate.net
Recently, large language models (LLMs) have expanded into various domains. However,
there remains a need to evaluate how these models perform when prompted with …

Analyzing Natural Language Processing Techniques to Extract Meaningful Information on Skills Acquisition from Textual Content

LJ Gonzalez-Gomez, SM Hernandez-Munoz… - IEEE …, 2024 - ieeexplore.ieee.org
Natural Language Processing (NLP) combines linguistics, computer science, and AI to
enable computers to understand and interpret human language, making it crucial for …

Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings

L Gao - arxiv preprint arxiv:2408.15650, 2024 - arxiv.org
Text classification is crucial for applications such as sentiment analysis and toxic text
filtering, but it still faces challenges due to the complexity and ambiguity of natural language …

Evaluating the Efficacy of Foundational Models: Advancing Benchmarking Practices to Enhance Fine-Tuning Decision-Making

OE Amujo, SJ Yang - arxiv preprint arxiv:2407.11006, 2024 - arxiv.org
Recently, large language models (LLMs) have expanded into various domains. However,
there remains a need to evaluate how these models perform when prompted with …

Mining the Explainability and Generalization: Fact Verification Based on Self-Instruction

G Lu, Y Liu - arxiv preprint arxiv:2405.12579, 2024 - arxiv.org
Fact-checking based on commercial LLMs has become mainstream. Although these
methods offer high explainability, it falls short in accuracy compared to traditional fine-tuning …

[PDF][PDF] Simulating Intrinsic and Extrinsic User Behaviour in Task-Oriented Dialogues

HC Lin - 2024 - docserv.uni-duesseldorf.de
Task-oriented dialogue systems guide users to accomplish their goals with respect to
specific tasks, such as searching for restaurants or booking flight tickets. These systems …

Towards Conversational Recommendation in Education: Synthesizing Conversations from Structured Course Data and Benchmarking for Conversational Curriculum …

E Steira - 2024 - ntnuopen.ntnu.no
Conversational Recommender Systems (CRS) introduce a conversational element to
traditional Recommender Systems (RS), allowing for dynamic and interactive …

Tibetan Web Text Clustering Based on Contrast Learning

R Zhang, Y Sun - … Conference on Computer, Big Data and …, 2023 - ieeexplore.ieee.org
In the field of natural language processing, text clustering, as a classical data analysis
method, can be used to classify and analyze text well. This paper describes a novel …