Recently, instruction-following audio-language models have received broad attention for
audio interaction with humans. However, the absence of pre-trained audio models capable …

Recently, instruction-following audio-language models have received broad attention for
human-audio interaction. However, the absence of benchmarks capable of evaluating audio …

Recent studies have highlighted the importance of fully open foundation models. The Open
Whisper-style Speech Model (OWSM) is an initial step towards reproducing OpenAI Whisper …

Large language models (LLMs) have shown incredible proficiency in performing tasks that
require semantic understanding of natural language instructions. Recently, many works …

Recent research shows a big convergence in model architecture, training objectives, and
inference methods across various tasks for different modalities. In this paper, we propose …

We present a cost-effective method to integrate speech into a large language model (LLM),
resulting in a Contextual Speech Model with Instruction-following/in-context-learning …

Speech dysfluency modeling is the core module for spoken language learning, and speech
therapy. However, there are three challenges. First, current state-of-the-art solutions~~\cite …

Incorporating speech understanding capabilities into pretrained large-language models has
become a vital research direction (SpeechLLM). The previous architectures can be …

We recently developed a joint speech and language model (SLM [1]) which fuses a
pretrained foundational speech model and a large language model (LLM), while preserving …

Recent speech language models (SLMs) typically incorporate pre-trained speech models to
extend the capabilities from large language models (LLMs). In this paper, we propose a …