Fairness in deep learning: A survey on vision and language research

O Parraga, MD More, CM Oliveira, NS Gavenski… - ACM Computing …, 2023 - dl.acm.org
Despite being responsible for state-of-the-art results in several computer vision and natural
language processing tasks, neural networks have faced harsh criticism due to some of their …

A survey on model compression for large language models

X Zhu, J Li, Y Liu, C Ma, W Wang - Transactions of the Association for …, 2024 - direct.mit.edu
Abstract Large Language Models (LLMs) have transformed natural language processing
tasks successfully. Yet, their large size and high computational needs pose challenges for …

Weak-to-strong generalization: Eliciting strong capabilities with weak supervision

C Burns, P Izmailov, JH Kirchner, B Baker… - arxiv preprint arxiv …, 2023 - arxiv.org
Widely used alignment techniques, such as reinforcement learning from human feedback
(RLHF), rely on the ability of humans to supervise model behavior-for example, to evaluate …

Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction

S Alam, L Liu, M Yan, M Zhang - Advances in neural …, 2022 - proceedings.neurips.cc
Most cross-device federated learning (FL) studies focus on the model-homogeneous setting
where the global server model and local client models are identical. However, such …

Flexivit: One model for all patch sizes

L Beyer, P Izmailov, A Kolesnikov… - Proceedings of the …, 2023 - openaccess.thecvf.com
Vision Transformers convert images to sequences by slicing them into patches. The size of
these patches controls a speed/accuracy tradeoff, with smaller patches leading to higher …

Efficient methods for natural language processing: A survey

M Treviso, JU Lee, T Ji, B Aken, Q Cao… - Transactions of the …, 2023 - direct.mit.edu
Recent work in natural language processing (NLP) has yielded appealing results from
scaling model parameters and training data; however, using only scale to improve …

Hoiclip: Efficient knowledge transfer for hoi detection with vision-language models

S Ning, L Qiu, Y Liu, X He - … of the IEEE/CVF conference on …, 2023 - openaccess.thecvf.com
Abstract Human-Object Interaction (HOI) detection aims to localize human-object pairs and
recognize their interactions. Recently, Contrastive Language-Image Pre-training (CLIP) has …

Beyond efficiency: A systematic survey of resource-efficient large language models

G Bai, Z Chai, C Ling, S Wang, J Lu, N Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated
models like OpenAI's ChatGPT, represents a significant advancement in artificial …

Teacher-student architecture for knowledge distillation: A survey

C Hu, X Li, D Liu, H Wu, X Chen, J Wang… - arxiv preprint arxiv …, 2023 - arxiv.org
Although Deep neural networks (DNNs) have shown a strong capacity to solve large-scale
problems in many areas, such DNNs are hard to be deployed in real-world systems due to …

Generalizable heterogeneous federated cross-correlation and instance similarity learning

W Huang, M Ye, Z Shi, B Du - IEEE Transactions on Pattern …, 2023 - ieeexplore.ieee.org
Federated learning is an important privacy-preserving multi-party learning paradigm,
involving collaborative learning with others and local updating on private data. Model …