What does clip know about a red circle? visual prompt engineering for vlms

A Shtedritski, C Rupprecht… - Proceedings of the IEEE …, 2023 - openaccess.thecvf.com
Abstract Large-scale Vision-Language Models, such as CLIP, learn powerful image-text
representations that have found numerous applications, from zero-shot classification to text …

No representation rules them all in category discovery

S Vaze, A Vedaldi, A Zisserman - Advances in Neural …, 2024 - proceedings.neurips.cc
In this paper we tackle the problem of Generalized Category Discovery (GCD). Specifically,
given a dataset with labelled and unlabelled images, the task is to cluster all images in the …

A systematic survey of prompt engineering on vision-language foundation models

J Gu, Z Han, S Chen, A Beirami, B He, G Zhang… - arxiv preprint arxiv …, 2023 - arxiv.org
Prompt engineering is a technique that involves augmenting a large pre-trained model with
task-specific hints, known as prompts, to adapt the model to new tasks. Prompts can be …

Open-world machine learning: A review and new outlooks

F Zhu, S Ma, Z Cheng, XY Zhang, Z Zhang… - arxiv preprint arxiv …, 2024 - arxiv.org
Machine learning has achieved remarkable success in many applications. However,
existing studies are largely based on the closed-world assumption, which assumes that the …

Learning semi-supervised gaussian mixture models for generalized category discovery

B Zhao, X Wen, K Han - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
In this paper, we address the problem of generalized category discovery (GCD), ie, given a
set of images where part of them are labelled and the rest are not, the task is to automatically …

Active generalized category discovery

S Ma, F Zhu, Z Zhong, XY Zhang… - Proceedings of the …, 2024 - openaccess.thecvf.com
Abstract Generalized Category Discovery (GCD) is a pragmatic and challenging open-world
task which endeavors to cluster unlabeled samples from both novel and old classes …

Textual knowledge matters: Cross-modality co-teaching for generalized visual class discovery

H Zheng, N Pu, W Li, N Sebe, Z Zhong - European Conference on …, 2024 - Springer
In this paper, we study the problem of Generalized Category Discovery (GCD), which aims to
cluster unlabeled data from both known and unknown categories using the knowledge of …

Learn to categorize or categorize to learn? self-coding for generalized category discovery

S Rastegar, H Doughty… - Advances in Neural …, 2024 - proceedings.neurips.cc
In the quest for unveiling novel categories at test time, we confront the inherent limitations of
traditional supervised recognition models that are restricted by a predefined category set …

Labeled data selection for category discovery

B Zhao, N Lang, S Belongie, OM Aodha - European Conference on …, 2024 - Springer
Visual category discovery methods aim to find novel categories in unlabeled visual data. At
training time, a set of labeled and unlabeled images are provided, where the labels …

Prediction consistency regularization for generalized category discovery

Y Duan, J He, R Zhang, R Wang, X Li, F Nie - Information Fusion, 2024 - Elsevier
Abstract Generalized Category Discovery (GCD) is a recently proposed open-world problem
that aims to automatically discover and cluster based on partially labeled data. The …