Understanding design collaboration between designers and artificial intelligence: a systematic literature review

Y Shi, T Gao, X Jiao, N Cao - Proceedings of the ACM on Human …, 2023 - dl.acm.org
Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing.
Designers, as a special user group interacting with AI, have received more attention in the …

UEyes: Understanding visual saliency across user interface types

Y Jiang, LA Leiva, H Rezazadegan Tavakoli… - Proceedings of the …, 2023 - dl.acm.org
While user interfaces (UIs) display elements such as images and text in a grid-based layout,
UI types differ significantly in the number of elements and how they are displayed. For …

EyeFormer: predicting personalized scanpaths with transformer-guided reinforcement learning

Y Jiang, Z Guo, H Rezazadegan Tavakoli… - Proceedings of the 37th …, 2024 - dl.acm.org
From a visual-perception perspective, modern graphical user interfaces (GUIs) comprise a
complex graphics-rich two-dimensional visuospatial arrangement of text, images, and …

The task-attention theory of game learning: a theory and research agenda

J Cutting, S Deterding - Human–Computer Interaction, 2024 - Taylor & Francis
Video games have long been heralded as powerful learning tools. Yet half a century since
the term “serious games” was coined (Abt, 1970), the evidence on whether and when games …

Impact of Design Decisions in Scanpath Modeling

P Emami, Y Jiang, Z Guo, LA Leiva - Proceedings of the ACM on Human …, 2024 - dl.acm.org
Modeling visual saliency in graphical user interfaces (GUIs) allows to understand how
people perceive GUI designs and what elements attract their attention. One aspect that is …

An f-shape click model for information retrieval on multi-block mobile pages

L Fu, J Lin, W Liu, R Tang, W Zhang, R Zhang… - Proceedings of the …, 2023 - dl.acm.org
Most click models focus on user behaviors towards a single list. However, with the
development of user interface (UI) design, the layout of displayed items on a result page …

UEyes: An Eye-Tracking Dataset across User Interface Types

Y Jiang, LA Leiva, PRB Houssel, HR Tavakoli… - arxiv preprint arxiv …, 2024 - arxiv.org
Different types of user interfaces differ significantly in the number of elements and how they
are displayed. To examine how such differences affect the way users look at UIs, we …

Describing ui screenshots in natural language

LA Leiva, A Hota, A Oulasvirta - ACM Transactions on Intelligent …, 2022 - dl.acm.org
Being able to describe any user interface (UI) screenshot in natural language can promote
understanding of the main purpose of the UI, yet currently it cannot be accomplished with …

Scanpath prediction on information visualisations

Y Wang, A Bulling - IEEE Transactions on Visualization and …, 2023 - ieeexplore.ieee.org
We propose Unified Model of Saliency and Scanpaths (UMSS)-a model that learns to predict
multi-duration saliency and scanpaths (ie sequences of eye fixations) on information …

Predicting visual attention in graphic design documents

S Chakraborty, Z Wei, C Kelton, S Ahn… - IEEE Transactions …, 2022 - ieeexplore.ieee.org
We present a model for predicting visual attention during the free viewing of graphic design
documents. While existing works on this topic have aimed at predicting static saliency of …