Modeling user interfaces (UIs) from visual information allows systems to make inferences
about the functionality and semantics needed to support use cases in accessibility, app …

Many accessibility features available on mobile platforms require applications (apps) to
provide complete and accurate metadata describing user interface (UI) components …

It is common practice for developers of user-facing software to transform a mock-up of a
graphical user interface (GUI) into code. This process takes place both at an application's …

When develo** the user interface code of a mobile application, in practice a big gap exists
between the digital conceptual drawings of graphic artists and working user interface code …

Automated understanding of user interfaces (UIs) from their pixels can improve accessibility,
enable task automation, and facilitate interface design without relying on developers to …

INTRODUCTION: The Physics of Notations (PoN) is a theory for the design of cognitively
effective visual notations, emphasizing the need for design grounded in objective and …

When a bug manifests in a user-facing application, it is likely to be exposed through the
graphical user interface (GUI). Given the importance of visual information to the process of …

In this paper, we study the app store as a phenomenon from the developers' perspective to
investigate the extent to which app stores affect software engineering tasks. Through …

Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused
understanding of images and language, but many perform poorly on UI tasks due to the lack …

Programming screencasts have two important applications in software engineering context:
study developer behaviors, information needs and disseminate software engineering …