Webui: A dataset for enhancing visual ui understanding with web semantics

J Wu, S Wang, S Shen, YH Peng, J Nichols… - Proceedings of the …, 2023 - dl.acm.org
Modeling user interfaces (UIs) from visual information allows systems to make inferences
about the functionality and semantics needed to support use cases in accessibility, app …

Toward interactive bug reporting for (android app) end-users

Y Song, J Mahmud, Y Zhou, O Chaparro… - Proceedings of the 30th …, 2022 - dl.acm.org
Many software bugs are reported manually, particularly bugs that manifest themselves
visually in the user interface. End-users typically report these bugs via app reviewing …

Combining Language and App UI Analysis for the Automated Assessment of Bug Reproduction Steps

J Mahmud, A Saha, O Chaparro, K Moran… - arxiv preprint arxiv …, 2025 - arxiv.org
Bug reports are essential for developers to confirm software problems, investigate their
causes, and validate fixes. Unfortunately, reports often miss important information or are …

Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study

R Tufano, A Mastropaolo, F Pepe… - 2024 IEEE/ACM 21st …, 2024 - ieeexplore.ieee.org
Large Language Models (LLMs) have gained significant attention in the software
engineering community. Nowadays developers have the possibility to exploit these models …

Toward the automated localization of buggy mobile app uis from bug descriptions

A Saha, Y Song, J Mahmud, Y Zhou, K Moran… - Proceedings of the 33rd …, 2024 - dl.acm.org
Bug report management is a costly software maintenance process comprised of several
challenging tasks. Given the UI-driven nature of mobile apps, bugs typically manifest …

MotorEase: Automated detection of motor impairment accessibility issues in mobile app UIs

A Krishna Vajjala, SMH Mansur, J Jose… - Proceedings of the IEEE …, 2024 - dl.acm.org
Recent research has begun to examine the potential of automatically finding and fixing
accessibility issues that manifest in software. However, while recent work makes important …

On Using GUI Interaction Data to Improve Text Retrieval-based Bug Localization

J Mahmud, N De Silva, SA Khan, SH Mostafavi… - Proceedings of the 46th …, 2024 - dl.acm.org
One of the most important tasks related to managing bug reports is localizing the fault so that
a fix can be applied. As such, prior work has aimed to automate this task of bug localization …

Vision-Based Mobile App GUI Testing: A Survey

S Yu, C Fang, Z Tuo, Q Zhang, C Chen, Z Chen… - arxiv preprint arxiv …, 2023 - arxiv.org
Graphical User Interface (GUI) has become one of the most significant parts of mobile
applications (apps). It is a direct bridge between mobile apps and end users, which directly …

AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding

SA Khan, W Wang, Y Ren, B Zhu, J Shi… - arxiv preprint arxiv …, 2024 - arxiv.org
Nearly a decade of research in software engineering has focused on automating mobile app
testing to help engineers in overcoming the unique challenges associated with the software …

SmartRecorder: An IMU-based Video Tutorial Creation by Demonstration System for Smartphone Interaction Tasks

X Hu, Y Huang, B Liu, R Wu, Y Hu, AJ Quigley… - Proceedings of the 28th …, 2023 - dl.acm.org
This work focuses on an active topic in the HCI community, namely tutorial creation by
demonstration. We present a novel tool named SmartRecorder that facilitates people …