Visiontasker: Mobile task automation using vision based ui understanding and llm task planning
Mobile task automation is an emerging field that leverages AI to streamline and optimize the
execution of routine tasks on mobile devices, thereby enhancing efficiency and productivity …
execution of routine tasks on mobile devices, thereby enhancing efficiency and productivity …
AutoTask: Executing Arbitrary Voice Commands by Exploring and Learning from Mobile GUI
L Pan, B Wang, C Yu, Y Chen, X Zhang… - arxiv preprint arxiv …, 2023 - arxiv.org
Voice command interfaces (VCIs) have gained increasing importance, enabling hands-free
and eyes-free interaction with digital devices. However, the inherent complexity in …
and eyes-free interaction with digital devices. However, the inherent complexity in …
Navigating Interfaces with AI for Enhanced User Interaction
This study introduces an innovative framework designed to automate tasks by interacting
with UIs through a sequential, human-like problem-solving approach. Our approach initially …
with UIs through a sequential, human-like problem-solving approach. Our approach initially …
PromptRPA: Generating Robotic Process Automation on Smartphones from Textual Prompts
Robotic Process Automation (RPA) offers a valuable solution for efficiently automating tasks
on the graphical user interface (GUI), by emulating human interactions, without modifying …
on the graphical user interface (GUI), by emulating human interactions, without modifying …
Test2VA: Reusing GUI Test Cases for Voice Assistant Features Development in Mobile Applications
G Weaver, X Qin - arxiv preprint arxiv:2407.18155, 2024 - arxiv.org
Voice Assistant (VA) in smartphones has become very popular with millions of users
nowadays. A key trend is the rise of custom VA embedding, which enables users to perform …
nowadays. A key trend is the rise of custom VA embedding, which enables users to perform …
Building Python Application for Webmail Interfaces Navigation Using Voice Recognition Technology
M Alkhattali, M Dow, K Azwee… - International Journal of …, 2023 - papers.ssrn.com
Abstract Voice Recognition Technology (VRT) has played a crucial role in technology
development, finding extensive use in the development of humanitarian assistance …
development, finding extensive use in the development of humanitarian assistance …
iTutor: A Generative Tutorial System for Teaching the Elders to Use Smartphone Applications
We present iTutor, a generative tutorial system for promoting smartphone use proficiency
among elders. iTutor is unique because it can dynamically generate tutorials based on …
among elders. iTutor is unique because it can dynamically generate tutorials based on …
Prompt2Task: Automating UI Tasks on Smartphones from Textual Prompts
UI task automation enables efficient task execution by simulating human interactions with
graphical user interfaces (GUIs), without modifying the existing application code. However …
graphical user interfaces (GUIs), without modifying the existing application code. However …