Seven failure points when engineering a retrieval augmented generation system

S Barnett, S Kurniawan, S Thudumu… - Proceedings of the …, 2024 - dl.acm.org
Software engineers are increasingly adding semantic search capabilities to applications
using a strategy known as Retrieval Augmented Generation (RAG). A RAG system involves …

Software architecture for ML-based systems: What exists and what lies ahead

H Muccini, K Vaidhyanathan - … IEEE/ACM 1st Workshop on AI …, 2021 - ieeexplore.ieee.org
The increasing usage of machine learning (ML) coupled with the software architectural
challenges of the modern era has resulted in two broad research areas: i) software …

(why) is my prompt getting worse? Rethinking regression testing for evolving llm apis

W Ma, C Yang, C Kästner - Proceedings of the IEEE/ACM 3rd …, 2024 - dl.acm.org
Large Language Models (LLMs) are increasingly integrated into software applications.
Downstream application developers often access LLMs through APIs provided as a service …

Integrating AIaaS into Existing Systems: The Gokind Experience

BB Musabimana, A Bucaioni - International Conference on Information …, 2024 - Springer
In this research paper, we present the results of our collaborative study with Gokind AB on
the integration of artificial intelligence as a service into an existing system. Initially, we …

Agile4MLS—Leveraging agile practices for develo** machine learning-enabled systems: An industrial experience

K Vaidhyanathan, A Chandran, H Muccini… - IEEE Software, 2022 - ieeexplore.ieee.org
Agile4MLS - Leveraging Agile Practices for Develo** ML-enabled systems: An Industrial
Experience Page 1 Agile4MLS - Leveraging Agile Practices for Develo** ML-enabled systems …

Mlguard: Defend your machine learning model!

S Wong, S Barnett, J Rivera-Villicana… - Proceedings of the 1st …, 2023 - dl.acm.org
Machine Learning (ML) is used in critical highly regulated and high-stakes fields such as
finance, medicine, and transportation. The correctness of these ML applications is important …

Robustness Attributes to Safeguard Machine Learning Models in Production

H Abdelkader, M Abdelrazek… - 2023 IEEE …, 2023 - ieeexplore.ieee.org
Machine learning (ML) has revolutionized various industries by enabling the development of
complex models that learn from data and make accurate predictions. However, moving from …

Threshy: Supporting safe usage of intelligent web services

A Cummaudo, S Barnett, R Vasa, J Grundy - Proceedings of the 28th …, 2020 - dl.acm.org
Increased popularity of 'intelligent'web services provides end-users with machine-learnt
functionality at little effort to developers. However, these services require a decision …

Towards robust production machine learning systems: Managing dataset shift

H Abdelkader - Proceedings of the 35th IEEE/ACM International …, 2020 - dl.acm.org
The advances in machine learning (ML) have stimulated the integration of their capabilities
into software systems. However, there is a tangible gap between software engineering and …

The Product Beyond the Model--An Empirical Study of Repositories of Open-Source ML Products

N Nahar, H Zhang, G Lewis, S Zhou… - arxiv preprint arxiv …, 2023 - arxiv.org
Machine learning (ML) components are increasingly incorporated into software products for
end-users, but developers face challenges in transitioning from ML prototypes to products …