A survey of flaky tests

O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

Know you neighbor: Fast static prediction of test flakiness

R Verdecchia, E Cruciani, B Miranda… - IEEE Access, 2021 - ieeexplore.ieee.org
Context: Flaky tests plague regression testing in Continuous Integration environments by
slowing down change releases and wasting testing time and effort. Despite the growing …

The effects of computational resources on flaky tests

D Silva, M Gruber, S Gokhale, E Arteca… - IEEE Transactions …, 2024 - ieeexplore.ieee.org
Flaky tests are tests that non-deterministically pass and fail in unchanged code. These tests
can be detrimental to developers' productivity. Particularly when tests run in continuous …

SleepReplacer: a novel tool-based approach for replacing thread sleeps in selenium WebDriver test code

D Olianas, M Leotta, F Ricca - Software Quality Journal, 2022 - Springer
Assuring quality of web applications is fundamental, given their relevance in the today's
world. A possible way to reach this goal is through end-to-end (E2E) testing, an approach in …

[HTML][HTML] Test flakiness' causes, detection, impact and responses: A multivocal review

A Tahir, S Rasheed, J Dietrich, N Hashemi… - Journal of Systems and …, 2023 - Elsevier
Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software
testing. They are known to cause significant issues, such as reducing the effectiveness and …

Flakime: laboratory-controlled test flakiness impact assessment

M Cordy, R Rwemalika, A Franci, M Papadakis… - Proceedings of the 44th …, 2022 - dl.acm.org
Much research on software testing makes an implicit assumption that test failures are
deterministic such that they always witness the presence of the same defects. However, this …

What we talk about when we talk about software test flakiness

M Barboni, A Bertolino, G De Angelis - … on the Quality of Information and …, 2021 - Springer
Software test flakiness is drawing increasing interest among both academic researchers and
practitioners. In this work we report our findings from a sco** review of white and grey …

FlakeSync: Automatically Repairing Async Flaky Tests

S Rahman, A Shi - Proceedings of the IEEE/ACM 46th International …, 2024 - dl.acm.org
Regression testing is an important part of the development process but suffers from the
presence of flaky tests. Flaky tests nondeterministically pass or fail when run on the same …

Non-Flaky and Nearly-Optimal Time-based Treatment of Asynchronous Wait Web Tests

Y Pei, J Sohn, S Habchi, M Papadakis - ACM Transactions on Software …, 2024 - dl.acm.org
Asynchronous waits are a common root cause of flaky tests and a major time-influential
factor of web application testing. We build a dataset of 49 reproducible asynchronous wait …

On Technical Debt in Software Testing-Observations from Industry

S Eldh - International Symposium on Leveraging Applications of …, 2022 - Springer
Testing large complex systems in an agile way of working was a tough transition for systems
having large active legacy and honouring backward compatibility. Transition from manual …