A survey of flaky tests

O Parry, GM Kapfhammer, M Hilton… - ACM Transactions on …, 2021 - dl.acm.org
Tests that fail inconsistently, without changes to the code under test, are described as flaky.
Flaky tests do not give a clear indication of the presence of software bugs and thus limit the …

An empirical analysis of flaky tests

Q Luo, F Hariri, L Eloussi, D Marinov - Proceedings of the 22nd ACM …, 2014 - dl.acm.org
Regression testing is a crucial part of software development. It checks that software changes
do not break existing functionality. An important assumption of regression testing is that test …

DeFlaker automatically detecting flaky tests

J Bell, O Legunsen, M Hilton, L Eloussi… - Proceedings of the 40th …, 2018 - dl.acm.org
Developers often run tests to check that their latest changes to a code repository did not
break any previously working functionality. Ideally, any new test failures would indicate …

iDFlakies: A framework for detecting and partially classifying flaky tests

W Lam, R Oei, A Shi, D Marinov… - 2019 12th ieee …, 2019 - ieeexplore.ieee.org
Regression testing is increasingly important with the wide use of continuous integration. A
desirable requirement for regression testing is that a test failure reliably indicates a problem …

Understanding flaky tests: The developer's perspective

M Eck, F Palomba, M Castelluccio… - Proceedings of the 2019 …, 2019 - dl.acm.org
Flaky tests are software tests that exhibit a seemingly random outcome (pass or fail) despite
exercising unchanged code. In this work, we examine the perceptions of software …

Root causing flaky tests in a large-scale industrial setting

W Lam, P Godefroid, S Nath, A Santhiar… - Proceedings of the 28th …, 2019 - dl.acm.org
In today's agile world, developers often rely on continuous integration pipelines to help build
and validate their changes by executing tests in an efficient manner. One of the significant …

A study on the lifecycle of flaky tests

W Lam, K Muşlu, H Sajnani… - Proceedings of the ACM …, 2020 - dl.acm.org
During regression testing, developers rely on the pass or fail outcomes of tests to check
whether changes broke existing functionality. Thus, flaky tests, which nondeterministically …

iFixFlakies: A framework for automatically fixing order-dependent flaky tests

A Shi, W Lam, R Oei, T **e, D Marinov - … of the 2019 27th ACM Joint …, 2019 - dl.acm.org
Regression testing provides important pass or fail signals that developers use to make
decisions after code changes. However, flaky tests, which pass or fail even when the code …

Empirically revisiting the test independence assumption

S Zhang, D Jalali, J Wuttke, K Muşlu, W Lam… - Proceedings of the …, 2014 - dl.acm.org
In a test suite, all the test cases should be independent: no test should affect any other test's
result, and running the tests in any order should produce the same test results. Techniques …

Notice of retraction: Does refactoring of test smells induce fixing flaky tests?

F Palomba, A Zaidman - 2017 IEEE international conference on …, 2017 - ieeexplore.ieee.org
Notice of Retraction: Does Refactoring of Test Smells Induce Fixing Flaky Tests? Page 1
Document Retracted from IEEE Xplore® It was recommended and agreed upon by the authors …