summaryrefslogtreecommitdiffstats
path: root/scripts/lib/scriptpath.py
diff options
context:
space:
mode:
authorAlexis Lothoré <alexis.lothore@bootlin.com>2023-02-28 19:10:52 +0100
committerRichard Purdie <richard.purdie@linuxfoundation.org>2023-03-12 23:39:13 +0000
commit1eae53e277b60347753f298e1cf3376c52a981ca (patch)
tree7ece9187dd78b5041d91331bd7bf1675afe16b12 /scripts/lib/scriptpath.py
parentc31c140746c9ccf3183b8046a24ed040d5c94934 (diff)
downloadpoky-1eae53e277b60347753f298e1cf3376c52a981ca.tar.gz
scripts/resulttool: do not count newly passing tests as regressions
resulttool regression module simply compare a base test status to a target test result status. This approach raises many false positives since all XXX -> PASS transitions (XXX being any status different from PASS) are flagged as regression. - Do not list XXX -> PASS transitions in regression report, instead count them and print a summary of "newly passing tests" - If an inspected pair has only "newly passing tests", do not print detailed list and print it as "Improvement" instead of "Regression" Updated output example looks like the following: [...] Improvement: oeselftest_fedora-37_qemux86-64_20230127010225 oeselftest_ubuntu-22.04_qemux86-64_20230226120516 (+1 test(s) passing) [...] Match: oeselftest_almalinux-8.7_qemuarm64_20230127015830 oeselftest_almalinux-8.7_qemuarm64_20230227015258 [...] Regression: oeselftest_almalinux-9.1_qemumips_20230127000217 oeselftest_opensuseleap-15.4_qemumips_20230226130046 ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL Additionally, 44 previously failing test(s) is/are now passing (From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'scripts/lib/scriptpath.py')
0 files changed, 0 insertions, 0 deletions