linux/poky.git/scripts/lib/resulttool, branch scarthgap-5.0.8

resulttool: Improve repo layout for oeselftest results

2024-12-09T14:54:15+00:00

Having all oe-selftest results on top of each other results in a large 640MB json file which is hard to use. Split the results out per machine and test type. This also stops the toolchain raw logs from overwriting each other meaning more than one MACHINE is preserved. (From OE-Core rev: 92cb4641ff4ec8c1f681bca21cfeaf2ba6923ab7) Signed-off-by: Richard Purdie (cherry picked from commit 4b890f04bc7d147b4a11b824a84f3d2abd75ac54) Signed-off-by: Steve Sakoman

resulttool: Trim the precision of duration information

2024-12-09T14:54:15+00:00

The duration values have pointless amounts of precision. Removing some of the least significant digits reduces result size and makes the results easier to read. (From OE-Core rev: 7460a4bb50c696c39add79c2b299b5e636532b8c) Signed-off-by: Richard Purdie (cherry picked from commit a789a2e6d97bb8efd663226a17db8d1ca6c1e40f) Signed-off-by: Steve Sakoman

resulttool: Clean up repoducible build logs

2024-12-09T14:54:15+00:00

We've improved the data stored for reproduicible builds. Teach resulttool how to apply those cleanups when reprocessing data so we can reduce results file sizes and make the data easier to process. (From OE-Core rev: 82acc3b1e410afd40fe8a67c77bfe69add95066e) Signed-off-by: Richard Purdie (cherry picked from commit b799c57ae6d61c1b1c7035c8a2c4ba6ee08d1a81) Signed-off-by: Steve Sakoman

resulttool: Handle ltp rawlogs as well as ptest

2024-12-09T14:54:15+00:00

Improve the rawlogs handling to include ltp logs as well as the ptest ones to reduce the size of the results git repos. (From OE-Core rev: 6b164f7fe556824c85873ce2ca4257b3a4956cdf) Signed-off-by: Richard Purdie (cherry picked from commit a0a1954d559609c2c1ca16936d0d68eb3c4c6b45) Signed-off-by: Steve Sakoman

resulttool: Add --logfile-archive option to store mode

2024-12-09T14:54:15+00:00

Storing the log files inside the testresults git repo isn't scaling and isn't really appropriate use of a git repository. Allow these to be optionally stored in a separate filesystem location so the git repo can remain managable. (From OE-Core rev: 3472833b50b5ec97ec19f510ba926ab6aa5429b3) Signed-off-by: Richard Purdie (cherry picked from commit 1afc0f3d7e93fa8496be241e9622d3b9a6904bd5) Signed-off-by: Steve Sakoman

resulttool: Fix passthrough of --all files in store mode

2024-12-09T14:54:15+00:00

When using store mode, --all was broken as not all files were being preserved. Fix this by limiting the scope of the git rm command. (From OE-Core rev: 461140bede110b561f0bae46255d6a9435329e97) Signed-off-by: Richard Purdie (cherry picked from commit 9604561d2022b6c76b1cb4186d40800d1affdd2b) Signed-off-by: Steve Sakoman

resulttool: Use single space indentation in json output

2024-12-09T14:54:14+00:00

Using 4 space indentation in resulted in hundreds of megabytes of extra file size in general use. Reduce this to make filesizes more managable and reduce the processing cost. Some level of indentation and spacing does make the files more readable and allows use of git diff so we need to retain some of it. (From OE-Core rev: 03fef3a99186863a0c460bd89bad40f2360a6930) Signed-off-by: Richard Purdie (cherry picked from commit a274cdcaf852cca9497f0358f44dda99c06aacbe) Signed-off-by: Steve Sakoman

resulttool: Allow store to filter to specific revisions

2024-12-09T14:54:14+00:00

We have a challenge on the autobuilder where test results from both OE-Core and poky are being mixed together during result storage which is confusing the data. Add a way to filter to specific revisions as the least worst way to fix the various issues this is causing. (From OE-Core rev: 974a6d5ebebc279df7c6994df7bd8c5d462e4447) Signed-off-by: Richard Purdie (cherry picked from commit 3f276a0dc65341668788853be2cf27ab6aa12b13) Signed-off-by: Steve Sakoman

scripts/resulttool: group all regressions in regression report

2023-11-05T08:44:06+00:00

Commit c304fcbe0588b1078373558c2ddf36064bcdf214 introduced a grouping when listing regressions. This grouping has been added only for ptests. It has been observed that any other kind of tests could benefit from it. For example, current regression reports can show the following: 1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED 1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED [...] This output is not so useful in its current state and should be grouped per test type too. Enable grouping for all kind of tests, to make it llok like the following in reports: 5 regression(s) for oescripts oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED (From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

scripts/resulttool: make additional info more compact

2023-10-23T09:49:19+00:00

Since "matched" and "improved" tests are not as important as regressions, reduce the place they take in the regression report: - merge "matched" and "improved" tests, while removing the label - add a single line of additional info per pair Those changes make the "Matches and improvements" look like the following sample: oeselftest_almalinux-9.2_qemux86-64_20230910083156 oeselftest_almalinux-8.8_qemux86-64_20231018010951 -> +7 test(s) present oeselftest_almalinux-9.2_qemux86-64_20230911010538 oeselftest_debian-11_qemux86-64_20231017150459 oeselftest_debian-11_qemux86-64_20230910012927 oeselftest_debian-11_qemux86-64_20231017151319 -> +7 test(s) present [...] (From OE-Core rev: 6de4426d9a7da67deed7d3a3918892fb56238ff3) Signed-off-by: Alexis Lothoré Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie