linux/poky.git/meta/lib/oeqa/runtime/cases/ptest.py, branch scarthgap

ptest: report tests that were killed on timeout

2023-09-18T07:53:10+00:00

I'm not sure if this was reported correctly before, but it currently is not. Test that is stuck is an error in itself. (From OE-Core rev: 002e27c9932a83e46be0b03a5232594cfba7212c) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

oeqa/runtime/ptest: Make returning no test results a failure

2023-05-03T06:33:18+00:00

Ensure that even if a ptests results section is empty, the log parser adds that empty section. Then ensure that empty sections trigger warnings. This means if a ptest suddently stops returning any results, we notice and see warnings about it. This has gone unnoticed on the autobuilder far too many times so is very much worth highlighting as a regression. We shouldn't have empty ptests. (From OE-Core rev: 5ad0cf57b41ec7f44647a03bc568d0b24906cc8d) Signed-off-by: Richard Purdie

lib: Add copyright statements to files without one

2022-08-12T11:00:43+00:00

Where there isn't a copyright statement, add one to make it explicit. Also add license identifiers as MIT if there isn't one. (From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040) Signed-off-by: Richard Purdie

oeqa/runtime/cases/ptest: Increase test timeout from 300s to 450s

2021-08-06T05:34:58+00:00

Some tests such as lttng-tools are marginal and timing out on the autobuilder with the current 300s default. Increase to avoid this noise in the ptest failures list. (From OE-Core rev: 5fb902a52e35130af6b0735a087c709daa35655f) Signed-off-by: Richard Purdie

oeqa/ptest: print a warning if ptests failed

2021-01-20T00:45:05+00:00

This allows spotting ptest regressions without having hard ptest failures (for that full ptest stability should be achieved). (From OE-Core rev: 79b54d902df2788b0822b6c9cc14705ab00e6a5b) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

oeqa/runtime/cases/ptest: Make output content path absolute

2020-06-23T11:31:03+00:00

The output content is created in current directory, because json content has no defined absolute path to WORKDIR as in bitbake. (From OE-Core rev: 14203b2cb9aa62f55cb12230ac8012b3cd995be7) Signed-off-by: Andrej Valek Signed-off-by: Peter Marko Signed-off-by: Richard Purdie

oeqa/runtime: Use libdir to run ptest-runner

2020-05-02T08:08:53+00:00

In multilib build configs libs can be installed in /usr/lib{32,64,x32} so use libdir to specify the correct ptest directory along with default /usr/lib. [YOCTO #12604] (From OE-Core rev: e44ca998c27f494466a524d09f751f963ec9be20) Signed-off-by: Aníbal Limón Signed-off-by: Richard Purdie

oeqa-runtime: add missing import os to ptest case

2020-03-06T08:19:18+00:00

Add missing import os statement to the oeqa runtime ptest.py (From OE-Core rev: e41c92c60694052c8292c1ea6b3d0b427aafafd7) Signed-off-by: Stefan Kral Signed-off-by: Richard Purdie

ptest: add an option to enforce a failure (e.g. ptests are required to pass)

2020-02-25T10:41:22+00:00

This can be useful with a more specific, targeted and robust set of ptest packages; the benefit is that ptest regressions are caught as they happen and can be more easily traced to changes that caused them. The existing AB ptest image continues to be expected to fail, my observation of the AB runs is that the full set of ptests is not robust enough (particularly around socket/network related tests) and sporadically fails in random places. This can probably be addressed by making ptests exclusive to a worker (e.g. there is no other workload happening at the same time as ptests). (From OE-Core rev: 102afcc7ea7d7673183f272a29ab233084ad168b) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

ptest: report ptests that couldn't be run at all

2020-01-10T21:18:21+00:00

Currently if a ptest does not produce PASS or FAIL, but simply errors out, this is not caught or reported; I think some ptests may have silently regressed due to this. (From OE-Core rev: 4b78b6571a3653d7d8e9dab9512bad5d39483988) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie