| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
There were warnings about non-existent files when there was
a duplicate in the remove-for-aarch list and the non-deterministic lists
so check if the files still exists before trying to deal with it.
Also restore the tests in the opposite order that they were hidden.
(From OE-Core rev: 2e1dc5782dcb879410d535f937c633edea484d11)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Install list of non-deterministic threaded ptests to be run using taskset
to force them to a single core. This commit works with b318944d7, which
updated the testing script to run the non-deterministic tests separately
but didn't install the list of tests, so these tests were being run
without taskset.
The taskset_nondeterministic_tests file is the list of tests that will
be run separately with taskset, and ignored during the other tests. This
is installed to /usr/lib/valgrind/ptest similar to the 2 existing lists
for tests to skip on ARM and all architectures.
Removed bar_bad and bar_bad_xml to be included separately as they cause
issues on non-kvm QEMU instances.
See:
b318944dd7 valgrind: Improve non-deterministic ptest reliability
for more info.
(From OE-Core rev: 3d23985d0d653844863ed513d75d93a36359992f)
Signed-off-by: Tony Tascioglu <tony.tascioglu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Several of the valgrind tests (particulaly helgrind) are unreliable and
can fail with a different output.
Particularly, there is a higher chance of failure on QEMU instances with
SMP enabled and on systems with more interrupts such as laptops on powersave.
The tests have been reported upstream as being unreliable dating back
over 5 years, due in part to the ordering of threads during
an "unwinding" process in helgrind.
https://bugs.kde.org/show_bug.cgi?id=345121
https://bugs.kde.org/show_bug.cgi?id=430321
A workaround to improve the reliability of such tests is to force them
to run on a single CPU core using taskset. This greatly reduces the
chance of a failure.
>From my testing, I have found it can help reduce the rate of failures
on both a laptop and QEMU by over 5x. Stress-testing in QEMU for several
hours did not result in a failure while running the test normally did.
The flaky or undeterministic thread-based tests are defined in the
taskset_nondeterministic_tests file. These test cases will be run with
taskset 0x00000001 to run on a single CPU core rather then the regular
test.
The edited run-ptest executes the flaky tests first, then ignores them
to not duplicate the results from the main tests. Everything modified is
restored when testing is complete.
The drawback is that this isn't a foolproof solution. It helps the tests
fail much less frequently, and considering how this issue has been documented
for a long time, a workaround such as this is needed.
(From OE-Core rev: b318944dd72ca7b0408e955f3599381ab3ac3ba8)
Signed-off-by: Tony Tascioglu <tony.tascioglu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Notable changes:
* library is now in libexecdir instead of libdir
Added patches:
* Add musl.supp: missing musl.supp in 3.17.0
Dropped backport patches:
* nlcontrolc: found in c79180a3afcf65902e578646c3b716cc749db406
* drd Fedora33: found in 15330adf7c2471fbaa6a0818db07078d81dbff97
* lmw lswi ppc64le: found in 74b74174d572fee4015b8f4e326db3cd949bcdc3
Other dropped patches
* helgrind intercept: found in d2d54dbcc74244adfc0c80b40862edf2b82f53b9
* drd musl fix: found in d2d54dbcc74244adfc0c80b40862edf2b82f53b9
TESTING RESULTS:
qemux86-64:
FAIL: drd/tests/swapcontext
3.17.0 3.16.1
===================
TOTAL: 736 726
PASSED: 694 688
FAILED: 1 0
SKIPPED: 41 38
(From OE-Core rev: 7c8c04ad933be38a806da355158c1e13e2c1b84c)
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Some intermittent failures in valgrind are hard
reproduce.
Printing the difference between actual and expected
will make understanding them slightly easier.
[YOCTO #14294]
(From OE-Core rev: 099313ef541920d4a84b801d9d8788a56ba7ec61)
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
exp-dhat:
commit 441bfc5f5 promoted exp-dhat to dhat
exp-sgcheck:
commit 40187fcd6 removed the exp-sgcheck tool.
(From OE-Core rev: f7b32e0d5bb9c80f6b37ed8b7f0bd29c562d3f70)
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The tests' output is non-deterministic and sometimes
doesn't match the sample output. This has been reported at
https://bugs.kde.org/show_bug.cgi?id=430321
(see also an earlier related bug https://bugs.kde.org/show_bug.cgi?id=358213)
Until upstream figures out how to fix this, let's not run the tests.
(From OE-Core rev: 7f7d2fa18267090891754d976cbc3e628324d3dd)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On qemuarm64 systems, about half of the valgrind tests fail and
often result in defunct processes, eg:
root 819 818 25 00:12 pts/0 00:01:52 /usr/bin/valgrind --command-line-only=yes --memcheck:leak-check=no --tool=memcheck -q --track-origins=
root 861 420 0 00:13 pts/0 00:00:00 [sh] <defunct>
Eventually these processes use so much memory that the
out of memory killer runs.
Hide the tests that fail as a horrid work-around until the
root cause has been resolved since this at least allows the
ptest run to complete. This work-around is done in the run-ptest
script using a sorted list of tests so that they can be easily
restored one by one without a rebuild during testing.
With core-image-minimal on qemuarm64:
Recipe | Passed | Failed | Skipped | Time(s)
valgrind | 333 | 49 | 17 | 7637
(From OE-Core rev: 208023f8fcbf4aee34544a80f962ae25f25ffb8d)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Make some changes to the run-ptest script:
- after main tests run integrity check like the pkg Makefile.
- aesthetic and ordering changes
Add the .in_place directory and its contents which allows valgrind
to be run in-place thereby enabling the gdbserver_tests to
complete rather than hang. Unfortunately directory paths embedded
in binaries still cause many of these test to fail.
Add the exp-sgcheck tests.
With core-image-minimal on qemux86-64/kvm:
Recipe | Passed | Failed | Skipped | Time(s)
before | 648 | 12 | 60 | 1541
after | 662 | 20 | 38 | 1429
ppc-no-gdbserv | 415 | 196 | 34 | 10689
Since fewer tests timeout, the overall time has decreased.
With core-image-sato on qemux86-64/kvm the results are now
the same as core-image-minimal.
qemuppc/arm64 runs result in the oom-killer eventually running since
some processes do not terminate properly and accumulate as defunct
processes in memory. Without the gdbserver_tests, the tests
complete without defunct process or the oom-killer running for
ppc but not for arm64.
(From OE-Core rev: 6dbaaeec17eae8329031188b688b33306a871870)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
| |
* Add statistic summary for run-ptest
* Ensure the script can be run anywhere
(From OE-Core rev: 8a6f5a2bc55975f38ad285ddea7122643b303c53)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
| |
|
|
|
|
|
|
|
|
| |
There is no bashism in the scripts, so use /bin/sh, and add /bin/sh
interpreter for systemd's run-ptest.
(From OE-Core rev: 62d455f89fb1d2c22cf987bdbb56a55e6d031ce0)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
Modifies valgrind's regression test framework to be compatible
with the yocto PTEST framework as follows:
* existing recipe valgrind*bb adds new methods: do_compile_ptest and
do_install_ptest.
* new file run-ptest adds the wrapper interface to the valgrind
regression test script vg_regtest.
* existing valgrind regression test script 'vg_regtest' changes
to report the status of the valgrind component tests in the
format that PTEST expects, instead of the valgrind formats, but only
when vg_regtest is invoked with an optional --yocto-ptest argument
* four new patches disable building tests that don't compile with
the yocto compiler and default options. See the patches for details.
(From OE-Core rev: d4438e421f448cdb7e25c038d657bbebc1b6486e)
Signed-off-by: Dave Lerner <dave.lerner@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
