summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/pam: Need shadow installed for the testsKhem Raj2021-03-181-0/+3
| | | | | | | | | | | | | | | | Distros may not use shadow for login manager secondly login --help returns 1 only with shadow provided login utility, with busybox it returns normal 0, the testcase clearly assumes shadow being part of image, therefore add needed checks to ensure the pre-requisites are met (From OE-Core rev: 0ac2dee8756573a793af56582565f221ff6b93c5) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0820aeba74869bf6e85b200f39bc610a43ab9a7a) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* df.py: Add feature check for read-only-rootfsWes Lindauer2021-03-041-0/+2
| | | | | | | | | | | | | | If the target is using a read-only rootfs, the available space on '/' will be zero. This will cause the test to incorrectly fail and skipping seems appropriate in this case. (From OE-Core rev: 69c8987da199933554a9dd0eda0a045a4534bfab) Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit af1dbea3c9b9b42a3e6803b231e425423d70e210) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runlevel : add test for runlevelsTeoh Jay Shen2021-03-041-0/+22
| | | | | | | | | | | | | This test mimic the boot_from_runlevel_3 and boot_from_runlevel_5 test cases from oeqa/manual/bsp-hw.json. The boot_from_runlevel_3 and boot_from_runlevel_5 manual test cases should be remove from oeqa/manual/bsp-hw.json if this patch get merged. (From OE-Core rev: 56d4c2867743b4b6bf79a4a9472799038522a331) Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6effd30f0e3726bc1f2eb7768c57c6b95eddb079) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/ptest: print a warning if ptests failedAlexander Kanavin2021-02-051-0/+1
| | | | | | | | | | | | | | | This allows spotting ptest regressions without having hard ptest failures (for that full ptest stability should be achieved). (From OE-Core rev: 6097b1cbffc048185c1ed658f16f0f90a9d19054) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 79b54d902df2788b0822b6c9cc14705ab00e6a5b) Signed-off-by: Steve Sakoman <steve@sakoman.com> (cherry picked from commit 5232c0b480eba4537c8db71c95189950011fb64d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: make copyFrom scp command non-fatalMatthew2020-08-121-2/+3
| | | | | | | | | | | | | | | [YOCTO #13802] Make the scp failure non-fatal so the ltp tests continue to run and the rest of the logs will be available to see afterwards. (From OE-Core rev: 9390846e2abdd6837094a619ba4703a1be4a04cf) Signed-off-by: Mingde (Matthew) Zeng <matthew.zeng@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0f7d093038274f4f21f6cca39a96aac4f6c32ee3) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logrotate.py: fix testimage occasionally failureChangqing Li2020-07-071-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | testcase test_systemd_failed occasionally failed with below error: Failed to start Rotate log files. logrotate.service: Failed with result 'exit-code'. logrotate.service: Main process exited, code=exited, status=1/FAILURE error: stat of /var/log/logrotate_test failed: No such file or directory error: logrotate_test:1 lines must begin with a keyword or a filename (possibly in double quotes) above failure caused since testcase test_logrotate_wtmp add /etc/logrotate.d/logrotate_test, which need /var/log/logrotate_test, but there is no such file. so when logrotate.service is triggerd by logrotate.timer after testcase test_logrotate_wtmp is runned, the testcase test_systemd_failed will fail. these 3 lines are useless, so remove them to fix above problem. (From OE-Core rev: 198fe0d64e4bdeda84c0f3e44afcc1d99ca9dbbe) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 22f5f7f86a4d47624c476be00e5121009c48cb7b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases/ptest: Make output content path absoluteAndrej Valek2020-07-021-0/+4
| | | | | | | | | | | | | | The output content is created in current directory, because json content has no defined absolute path to WORKDIR as in bitbake. (From OE-Core rev: 394d90efdc8d06dc61114bf6279ba6d094667289) Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 14203b2cb9aa62f55cb12230ac8012b3cd995be7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: ignore pulseaudio startup warning messagesChangqing Li2020-05-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | If set default syslog to rsyslog, we can see below messages in user.log, [pulseaudio] authkey.c: Failed to open cookie file [pulseaudio] authkey.c: Failed to load authentication key They are only warnings when cookie file is not found. And PulseAudio will create it if it doesn't exist. refer: https://wiki.archlinux.org/index.php/PulseAudio/Configuration https://lists.freedesktop.org/archives/pulseaudio-discuss/2014-December/022719.html (From OE-Core rev: 2cc3fac9cd1a0d77931c9e49dbe2941fa8619c51) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: update network interface related messagesChangqing Li2020-04-071-1/+2
| | | | | | | | | | along with systemd upgrade, error message related change network interface have changed, update it. (From OE-Core rev: a8b2cd7470bcc25527577b95a26a0a528949232d) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: default runtime json results for testexportStefan Kral2020-03-291-0/+1
| | | | | | | | | | | | | Set the json result output dir in the oeqe runtime context to create testresults.json file by default for exported runtime test runs. Use current datetime for the json result property name (not DATETIME from build) to allow multiple result entries. (From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f) Signed-off-by: Stefan Kral <sk@typedivision.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: Disable and stop systemd-timesyncdAlex Kiernan2020-03-291-2/+2
| | | | | | | | | | | | Stopping systemd-timesyncd doesn't prevent it being restarted by a different transaction within systemd. Disable the service instead during the date test to ensure it can't be restarted. (From OE-Core rev: 3dd4e637c11abdd6341a3e0c6b67639d3d703862) Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Acked-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemuarm64: Ignore logind: failed to get session seatKhem Raj2020-03-131-0/+1
| | | | | | | | | | | | | When booting weston images this error is seen commonly, but Qemu boots the image fine, session seat error is thrown by libweston perhaps using --seat option or setting XDG_SEAT variable in weston.ini could fix it [YOCTO #13828] (From OE-Core rev: 1f9d36722132c93c19651ea59062b3d5be01bb72) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: fix typoStefan Kral2020-03-121-2/+2
| | | | | | | (From OE-Core rev: bf959e6fd175d0841a1c042bf925c54200bea3d4) Signed-off-by: Stefan Kral <sk@typedivision.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: ignore rdrand initialization failureMingli Yu2020-03-071-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On the system whose cpu doesn't support rdrand, there comes below message when start rngd service #systemctl status rngd [snip] Feb 25 05:08:14 qemux86-64 rngd[133]: [rdrand]: Initialization Failed [snip] Actually the failed message doesn't matter as it only indicates one entropy source as rdrand fails to initialize and won't affect rngd function. So add to ignore the failure message to fix below error during do_testimage: NOTE: ====================================================================== NOTE: FAIL: test_parselogs (parselogs.ParseLogsTest) NOTE: ---------------------------------------------------------------------- NOTE: Traceback (most recent call last): File "/buildarea/layers/oe-core/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f return func(*args, **kwargs) File "/buildarea/layers/oe-core/meta/lib/oeqa/runtime/cases/parselogs.py", line 370, in test_parselogs self.assertEqual(errcount, 0, msg=self.msg) AssertionError: 1 != 0 : Log: /buildarea/tmp/work/qemux86-64-wrs-linux/wrlinux-image-std/1.0-r5/target_logs/daemon.log Central error: 2020-03-06T09:45:12.774286+00:00 qemux86-64 rngd[134]: [rdrand]: Initialization Failed Reference: https://github.com/nhorman/rng-tools/pull/84 (From OE-Core rev: 09c4f9068858bf59eb60584a497f987bdd1e196e) Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa-runtime: add missing import os to ptest caseStefan Kral2020-03-061-0/+1
| | | | | | | | | Add missing import os statement to the oeqa runtime ptest.py (From OE-Core rev: e41c92c60694052c8292c1ea6b3d0b427aafafd7) Signed-off-by: Stefan Kral <sk@typedivision.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: Update to 20200120Petr Vorel2020-03-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removed ffsb copying file and associated runtest file fs_ext4 (both removed in this release). Added LIBS=-lfts for MUSL. Removed patches (accepted upstream) * 0001-mkswap01.sh-Add-udevadm-trigger-before-swap-verifica.patch * 0001-testcases-fix-an-absent-format-string-issue.patch * 0001-nm01-Remove-prefix-zeros-of-the-addresses-output-by-.patch Merged patches * 0006-rt_tgsigqueueinfo-disable-test-on-musl.patch (into 0001-build-Add-option-to-select-libc-implementation.patch as they touch the same files) Refreshed patches (some tests have been fixed for musl) * 0001-Add-more-musl-exclusions.patch * 0001-build-Add-option-to-select-libc-implementation.patch * 0002-kernel-controllers-Link-with-libfts-explicitly-on-mu.patch IMHO this one shouldn't be needed not, but CI still fails without it Updated status * 0008-testcases-network-nfsv4-acl-acl1.c-Security-fix-on-s.patch * 0001-syscalls-Check-for-time64-unsafe-syscalls-before-usi.patch (From OE-Core rev: 62e2063698477a13c55ae370451e52a2f1e18e45) Signed-off-by: Petr Vorel <petr.vorel@gmail.com> Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ptest: add an option to enforce a failure (e.g. ptests are required to pass)Alexander Kanavin2020-02-251-1/+14
| | | | | | | | | | | | | | | | | | This can be useful with a more specific, targeted and robust set of ptest packages; the benefit is that ptest regressions are caught as they happen and can be more easily traced to changes that caused them. The existing AB ptest image continues to be expected to fail, my observation of the AB runs is that the full set of ptests is not robust enough (particularly around socket/network related tests) and sporadically fails in random places. This can probably be addressed by making ptests exclusive to a worker (e.g. there is no other workload happening at the same time as ptests). (From OE-Core rev: 102afcc7ea7d7673183f272a29ab233084ad168b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* weston: add a basic runtime testAlexander Kanavin2020-02-181-0/+19
| | | | | | | | | The test is checking that weston is able to start. (From OE-Core rev: 57700767f90eea8f2b78187c42581aca42d50bbf) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemux86: drop resolution setting via uvesafbAlexander Kanavin2020-02-181-4/+0
| | | | | | | | | | | | | | | | | | | | | | | I am not sure if this has ever worked, but uvesafb is a really outdated (VBE from the 1990s), awkward (needs v86d) and limited (no support for high resolutions) way to do it. The specific reason 640x480-32 was introduced (ages ago) was to force 32 bit mode with vmware driver, as 16bit had rendering issues. The modern, supported option is video=... kernel parameter documented here: https://wiki.archlinux.org/index.php/kernel_mode_setting#Forcing_modes_and_EDID https://github.com/torvalds/linux/blob/master/Documentation/fb/modedb.rst which can be passed directly to runqemu and doesn't require special kernel modules. Sato under X will continue to use 640x480 as that is hardcoded into xorg.conf under qemu. (From OE-Core rev: 1cf26f69fd89b43be24cd1232c43e5050b9d718a) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Use --disable-maintainer-mode configure optionKhem Raj2020-02-141-1/+1
| | | | | | | | | | | | | | | | | since the versions of autotools might differ on target and build host, plus difference in timestamps for configure and system can result in reconfigure lets avoid that by disabling maintainer mode Avoids error: newly created file is older than distributed files! [YOCTO #13779] Suggested-by: Andre McCurdy <armccurdy@gmail.com> (From OE-Core rev: 3dc4e27e7633ce3ca6b9647810d0996bdee48771) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Use cpio 2.13 as testcaseKhem Raj2020-02-081-1/+1
| | | | | | | | | | | | | | | | | | | | | cpio 2.12 was released in 2015 and might have used older autotools which could result in errors like https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779 Bumping to 2.13 will help in matching the tool versions A good change on top would be to run aclocal -I .; autoheader; autoconf; automake --add-missing -c before running configure step perhaps [YOCTO #13779] (From OE-Core rev: 84eb1dc4fe8a11cd2d05b703070a6fb6de05b873) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logrotate.py: improve oeqa test implementationTrevor Gamblin2020-02-051-18/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=13632 Autobuilder tests occasionally fail, reporting that a new logfile could not be created. While this failure did occur multiple times, it could not be manually reproduced. However, there are issues with the implementation of the logrotate.py script that can be fixed. These changes will help make the failures clearer, should they continue to occur. Previously, the test_2_logrotate test would, after running the logrotate tool, use "ls -al $HOME/logrotate_dir | wc -l" to count the number of files in the rotation directory and determine if the rotation was successful. The test to see if there are at least three files is problematic, because depending on the version of ls used, it may report the target value of 3 even when there are only hidden files in the directory, potentially reporting a pass for the test when it should actually fail. An example with coreutils: root@qemux86-64:~# ls -al emptydir/ total 2 drwxr-xr-x 2 root root 1024 Jan 14 19:50 . drwx------ 3 root root 1024 Jan 14 19:50 .. root@qemux86-64:~# Where "total" is the number of blocks used. Compare with busybox ls: root@qemux86-64:~# ls -al emptydir/ drwxr-xr-x 2 root root 1024 Jan 14 19:54 . drwx------ 3 root root 1024 Jan 14 19:54 .. root@qemux86-64:~# Instead of using ls to verify that a certain number of files exists in $HOME/logrotate_dir, the tests have been changed to rotate two specific logs: the log for wtmp and a new logrotate_testfile created during the second test. Both tests check that the logs are correctly rotated into $HOME/logrotate_dir by using find and grep on the expected filename (e.g. "wtmp" when rotated becomes "wtmp.1", so we check to see that wtmp.1 is present in $HOME/logrotate_dir). In addition, should the test fail, the output from logrotate -vf is included in the test log to aid debugging. It has also been seen that in some cases, the logrotate test fails because the /var/log/wtmp file is not yet present. Since the objective of the test is to check the logrotate functionality and not the presence of certain log files, test_logrotate_wtmp uses the touch command to help ensure that the file is present before the call to logrotate is issued. Finally, note that while the autobuilder failures that this patch addresses were only seen during core-image-full-cmdline tests, these changes were successfully tested on core-image-minimal and core-image-sato with the manual addition of logrotate and openssh-sshd to the images. (From OE-Core rev: b9392403e96c6d880de844b44da2c7300fcf411c) Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs: remove mpc8315e-rdb from errorsArmin Kuster2020-02-021-5/+0
| | | | | | | (From OE-Core rev: 0de55be071efff60944c89582ae0e736f3bdc828) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: update to 20190930Alexander Kanavin2020-01-161-1/+1
| | | | | | | | | | | | | | | Drop a big pile of backports. Add a patch to address absent string formatting error. Additional fixing of builds with musl. timers group has been merged upstream into syscalls. (From OE-Core rev: 7126431cc3512e3d7f55300c561339c87283b605) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ptest: report ptests that couldn't be run at allAlexander Kanavin2020-01-101-0/+5
| | | | | | | | | | | Currently if a ptest does not produce PASS or FAIL, but simply errors out, this is not caught or reported; I think some ptests may have silently regressed due to this. (From OE-Core rev: 4b78b6571a3653d7d8e9dab9512bad5d39483988) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: fix detection of failed testsAndré Draszik2019-12-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since commit d6065f136f6d ("oeqa/logparser: Various misc cleanups"), 7b17274c30c6 in poky, the ptest OEQA is unable to detect failures in any of the test results. The reason is that the test result string changed from 'fail' to 'FAILED', because the original mapping has been removed as part of that commit, but the code in here is still trying to match against the old string, resulting in no matches, i.e. everything is treated as successful, even if it shouldn't be. Update the OEQA ptest test to actually work again and report failure if there was a failure. Note that the ptest test is marked as @expectedfail, so even though this test now again starts to fail, the overall OEQA test result is not affected - but at least the overall OEQA test summary reflects the correct status again. In other words: RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.26s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner: PASSED (4.05s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.60s) SUMMARY: image-debug () - Ran 3 tests in 4.937s correctly changes to: AssertionError: Failed ptests: {'dummytest': ['check_True_is_True', 'test_basic']} RESULTS: RESULTS - ping.PingTest.test_ping: PASSED (0.24s) RESULTS - ssh.SSHTest.test_ssh: PASSED (0.56s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner: EXPECTEDFAIL (4.13s) SUMMARY: image-debug () - Ran 3 tests in 4.937s instead and we see a summary of the ptest subtests that failed. (From OE-Core rev: 08ff4b9195a9de91b8090173c5bd03a5ff263616) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sysklogd: update to 2.0.3Alexander Kanavin2019-12-181-1/+2
| | | | | | | | | | | | | | | | | | | | Drop all patches as they are no longer necessary with autotools, fix problems that are addressed upstream, or (in case of no-vectorization.patch) don't apply to the old version either, which means no one is using that hw target anymore. Remove custom systemd configs (provided by upstream), and custom syslog config (also provided by upstream but not installed by default). Adjust parselogs to not error out on not being able to find System.map file during boot (that is packaged into kernel-dev and is not normally installed). License-Update: various tweaks; the license terms are the same. (From OE-Core rev: dcf0bd14d78b0848f206eb35355f05bfbd51c02d) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/apt dnf opkg: support running from within docker containerAndré Draszik2019-12-163-3/+8
| | | | | | | | | | | | | | | | | If the user specified an http port to use for serving files, use that instead of a random one. At the same time, have the http server bind to all interfaces. Binding to the server_ip might not always be possible, e.g. in the case of running bitbake / oeqa from within a docker container. In this case, the ip address is valid outside the container, but not inside, and hence can't be bound to. So switch to simply binding to all interfaces. (From OE-Core rev: da529e6522959c2024023e90de52b5e42f160596) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: support listening port in TEST_SERVER_IPAndré Draszik2019-12-161-0/+6
| | | | | | | | | | | | | | | | | Similar to the existing possibility to specify a port in TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a port. The intention is for this port to be passed into e.g. the http server from the apt / dnf / opkg tests, or any other (custom) tests that might need the target to connect to a service spawned by bitbake / oeqa, where bitbake is actually running inside a docker container. (From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual/compliance-test: move crashme to runtimeArmin Kuster2019-11-211-0/+98
| | | | | | | | | | | [v3] remove fork12 from crashme. runs forever missed in v2 resend (From OE-Core rev: 5d32ff52a36bdcc6abe55b89bf81c3312a03160a) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual/bsp-hw: move storage tests to runtimeArmin Kuster2019-11-211-0/+149
| | | | | | | | | | move usb mount, r/w and umount tests to runtime move SD card mount, r/w and umount tests to runtim (From OE-Core rev: 246d73b35f6be7880d98274e87033a14cdced9ba) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/boot: add reboot testArmin Kuster2019-11-211-0/+33
| | | | | | | (From OE-Core rev: c8ddebee04a7c752986b407b98f4d0175f447d01) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: ignore more files when loading controllersAndré Draszik2019-10-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When loading controllers as (external) modules, the code currently tries to load all files ending with .py. This is a problem when during development using an editor that creates a lock-file in the same directory as the .py file, as the lock file is typically called '.#xxxx.py'. Python will try to load the lock file and fail miserably with an exception: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_testimage(d) 0003: File: 'poky/meta/classes/testimage.bbclass', lineno: 114, function: do_testimage 0110: netstat -an 0111:} 0112: 0113:python do_testimage() { *** 0114: testimage_main(d) 0115:} 0116: 0117:addtask testimage 0118:do_testimage[nostamp] = "1" File: 'poky/meta/classes/testimage.bbclass', lineno: 294, function: testimage_main 0290: 0291: # the robot dance 0292: target = OERuntimeTestContextExecutor.getTarget( 0293: d.getVar("TEST_TARGET"), logger, d.getVar("TEST_TARGET_IP"), *** 0294: d.getVar("TEST_SERVER_IP"), **target_kwargs) 0295: 0296: # test context 0297: tc = OERuntimeTestContext(td, logger, target, host_dumper, 0298: image_packages, extract_dir) File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 116, function: getTarget 0112: # XXX: Don't base your targets on this code it will be refactored 0113: # in the near future. 0114: # Custom target module loading 0115: target_modules_path = kwargs.get('target_modules_path', '') *** 0116: controller = OERuntimeTestContextExecutor.getControllerModule(target_type, target_modules_path) 0117: target = controller(logger, target_ip, server_ip, **kwargs) 0118: 0119: return target 0120: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 128, function: getControllerModule 0124: # ImportError raised if a provided module can not be imported. 0125: @staticmethod 0126: def getControllerModule(target, target_modules_path): 0127: controllerslist = OERuntimeTestContextExecutor._getControllerModulenames(target_modules_path) *** 0128: controller = OERuntimeTestContextExecutor._loadControllerFromName(target, controllerslist) 0129: return controller 0130: 0131: # Return a list of all python modules in lib/oeqa/controllers for each 0132: # layer in bbpath File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 163, function: _loadControllerFromName 0159: # Raise ImportError if a provided module can not be imported 0160: @staticmethod 0161: def _loadControllerFromName(target, modulenames): 0162: for name in modulenames: *** 0163: obj = OERuntimeTestContextExecutor._loadControllerFromModule(target, name) 0164: if obj: 0165: return obj 0166: raise AttributeError("Unable to load {0} from available modules: {1}".format(target, str(modulenames))) 0167: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 173, function: _loadControllerFromModule 0169: @staticmethod 0170: def _loadControllerFromModule(target, modulename): 0171: obj = None 0172: # import module, allowing it to raise import exception *** 0173: module = __import__(modulename, globals(), locals(), [target]) 0174: # look for target class in the module, catching any exceptions as it 0175: # is valid that a module may not have the target class. 0176: try: 0177: obj = getattr(module, target) Exception: ImportError: No module named 'oeqa.controllers.' Simply ignore those when collecting the list of files to try to load. (From OE-Core rev: 682f223cf2e2dabe8cf60634b6779bb2d5e359bd) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/systemd: skip unit enable/disable on read-only-rootfsAndré Draszik2019-10-191-1/+15
| | | | | | | | | | | | | | | | | | | | This doesn't work on read-only-rootfs: AssertionError: 1 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl disable avahi-daemon.service Failed to disable unit: File /etc/systemd/system/multi-user.target.wants/avahi-daemon.service: Read-only file system This patch does two things: 1) Decorate the existing test to be skipped if the rootfs is read-only 2) add a new test to be executed only if the rootfs is read-only. This new test remounts the rootfs read-write before continuing to execute the existing test, making sure to clean up correctly after itself (remount r/o again). (From OE-Core rev: ec856301887b1139f93aa54d04bed9b842357b4f) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/opkg: skip install on read-only-rootfsAndré Draszik2019-10-191-1/+3
| | | | | | | | | | | | | | | Images can have package management enabled, but be generally running as read-only. In this case, the test fails at the moment with various errors due to that. Use the new @skipIfFeature decorator to also skip this test in that case. (From OE-Core rev: 84640e2b4daf4cf22c5b0324c22332f59e4d51e3) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/df: don't fail on long device namesAndré Draszik2019-10-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | When device names are long (more than 20 characters), the df test will fail with an exception: self.assertTrue(int(output)>5120, msg=msg) ValueError: invalid literal for int() with base 10: '' at least when busybox is in use. The reason is that busybox breaks the line in that case: Filesystem 1K-blocks Used Available Use% Mounted on /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02 1998672 87024 1790408 5% / and the code tries to extract the fourth field from the second line, which is empty of course. df can be told not to break lines, though, using the -P flag, which turns on the POSIX output format, and is supported by busybox df and coreutils df: Filesystem 1024-blocks Used Available Capacity Mounted on /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02 1998672 87024 1790408 5% / (From OE-Core rev: 8c23c1476d0c64b9bc8806db03414fa914c1e658) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* dnf.py: check busybox for case test_dnf_installrootKai Kang2019-10-091-0/+2
| | | | | | | | | | Check package busybox which is required by cases test_dnf_installroot and test_dnf_installroot_usrmerge. (From OE-Core rev: 0f09c5b7a9a66af825951354b436b69dd9a596a4) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Recognise svlogd as another loggerKhem Raj2019-09-071-1/+1
| | | | | | | | | | This is provided by runit which another init system like systemd, sysvinit, this lets oeqa run on systems which are using runit (From OE-Core rev: e4eb3ca113985ab2be123eb5b5ea76764761df88) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* dnf.py: installroot support usrmergeChangqing Li2019-09-071-1/+33
| | | | | | | (From OE-Core rev: 65f8dfd81abcf48d472ee28c2cdae819fd87ef32) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mipsAlexander Kanavin2019-09-011-0/+4
| | | | | | | | | | | | | | | | | | | | This is the qemu default since qemu 2.2, is generally supported better, and is recommended by upstream. It also has already been in use for arm/risc and ovmf. Additional information: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13466 https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/ '-vga virtio' emulated hardware remains in use when virgl is enabled via a runqemu override. Also, adjust the error whitelist, as there is a number of new messages coming from the drivers that are not actual errors. (From OE-Core rev: 73cb104f3307736f4922f2e0c9648f9b2d3b3b6b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/parselogs: grep for exact errors list keywordsKhem Raj2019-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Given grep expression looks for error strings regardless of the semantics and this ends up it flagging things like *********************** Log: /mnt/a/yoe/build/tmp/work/raspberrypi3_64-yoe-linux-musl/yoe-qt5-wayland-image/1.0-r0/target_logs/weston.log ----------------------- Central error: EGL_KHR_create_context_no_error EGL_KHR_fence_sync *********************** here EGL_KHR_create_context_no_error is an API, therefore the check should be tightened a bit so explicitly look for error words so such false positives don't show up This helps in fixing wrong matches e.g. on rpi3 dmesg have strings like above (From OE-Core rev: e54ecea0a7d309533973b31bdbda178d9af92632) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/buildgalculator.py: Add dependency on gtk+3Khem Raj2019-08-141-0/+1
| | | | | | | | | | | This helps in skipping this test on images which are not using gtk and as per the gcalculator website it needs gtk+3 or gtk+2 and libglade as the build first looks for gtk+3 (From OE-Core rev: 2cdb4b3697e02947352803a9db013d61b812c4fd) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: Move test_rpm_query_nonroot test case to RpmBasicTestChen Qi2019-07-151-29/+30
| | | | | | | | | | The test_rpm_query_nonroot test case was in RpmInstallRemoveTest. But it should logically belong to RpmBasicTest. So move it there. (From OE-Core rev: 506388a10a26613524602dcb4e630f216c6fee60) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: ensure no user process running before deleting userChen Qi2019-07-151-0/+15
| | | | | | | | | | | | | | | | | | In case of systemd, `su -c 'xxx' test1' via ssh will create several processes owned by test1, e.g. /lib/system/systemd --user. These processes are actually managed by user@UID.service (e.g. user@1000.service). And such service is managed automatically by systemd. In other words, it will be cleaned up by systemd automatically. So we need to wait for systemd to clean it up before trying to use `userdel' to delete the user. (From OE-Core rev: 9d398be42a69d25277b929d760aaed1679f3cd54) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/syslog: Add delay to test to avoid failuresRichard Purdie2019-06-271-0/+3
| | | | | | | | | | | On a loaded builder we've seen the log message not make it to the log file before the ssh command completes. Add a short delay to try and ensure this does happen. There is unforunately no way to flush syslog in all cases we test. (From OE-Core rev: 66322b689e46520647e2d94d5e3f3ce282a41247) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/oesyslog: systemd syslog restart doesn't change pidRichard Purdie2019-06-271-2/+5
| | | | | | | | | The systemd-journald process doesn't restart/change the way syslog does, don't test/error in this case. (From OE-Core rev: be48190dca0643df4881624d29c2eae453395919) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/syslog: Improve test debug messagesRichard Purdie2019-06-271-12/+20
| | | | | | | | | | | | Its useful to test whether the restart command returned an error code and exit early from the test if so. Also add different messages to tell if the syslog processes didn't die or didn't restart. (From OE-Core rev: f19e95b8571a0d8213c4dec0da056e3d243fbbd1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe_syslog.py: Handle syslogd/klogd restart raceJon Mason2019-06-271-9/+59
| | | | | | | | | | | | | | | | syslogd and klogd can occasionally take too long to restart, which causes tests to fail by starting before the log daemons are ready. To work around this problem, poll for up to 30 seconds on the processes to verify the old ones are killed and the new ones are up and running. Similarly, add checks for rsyslogd and systemd-journald to possibly catch issues with those daemons. [YOCTO #13379] (From OE-Core rev: dc73872b828ea271678fa624c15199364a5cba9e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime/cases/logrotate: make test more reliableAnuj Mittal2019-06-191-1/+3
| | | | | | | | | | | | | | | | | | | | | By default logrotate uses 'dateext' in logrotate.conf which results in a date string being appended at the end of log name. In cases when a test that installs configuration in logrotate.d is executed first (dnf for example), it might result in errors when logrotate test is executed: error: destination /var/log/dnf.librepo.log-20190528 already exists, skipping rotation error: destination /var/log/dnf.rpm.log-20190528 already exists, skipping rotation error: destination /var/log/hawkey.log-20190528 already exists, skipping rotation Tweak the test to use it's own temporary configuration that doesn't enable dateext. Fixes [YOCTO #13363] (From OE-Core rev: d0a30c1c57edc9a724658e2ed312b34eed3af5a9) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: add simple test for sconsTim Orling2019-06-083-0/+43
| | | | | | | | | This test simply compiles a hello world program using scons. (From OE-Core rev: bf6e3f0f3a7a134e8e3cb16366ef01b8c956e4c8) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-14 03:04:16 +0000