summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
Commit message (Collapse)AuthorAgeFilesLines
* dnf.py: check busybox for case test_dnf_installrootKai Kang2019-10-091-0/+2
| | | | | | | | | | Check package busybox which is required by cases test_dnf_installroot and test_dnf_installroot_usrmerge. (From OE-Core rev: 0f09c5b7a9a66af825951354b436b69dd9a596a4) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Recognise svlogd as another loggerKhem Raj2019-09-071-1/+1
| | | | | | | | | | This is provided by runit which another init system like systemd, sysvinit, this lets oeqa run on systems which are using runit (From OE-Core rev: e4eb3ca113985ab2be123eb5b5ea76764761df88) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* dnf.py: installroot support usrmergeChangqing Li2019-09-071-1/+33
| | | | | | | (From OE-Core rev: 65f8dfd81abcf48d472ee28c2cdae819fd87ef32) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mipsAlexander Kanavin2019-09-011-0/+4
| | | | | | | | | | | | | | | | | | | | This is the qemu default since qemu 2.2, is generally supported better, and is recommended by upstream. It also has already been in use for arm/risc and ovmf. Additional information: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13466 https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/ '-vga virtio' emulated hardware remains in use when virgl is enabled via a runqemu override. Also, adjust the error whitelist, as there is a number of new messages coming from the drivers that are not actual errors. (From OE-Core rev: 73cb104f3307736f4922f2e0c9648f9b2d3b3b6b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/parselogs: grep for exact errors list keywordsKhem Raj2019-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Given grep expression looks for error strings regardless of the semantics and this ends up it flagging things like *********************** Log: /mnt/a/yoe/build/tmp/work/raspberrypi3_64-yoe-linux-musl/yoe-qt5-wayland-image/1.0-r0/target_logs/weston.log ----------------------- Central error: EGL_KHR_create_context_no_error EGL_KHR_fence_sync *********************** here EGL_KHR_create_context_no_error is an API, therefore the check should be tightened a bit so explicitly look for error words so such false positives don't show up This helps in fixing wrong matches e.g. on rpi3 dmesg have strings like above (From OE-Core rev: e54ecea0a7d309533973b31bdbda178d9af92632) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/buildgalculator.py: Add dependency on gtk+3Khem Raj2019-08-141-0/+1
| | | | | | | | | | | This helps in skipping this test on images which are not using gtk and as per the gcalculator website it needs gtk+3 or gtk+2 and libglade as the build first looks for gtk+3 (From OE-Core rev: 2cdb4b3697e02947352803a9db013d61b812c4fd) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: Move test_rpm_query_nonroot test case to RpmBasicTestChen Qi2019-07-151-29/+30
| | | | | | | | | | The test_rpm_query_nonroot test case was in RpmInstallRemoveTest. But it should logically belong to RpmBasicTest. So move it there. (From OE-Core rev: 506388a10a26613524602dcb4e630f216c6fee60) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: ensure no user process running before deleting userChen Qi2019-07-151-0/+15
| | | | | | | | | | | | | | | | | | In case of systemd, `su -c 'xxx' test1' via ssh will create several processes owned by test1, e.g. /lib/system/systemd --user. These processes are actually managed by user@UID.service (e.g. user@1000.service). And such service is managed automatically by systemd. In other words, it will be cleaned up by systemd automatically. So we need to wait for systemd to clean it up before trying to use `userdel' to delete the user. (From OE-Core rev: 9d398be42a69d25277b929d760aaed1679f3cd54) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/syslog: Add delay to test to avoid failuresRichard Purdie2019-06-271-0/+3
| | | | | | | | | | | On a loaded builder we've seen the log message not make it to the log file before the ssh command completes. Add a short delay to try and ensure this does happen. There is unforunately no way to flush syslog in all cases we test. (From OE-Core rev: 66322b689e46520647e2d94d5e3f3ce282a41247) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/oesyslog: systemd syslog restart doesn't change pidRichard Purdie2019-06-271-2/+5
| | | | | | | | | The systemd-journald process doesn't restart/change the way syslog does, don't test/error in this case. (From OE-Core rev: be48190dca0643df4881624d29c2eae453395919) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/syslog: Improve test debug messagesRichard Purdie2019-06-271-12/+20
| | | | | | | | | | | | Its useful to test whether the restart command returned an error code and exit early from the test if so. Also add different messages to tell if the syslog processes didn't die or didn't restart. (From OE-Core rev: f19e95b8571a0d8213c4dec0da056e3d243fbbd1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe_syslog.py: Handle syslogd/klogd restart raceJon Mason2019-06-271-9/+59
| | | | | | | | | | | | | | | | syslogd and klogd can occasionally take too long to restart, which causes tests to fail by starting before the log daemons are ready. To work around this problem, poll for up to 30 seconds on the processes to verify the old ones are killed and the new ones are up and running. Similarly, add checks for rsyslogd and systemd-journald to possibly catch issues with those daemons. [YOCTO #13379] (From OE-Core rev: dc73872b828ea271678fa624c15199364a5cba9e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime/cases/logrotate: make test more reliableAnuj Mittal2019-06-191-1/+3
| | | | | | | | | | | | | | | | | | | | | By default logrotate uses 'dateext' in logrotate.conf which results in a date string being appended at the end of log name. In cases when a test that installs configuration in logrotate.d is executed first (dnf for example), it might result in errors when logrotate test is executed: error: destination /var/log/dnf.librepo.log-20190528 already exists, skipping rotation error: destination /var/log/dnf.rpm.log-20190528 already exists, skipping rotation error: destination /var/log/hawkey.log-20190528 already exists, skipping rotation Tweak the test to use it's own temporary configuration that doesn't enable dateext. Fixes [YOCTO #13363] (From OE-Core rev: d0a30c1c57edc9a724658e2ed312b34eed3af5a9) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: add simple test for sconsTim Orling2019-06-083-0/+43
| | | | | | | | | This test simply compiles a hello world program using scons. (From OE-Core rev: bf6e3f0f3a7a134e8e3cb16366ef01b8c956e4c8) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-0940-21/+155
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Drop OETestIDRichard Purdie2019-05-0928-85/+0
| | | | | | | | | | These IDs refer to testopia which we're no longer using. We would now use the test names to definitively reference tests and the IDs can be dropped, along with their supporting code. (From OE-Core rev: 8e2d0575e4e7036b5f60e632f377a8ab2b96ead8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp_compliance: add new runtimeArmin Kuster2019-04-291-0/+104
| | | | | | | | | | | | | | | | | test runtimes in sec. AIO: 14 MEM: 94 MSG: 89 SEM: 30 SIG: 194 THR: 399 TMR: 867 TPS: 23 (From OE-Core rev: e099204c34d1cf90ae4bb44fb0f56e72904c664a) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: add runtime testArmin Kuster2019-04-291-0/+124
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds the framework for running ltp tests. Here are some times: math: 61 syscalls: 3957 dio: 18472 io: 29 mm: 551 ipc: 48 sched: 165 nptl: 46 pty: 37 containers: 52 controllers: 9625 filecaps: 27 cap_bounds: 27 fcntl-locktests: 29 connectors: 27 timers: 37 commands: 165 net.ipv6_lib: 30 input: 29 fs_perms_simple: 31 fs: 3476 fsx: 30 fs_bind: 28 fs_ext4: 28 cve: 675 (From OE-Core rev: 1ec677549d490a29a7c713610db4a56be7b55e26) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: ignore regulatory.db load failure messages for x86*Naveen Saini2019-03-031-0/+2
| | | | | | | | | [YOCTO #13098] (From OE-Core rev: 9c20f8c6e4bb5157550e347ffd6ef790eb6a1200) Signed-off-by: Naveen Saini <naveen.kumar.saini@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logrotate.py: restore /etc/logrotate.d/wtmpMingli Yu2019-02-201-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During the test logrotate.LogrotateTest.test_1_logrotate_setup, there is below logic: # mkdir $HOME/logrotate_dir # sed -i "s#wtmp {#wtmp {\n olddir $HOME/logrotate_dir#" /etc/logrotate.d/wtmp After all logrotate.LogrotateTest finished, only cleanup $HOME/logrotate_dir as below, but don't restore the config file /etc/logrotate.d/wtmp. [snip] def tearDownClass(cls): cls.tc.target.run('rm -rf $HOME/logrotate_dir') [snip] That's to say, there is one additional line added to /etc/logrotate.d/wtmp and will make the logrotate service start failed when run systemd.SystemdBasicTests.test_systemd_failed Take an example as below when run test as root: # cat /etc/logrotate.d/wtmp # no packages own wtmp -- we'll rotate it here /var/log/wtmp { olddir /root/logrotate_dir missingok monthly create 0664 root utmp minsize 1M rotate 1 } # ls /root/logrotate_dir ls: cannot access '/root/logrotate_dir': No such file or directory # systemctl start logrotate Job for logrotate.service failed because the control process exited with error code. See "systemctl status logrotate.service" and "journalctl -xe" for details. # systemctl status logrotate logrotate.service - Rotate log files Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset> Active: failed (Result: exit-code) since Wed 2019-02-13 03:35:19 UTC; 7s ago Docs: man:logrotate(8) man:logrotate.conf(5) Process: 540 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE) Main PID: 540 (code=exited, status=1/FAILURE) Feb 13 03:35:18 qemumips systemd[1]: Starting Rotate log files... Feb 13 03:35:19 qemumips logrotate[540]: error: wtmp:9 error verifying olddir path /root/logrotate_dir: No such file or directory Feb 13 03:35:19 qemumips logrotate[540]: error: found error in file wtmp, skipping Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Failed with result 'exit-code'. Feb 13 03:35:19 qemumips systemd[1]: Failed to start Rotate log files. Add the logic to restore /etc/logrotate.d/wtmp to make the cleanup complete to fix the above issue. (From OE-Core rev: a2db9320d97d12d87524ff16a329f9c38a8da33f) Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/runtime/cases/gi.py: fix deprecation warningAlexander Kanavin2019-02-021-1/+1
| | | | | | | | | | | With newest Python, the following is printed: PyGIDeprecationWarning: GObject.markup_escape_text is deprecated; use GLib.markup_escape_text instead) (From OE-Core rev: 7b4302d84e6c9833089bd575c1083fe826082387) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: Ensure OOM errors are loggedRichard Purdie2019-02-021-1/+9
| | | | | | | | | | | Currently processed being killed by the OOM killer may not be spotted by ptest-runner. After we complete the tests, check the logs and report if there were any. This ensures the user is aware of OOM conditions affecting the ptest results. (From OE-Core rev: 20a441d53817f80e0ce1597e77f6e794422ac49a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* perl: add testdepends for sshArmin Kuster2019-01-311-0/+2
| | | | | | | | | | | | | | | fixes: DEBUG: [Running]$ ssh -l root -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR 192.168.7.4 export PATH=/usr/sbin:/sbin:/usr/bin:/bin; perl -e '$_="Uryyb, jbeyq"; tr/a-zA-Z/n-za-mN-ZA-M/;print' | DEBUG: time: 1548816904.4024463, endtime: 1548817204.397057 | DEBUG: Partial data from SSH call: ssh: connect to host 192.168.7.4 port 22: Connection refused for master/thud/sumo (From OE-Core rev: a590e7805e3bec5dd995f7ea0b9e79a21f82b48b) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/logparser: Various misc cleanupsRichard Purdie2019-01-311-8/+5
| | | | | | | | | | | | | Get rid of further unneeded code complications: * value mappings we could just direct use * ftools when we can write files easily ourself * test result status filtering we don't use * variable overwriting module imports (From OE-Core rev: d6065f136f6d353c3054cc3f440a4e259509f876) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/logparser: Improve results handlingRichard Purdie2019-01-311-6/+8
| | | | | | | | | | | | | | | | | Merge the results handling into the ptest log parser as a seperate method. Drop the weird "pass.skip.fail." prefix to the results filename, its just bizarre. Drop the code turning a list into a regex then searching the regex for an item, "x in y" is perfectly capable. Use a dict, sort the keys as needed and drop the list sorting code. (From OE-Core rev: f317800e950b4a37b4034133bc52e0c47f04dc29) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/logparser: Add in support for duration, exitcode and logs by sectionRichard Purdie2019-01-311-1/+3
| | | | | | | | | | | Allow parsing of the ptest duration, exit code and timeout keywords from the logs, returning data on each section. Also include the logs broken out per section. (From OE-Core rev: a9a67dccaa5be0f06eedcab46dcff7cbf9202850) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/logparser: Reform the ptest results parserRichard Purdie2019-01-311-37/+3
| | | | | | | | | | Now we have a dedicated ptest parser, merge in the remaining ptest specific pieces to further clarify and simplify the code, moving to a point where we can consider extending/enhancing it. (From OE-Core rev: 05991bb5bc8018275d03fdeecee3d5a757840c7c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/logparser: Further simplification/clarificationRichard Purdie2019-01-311-3/+3
| | | | | | | | | Rename the paster to be ptest specific and apply some further cleanups to the code to simplify and clarify what its doing. (From OE-Core rev: 45a5886f1ec458d4c306b8d68fd31d568bc36b47) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/logparser: Simplify ptest log parsing codeRichard Purdie2019-01-311-6/+1
| | | | | | | | | | logparser is only used by ptest. Its slightly overcomplicated as it was intended to be reusable but wasn't. Simplify it as a dedicated parser is likely to me more readable and maintainable. (From OE-Core rev: c7478345b2b4a85cb1fec40e762633871f0e94cb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: Avoid traceback for tests with no sectionRichard Purdie2019-01-311-1/+1
| | | | | | | | | Some tests end up without a section, avoid tracebacks trying to use None as a string in that case. (From OE-Core rev: 86fb5d898a29761f120c2eaa538a32cf2e078487) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime/ksample.py: correct a syntax and perfect item about architectureHongzhi.Song2019-01-241-3/+3
| | | | | | | (From OE-Core rev: 997ab2f5b9faaf656d677392224325f2a2cbc966) Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: ignore failure messages for beaglebone edgerouter and mpc8315e-rdbYi Zhao2019-01-221-0/+16
| | | | | | | (From OE-Core rev: fb74c4cd4d123da270e58578644857c9a37b6c64) Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ksample.py: skip kprobe case except x86 and powerpcHongzhi.Song2019-01-181-0/+7
| | | | | | | | | | | | | | | A new commit was introduced to kernel, which caused kprobe sample case failed on arm. [kernel commit: e46daee53bb50b, ARM: 8806/1: kprobes: Fix false positive with FORTIFY_SOURCE] And according to author, the case just works on x86 and powerpc. So we just promise that the case can pass on x86 and powerpc. (From OE-Core rev: 1a5b14bc3fe29618beea016ab2b32f55c19dd6ad) Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: whilelist one more amba error messageChen Qi2019-01-161-0/+1
| | | | | | | | | | When using linux-yocto-dev, we will have one more amba error message for qemuarm, so ignore it too. (From OE-Core rev: 37cb293cf0f4194ec923d3bc4ebb9645f07536c0) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: make it work for multiple usersRobert Yang2019-01-111-4/+1
| | | | | | | | | | | | | | | | | | | | | There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. (From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemu & runtime: qemu do not need ip input from externalYeoh Ee Peng2018-11-231-1/+1
| | | | | | | | | | | Qemu do not use the ip input from external. It will retrieve ip from QemuRunner instance and assign ip value. (From OE-Core rev: 14d99dc6c39c963ba3e0d9a30274846bd5369210) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/httpserver: Rework to avoid hangs and improve loggingRichard Purdie2018-11-163-3/+3
| | | | | | | | | | | | | | | | | | testimage.bbclass installs a SIGTERM handler which conflicts with the use of multiprocessing here. This is paritcularly problematic if the http service is terminated before its started and hence before its had a chance to reset the default signal handler (as the code was written). Instead, temporarily remove testimage's handler whilst forking the http process which means the correct handler is installed and won't deadlock. Also take the opportunity to add in some log messages about the server start and shutdown so that future debugging is easier and its clearer what the code is doing. (From OE-Core rev: 0762b6021b87ceb1f37952f3a6d64a36e99ae6a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: Improve dependencies of kernel/gcc/build testsRichard Purdie2018-11-146-30/+31
| | | | | | | | | Mark up these tests as needing a compiler, make and kernel source code as appropriate, the image feature requirements can then be retired. (From OE-Core rev: 9f64e6c25abdf494fb511e9cd401f8dcaa08be2a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: Improve test dependency informationRichard Purdie2018-11-1410-25/+32
| | | | | | | | | | | | Add the OEHasPackage decorator to a variety of tests so they determine automatically if they should run against a given image. To ensure tests can do this we need to move target operations such as scp commands into the tests and out of the class startup/teardown. (From OE-Core rev: 60d6580b85714b8960a964e775d76a7f937f5e5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/utils/targetbuildproject: Use a subdir within ~/Richard Purdie2018-11-141-1/+3
| | | | | | | | | Without doing this, the code can and sometimes does try and do "rm ~/" which I think we'd all prefer it didn't. (From OE-Core rev: 9dc1de59330c366783ea043c68c1b59b1b49e707) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: Add tests for gstreamer CLIYeoh Ee Peng2018-11-141-0/+14
| | | | | | | | | | | | | | | | | | | We should not be finding GStreamer issues right around release time but that's what we seem to do. Port gstreamer tests from Intel Ref Kit. https://github.com/intel/intel-iot-refkit/blob/master/meta-iotqa/ lib/oeqa/runtime/multimedia/gstreamer/gstreamer_cli.py Add tests for gstreamer commandline tools (gst-inspect-1.0 & gst-launch-1.0). [YOCTO #11418] (From OE-Core rev: 3e77f845b517e3f4b3e7d09d088a9a80bf171653) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: Inject results+logs into stored json results fileRichard Purdie2018-11-071-2/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the ptest results from ptest-runner, run in an image to be transferred over to the resulting json results output. Each test is given a pass/skip/fail so individual results can be monitored and the raw log output from the ptest-runner is also dumped into the results json file as this means after the fact debugging becomes much easier. Currently the log output is not split up per test but that would make a good future enhancement. I attempted to implement this as python subTests however it failed as the output was too confusing, subTests don't support any kind of log output handling, subTest successes aren't logged and it was making things far more complex than they needed to be. We mark ptest-runner as "EXPECTEDFAILURE" since its unlikely every ptest will pass currently and we don't want that to fail the whole image test run. Its assumed there would be later analysis of the json output to determine regressions. We do have to change the test runner code so that 'unexpectedsuccess' is not a failure. Also, the test names are manipuated to remove spaces and brackets with "_" used as a replacement and any duplicate occurrences truncated. (From OE-Core rev: a13e088942e2a3c3521e98954a394e61a15234e8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ksample.py: fix error messageChen Qi2018-10-121-1/+1
| | | | | | | | | | | | | | | The current error message is like: kobject-example.kodoesn't exist Add a space so that it looks like: kobject-example.ko doesn't exist (From OE-Core rev: 30f2cdd0a3ee0874bebd135e35ab591f630e8adc) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/opkg: Ensure the test works on multilibRichard Purdie2018-10-111-1/+4
| | | | | | | | | After allarch was disabled in multilib, this test broke. Fix the test to account for that change. (From OE-Core rev: 6f692d86c2c6742c67aba93fa24b930af02d8e85) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/dnf: Make sure test_dnf_install does not skippedYeoh Ee Peng2018-09-041-1/+5
| | | | | | | | | | | | | | | | | | | | During debugging dnf issue, we found that the test_dnf_install PASSED the testing even though the environment does not allow dnf install to run successfully. Further debugging had identified that current test_dnf_install will execute dnf install even when the package to be installed already exist, thus dnf install will just skipped and this test will PASSED even though it was not. To solve this, added additional logic to check if the package to be installed already exist, if yes then remove the package before actually run dnf install. This will make sure dnf install was tested as expected. (From OE-Core rev: 4f662b253f7313c4e02bfafb527cdac076b6309a) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ldd: Clean up testRichard Purdie2018-08-231-8/+7
| | | | | | | | | | | * Merge the two tests together as having them separate is pointless * Test that ldd runs correctly * Add in a dependency on the "ldd" package being installed instead of the sdk tools feature (From OE-Core rev: 80db456387fb63ee74d53a9719ab3997432f4c80) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* kernel-devsrc: restructure for out of tree (and on target) module buildsBruce Ashfield2018-08-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The existing kernel-devsrc package starts with a full copy of the kernel source and then starts to strip out elements that are not required. This results in extra time (I/O) and extra space being taken up in the final package. The main purpose of the kernel-devsrc package has been to build modules against the running kernel, not to include a full copy of the source code for re-building the kernel. The end result was a 600M kernel-devsrc package. This restructuring of the package uses an approach similar to other distros, where the kernel-devsrc package is for building against the running kernel and uses a curated set of copied infrastructure, versus a mass copy of the entire kernel. The differences in this approach versus other is largely due to the architecture support and the split build/source directory of the kernel. The result is a kernel-devsrc package of about 10M, which is capable of running "make scripts" and compiling kernel modules against the running kernel. Along with the changes to the copying of the infrascture, we also have the following changes: - a better/more explicit listing of dependencies for on-target builds of "make scripts" or "make modules_prepare" - The kernel source is installed into /lib/modules/<version>/build and a symlink created from /usr/src/kernel to the new location. This aligns with the standard location for module support code - There is also a symlink from /lib/modules/<version>/source -> build to reserve a spot for a new package that is simply the kernel source. That package is not part of this update. (From OE-Core rev: 007ef1f468110b2698a27ea9a6d43fed5a0a9fc2) Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime selftest: limit kernel hw bp archesHongzhi.Song2018-08-161-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | 1. So far, only qemux86[-64] support hw breakpoint, no matter whether or not with kvm. qemuppc: The oe-core configuration uses a PPC G4 system as the default cpu but qemu doesn't simulate the hw breakpoint register for G4. qemuarm: The arch more than v7 supports hw breakpoint, however arm use v5 as default. qemuarm64: We temporarily drop qemuarm64 for the moment. Normally it will print debug info once, but endlessly when we trigger the break point. Now it is hard to located the issue, but we will confirm it later. qemumips*: Kernel dosen't support hw bp for mips. 2. Syslog maybe not started, so we use dmesg to confirm. 3. Running 'ls' to trigger the hardware breakpoint test. (From OE-Core rev: 176e50fb177dc40e439700d5f3f838dd7eaaa427) Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runtime selftest: test_trace_events_sample of ksample.pyHongzhi.Song2018-08-161-2/+12
| | | | | | | | | | The debug info can not be written to trace buffer immediately by thread. So we should sleep some seconds. (From OE-Core rev: 5ccf8150b4c65f2feac16cc2881188f14d9e5390) Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: output correct log locationChen Qi2018-08-141-2/+2
| | | | | | | | | | | | | | | | | | The log entry in results is altered to remove 'target_logs'. This causes wrong log location in output. e.g. AssertionError: 1 != 0 : Log: /path/to/image/1.0-r0/postinstall.log But when user wants to check the log, the user will find the log is not present. The actual log file is /path/to/image/1.0-r0/target_logs/postinstall.log. So fix to use the correct log location. (From OE-Core rev: df3a2e21648d6b649ebda7e6032afbd63c939f2b) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-23 10:27:34 +0000