summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/runtime/ssh: Fix incorrect timeout fixRichard Purdie2024-11-261-2/+2
| | | | | | | | | | | I'd meant to change the command timeout in the previous change, fix the correct one. (From OE-Core rev: 419c96d17205fc9485869ab0fc1f27de52c31789) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bb991988cb23be2c8947171726ada321f27e6eed) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/ssh: Rework ssh timeoutRichard Purdie2024-11-261-2/+2
| | | | | | | | | | | | | | | | | | After the changes to improve this test, we keep seeing image testing ssh failures, particularly on mips. It looks like part of the problem is that on a loaded system, 5s is too short for mips to reliably establish an ssh connection. I've seen logs where it keeps timing out and fails to work, then the debug code successfully uses ssh later after everything else fails. Change the timings/retries to give slow platforms enough time to respond. (From OE-Core rev: 55780565831a1a5fa292d82b2b04943fb9ddc82e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ba64ccf3ad6e40461219b72d60eb0fe5cb38fddd) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/ssh: increase the number of attemptsJon Mason2024-09-091-1/+1
| | | | | | | | | | | | | Under high load, the ssh test is hitting the amount of retries. Increase it to 20 to avoid this issue. This would increase the maximum failure time from 50 seconds (5 * 10) to 100 seconds. (From OE-Core rev: 4581b5793f310d2f1f0c80bfe1a5f8743416c4fc) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c796438eec5dd6b4671b798f85506bc89ff402ab) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/ssh: check for all errors at the endJon Mason2024-08-261-3/+3
| | | | | | | | | | | | | | | | | With the retry for the -SIGTERM, it is possible to still see that error after the 5th attempt and mark the run a success. Check for any non-zero status in the final check and error out to close the gap. While there, make the error print match the one above and be a little more verbose. Also, I'm seeing it take roughly 6 attempts on my local (very slow) system to pass. So, increasing the number of attempts to 10. (From OE-Core rev: 52a67132d4d7e656a39d87c03b1c6162018e8908) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3c3ebe591eef6e0479d623ec2237cfea16db5c80) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/ssh: In case of failure, show exit code and handle -15 (SIGTERM)Richard Purdie2024-08-261-2/+5
| | | | | | | | | | | | | Ensure we show the failing exit code in case of failures. We're seeing autobuilder failures with -15 (SIGTERM) which is probably from slow boot/init. Retry in these cases for now. (From OE-Core rev: 1bd6b0e29650c34652c1027b6975eb8620a73c55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 127d3bd8d5509ae17e359c1365859fd362ffc74f) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/ssh: add retry logic and sleeps to allow for slower systemsJon Mason2024-08-261-9/+19
| | | | | | | | | | | | | | | | | On exceptionally slow systems, the ssh test can intermittently fail due to a race between when ping works and the networking applications being brought up. To work around this issue, add some retry logic when ssh fails to connect. According to the man page of ssh, "ssh exits with the exit status of the remote command or with 255 if an error occurred." So, only retry if the return code is 255, and limit the number of retries to prevent it looping forever. (From OE-Core rev: f6eacc39dc44c6b3dea9c44836addce5d03f20ef) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f0fe0b490d309cdf1c97754f85a61b5b948b7f28) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/scp: requires openssh-sftp-serverJose Quaresma2024-08-011-1/+1
| | | | | | | | | | | | | | | | | | | | | The SCP protocol was deprecated in favor of the SFTP. For the legacy SCP protocol scp should be run with "-O". Instead of adding "-O" on the scp_options ssh oeqa we can require the openssh-sftp-server to be instaled on the target. This way the test will work more deterministic regardless of the host machine client used. For the old fashion clients still using legacy SCP protocol the openssh-sshd server will be used, for the new ones using the SFTP the openssh-sftp-server will be picked. (From OE-Core rev: 7629a47d6cac36c78184788010f754175853bb03) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 2f43da91ba20d18bc419bca7651bb383a51f20af) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime: fix race-condition in minidebuginfo testEtienne Cordonnier2024-08-011-4/+13
| | | | | | | | | | | | | | | | | | | | Fix this error where 'coredumpctl info' warns that the coredump is still being processed: ``` AssertionError: 1 != 0 : MiniDebugInfo Test failed: No match found. -- Notice: 1 systemd-coredump@.service unit is running, output may be incomplete. ``` (From OE-Core rev: ad1ce64f5c1f22a7b10025d8cba20dc74354ac81) (From OE-Core rev: f7e824477ef75fcea8e0b777278413304def631c) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ed562345d5a5f2edb649028553199f3f7966e19e) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime: make minidebuginfo test work with coreutilsEtienne Cordonnier2024-08-011-1/+2
| | | | | | | | | | | The test was failing with core-image-full-cmdline which uses sleep from coreutils instead of sleep from busybox. (From OE-Core rev: 2b71641b9c1b8f7e67527d2242a6a65b1e849ded) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8497edf0c56da34ea7b7509b8fcd46fcba2fd0e3) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime: fix regression in minidebuginfo testEtienne Cordonnier2024-07-261-1/+2
| | | | | | | | | | | | The builtin command "sleep" was getting executed instead of the busybox binary. Hence "pidof sleep" was empty and the test was failing. (From OE-Core rev: dfde05c64a7d9f01f09a34e76509ef80501f0db7) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3188f3f0718b8a59bc9e3cae4cdf74e82bdb86c3) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime/login: Fix dbus-wait timeout and loop conditionalRichard Purdie2024-03-071-4/+5
| | | | | | | | | | | | | | | The dbus-wait command returns a timeout after 60s but reports "success", detect this. Unfortunately it does effectively break the test as the signal is nearly never being correctly detected since it was already sent. For that reason comment out the code instead too. Also fix the loop conditional as the logic was incorrect and it was looping indefinitely when an image match didn't occur. (From OE-Core rev: 89c930e9e4b38b116edcba59e88621a39f8bda67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Add screenshot sample logic/timeout/dbus-waitEilís 'pidge' Ní Fhlannagáin2024-03-071-38/+41
| | | | | | | | | | | | This patch uses dbus-wait to wait for matchbox to be up. Once that happens, it sets a timeout of 60 seconds and takes a screenshot and compares it, every 2 seconds. If diff=0 it passes. If the timeout ends, it fails. (From OE-Core rev: 287b4f0a8244f7214f6a1aaa84ef16cc528f8326) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Exclude qemuriscv64Eilís 'pidge' Ní Fhlannagáin2024-03-071-1/+1
| | | | | | | | | | | Excluding riscv64 due to mouse rather than a touchscreen which adds a moving cursor, so the diff ends up > 0. Need to fix the image to use the touchscreen rather than mouse input. (From OE-Core rev: 7f7032c7613abd62ce510c98211c75fc7c5e7090) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Mask out the mouse panel icon for nowRichard Purdie2024-03-071-2/+2
| | | | | | | | | | The mouse panel icon can move when the time changes between 3 and 4 digits. Ideally we'd fix the alignment of the clock on the panel but to get the tests working, increase the size of the mask for now. (From OE-Core rev: 8ec02142ab98741749281bdb29d5973c96b839d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Various code improvements and fixesRichard Purdie2024-03-071-43/+48
| | | | | | | | | | | | | | | | * Allow tools to be found from the host PATH so that imagemagick from a buildtools tarball/sdk can work * Reformat the code to have imports at the start of the file and have more standard formatting and whitespace * Always save copies of the images, the space imapct is negligle compared to the debug win * Write the images to ${T} * Use bb.utils.mkdirhier() instead of more complex code * Restrict the tests to images containing matchbox-desktop (From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/login: Proof of concept for screenshot testcasesEilís 'pidge' Ní Fhlannagáin2024-03-071-0/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This takes the work rburton did on image screenshot testing and expands it. Right now this works with most of the qemu based machines except for - qemuppc64 - qemuarmv5 - qemuriscv32 - qemuloongarch64 See "Known Issues" further down. This test takes a screendump of a qemu image, blanks out the clock and compares it to an image we have on record. If the diff is exact, the test passes. If not, it stores the image in build/failed-images and fails out. In order to enable this test, you will need meta-openembedded/meta-oe in your bblayers.conf for imagemagick and the following in local.conf: IMAGE_CLASSES += "testimage" TEST_SUITES = "login" TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot " Known Issues ------------ The main issue is that I've yet to find a gating factor that would allow me to tell when the qemu instance is fully up and rendered. I've tried a few tactics here, (dbus-wait, qmp) but for now a disgusting time.sleep(30) is there. You can replicate this by running qemumips. The screen load takes forever, but you even see it on qemux86 where the Home and Workspace Switch icons will sometimes take a while to fully load. Eventually I'm going to have to take multiple screenshots and compare them, but then you get into the issue where the question is, is the diff greater than 0 because it hasn't fully loaded or something is actually incorrect. There are the issues I know about: - runqemu qemuppc64 comes up blank. - qemuarmv5 comes up with multiple heads but sending "head" to screendump. seems to create a png with a bad header. - qemuriscv32 and qemuloongarch64 don't work with testimage apparently? - qemumips64 is missing mouse icon. - qemumips takes forever to render and is missing mouse icon. - qemuarm and qemuppc return incorrect width - All images have home and screen flipper icons not always rendered fully at first. The sleep seems to help this out some, depending on machine load. (From OE-Core rev: dc7cefbaccde50df6c4396e66d50659a45e00631) Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Co-authored-by: Ross Burton <ross.burton@arm.com> Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: fix typo in information messageMichael Opdenacker2024-02-092-2/+2
| | | | | | | (From OE-Core rev: 901ff496b97119add792912fbb2f1efb1152e7c2) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildcpio.py: Switch to using cpio-2.15Khem Raj2024-02-081-6/+2
| | | | | | | | | | | | | | | | | This helps in getting it building with newer architectures like riscv32 since it has upgraded gnulib over 2.14 which has the needed fixes. Drop the -fno-common workaround as it is already applied to cpio drop --disable-maintainer-mode Fixes configure: WARNING: unrecognized options: --disable-maintainer-mode (From OE-Core rev: 18d303497089d3a7a893ee0eec5b0f0c78cca06d) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: Enable extra test groupsRichard Purdie2024-02-061-1/+1
| | | | | | | | | Petr Vorel (upstream ltp maintainer) recommended some extra tests we should run. Enable these for extra test coverage. (From OE-Core rev: ebae56999dc2ed7abb65062a9abee53827c3bad3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ltp: Update to 20240129Petr Vorel2024-02-051-2/+2
| | | | | | | | | | | | | | | | | | | | Removed patch accepted upstream as 8c53cc947 ("testcases/kernel/syscalls/fcntl: define _LARGEFILE64_SOURCE") Rebase runtest/mm patch. [RP: Drop connectors group since dropped upstream: https://github.com/linux-test-project/ltp/commit/9b642d89c0bcf5885b051c2d5768fa94b61d86cb Drop fsx too: https://github.com/linux-test-project/ltp/commit/fb2b6a0b3c840aa80229acf4360b7bdc3ced5edb ] (From OE-Core rev: b2867cac5ac58862469f0c6c056607abeca0b0d0) Signed-off-by: Petr Vorel <petr.vorel@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: fail tests if test rpm file cannot be foundAlexander Kanavin2024-01-271-7/+11
| | | | | | | | | | | | | | | | | | Discovery of the test file was happening in a class initializer. That block of code cannot fail (it's not a test), and so it falls through to completion even if the needed file could not be found. Then the tests themselves fail later due to class variables not being set, but all information as to why is already lost at that point. This converts the discovery to a helper function called from the tests, so that the function can fail the tests precisely when the problems occur. (From OE-Core rev: 5d7a6ede105ea1efc9c324c7029f9d08dadf7255) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa systemd.py: settle() using "running" or "degraded" stateMikko Rapeli2024-01-071-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | systemd boot has completed when system is in "running" or "degraded" (some services failed) state. Check for that in the systemd settle() function instead of listing all services and checking their activation state since some services are in activation state even when whole system is already in "running" state. Examples of services which can be in activation state are rootfs auto mounting related generated services. Without this patch systemd test_systemd_list (systemd.SystemdBasicTests) times out on an image with dm-verity /usr partition and systemd generated rootfs: NOTE: ... FAIL Traceback (most recent call last): File "/home/builder/src/base/build/../poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f return func(*args, **kwargs) File "/home/builder/src/base/poky/meta/lib/oeqa/runtime/cases/systemd.py", line 97, in test_systemd_failed self.assertTrue(settled, msg=msg) AssertionError: False is not true : Timed out waiting for systemd to settle: UNIT LOAD ACTIVE SUB DESCRIPTION dev-disk-by\x2did-dm\x2dname\x2droot.device loaded activating tentativ e /dev/disk/by-id/dm-name-root dev-disk-by\x2did-dm\x2dname\x2dusr.device loaded activating tentativ e /dev/disk/by-id/dm-name-usr dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dLUKS2\x2df2b944f394174eb5918cb6af2c6b4cb2\x2droot.device loaded activating tentativ e /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-f2b944f394174eb5918cb6af2c6b4cb2-root dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dVERITY\x2d3dd703c88f1946658697a6d57617473b\x2dusr.device loaded activating tentativ e /dev/disk/by-id/dm-uuid-CRYPT-VERITY-3dd703c88f1946658697a6d57617473b-usr dev-disk-by\x2duuid-bfbf856e\x2d3c65\x2d4eb2\x2d9ffb\x2d8e0b11641d85.device loaded activating tentativ e /dev/disk/by-uuid/bfbf856e-3c65-4eb2-9ffb-8e0b11641d85 dev-dm\x2d0.device loaded activating tentativ e /dev/dm-0 dev-dm\x2d1.device loaded activating tentativ e /dev/dm-1 ... Fix is to check for the systemd global "running" or "degraded" state. Note that it would be possible to use a blocking call "systemctl is-system-running --wait" to exit after system enters "running" or "degraded" state but using the existing loop for a 2 minute timeout. (From OE-Core rev: 3b013ae441d117adeda0d9950e02e9f7d0deba2f) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/parselogs-ignores-qemuarmv5: add comments and organiseRoss Burton2024-01-041-2/+6
| | | | | | | | | Add a comment explaining why we need the jitterentropy ignore. (From OE-Core rev: ed4b202b11fd18e18502fe4e90f421dce4db8b44) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: add qemux86 ACPI ignore for kernel v6.6+Bruce Ashfield2024-01-031-0/+3
| | | | | | | | | | | | | | upstream commit 003e0694fcd3d5 [ACPI: processor: Refine messages in acpi_early_processor_control_setup()] changes logging during boot such that some of the informational messages are caught as QA issues. Adding the string to the ingore files so that 6.6+ will pass QA. (From OE-Core rev: 463d40f2d4bf30becce431bfc1cbdf3b673e8e60) Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.py: OEHasPackage: Add MLPREFIX to packagenameSaul Wold2023-12-281-2/+3
| | | | | | | | | | | | | | | | FIXES [YOCTO #12342] When testing a Multilib image, the package manifest list contains the fully qualified package name which includes the Multilib Prefix. This patch adds the MLPREFIX to the package names that are passed into the @OEHasPackage() decorator to ensure the set isdisjoint() matches correctly. (From OE-Core rev: ab87e4f92305b2a664cc473869e1615cf56e0936) Signed-off-by: Saul Wold <sgw@bigsur.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: add runtime 'maturin develop' test caseTim Orling2023-12-231-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Similar to the sdk test case, build the "guessing-game" example from https://maturin.rs/tutorial This test case: * creates a python3 venv * echoes "nameserver 8.8.8.8" to /etc/resolv.conf as we need to have functional DNS to fetch the crates on target * fetches crates, builds guessing-game crate and wheel Put the following in your local.conf: EXTRA_IMAGE_FEATURES += "tools-sdk" SDK_INCLUDE_TOOLCHAIN = '1' SDK_TOOLCHAIN_LANGS += 'rust' IMAGE_INSTALL:append = " python3-maturin" IMAGE_CLASSES += "testimage" TEST_QEMUPARAMS ?= "-m 8192 -smp 4" IMAGE_ROOTFS_EXTRA_SPACE = "10000000" NOHDD="1" NOISO="1" TEST_SUITES = "ping ssh python maturin" Test with: bitbake core-image-full-cmdline bitbake -c testimage core-image-full-cmdline (From OE-Core rev: ca7e78c8be6aaa2780702eab54715a74fc0dac5e) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selfest: add maturn runtime (testimage) testTim Orling2023-12-171-0/+23
| | | | | | | | | Basic smoke test for maturin to test the 'maturin list-python' case. (From OE-Core rev: 47c948c3cf6e582abd12021ceeff2c20a3e81fb5) Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/systemd: Ensure test runs only on systemd imagesRichard Purdie2023-12-081-0/+1
| | | | | | | | | When enabled in DISTRO_FEATURES the test may run on a system without systemd. Fix this. (From OE-Core rev: c2b473390dec0f5132d5b4bff6d3c35214eb898b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gdb/systemd: enable minidebuginfo support conditionallyEtienne Cordonnier2023-12-061-0/+21
| | | | | | | | | | | | | | Enabling minidebuginfo is not useful if gdb and systemd-coredump are unable to parse it. In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen). (From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9) Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: migrate ignoresRoss Burton2023-12-0611-180/+132
| | | | | | | | | | | | | | | | Move the ignores from a huge dict in the parselogs.py module to .txt files. This is just the common, tune, and qemu machine ignores; the machine ignores that are not in oe-core will be added to the relevant layers. The list of ignores has not been reviewed in any meaningful way, this should be done soon as I suspect a number of these are redundant. (From OE-Core rev: bba243e1d18b954578afcdb3c727d8f687187ee8) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: load ignores from diskRoss Burton2023-12-061-5/+35
| | | | | | | | | | | | | | | | | | | | | | | | | Instead of hardcoding the list of ignored errors/warnings in the test itself, read them plain text files on disk. This uses importlib to try to open a file called oeqa.runtime.cases.parselogs-ignores-[candidate].txt, where the candidate will be: - "common" - The TARGET_ARCH - Each of the MACHINEOVERRDES This allows the common and tune-specific ignores to be retained in oe-core, and machine-specific ignores added to the layer where the machine is defined. [ YOCTO #14604 ] (From OE-Core rev: 7a04063f7cff243fe2bee09664ad7979612110cb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/ssh: Handle SSHCall timeout error codeluca fancellu2023-11-101-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | The current code in ssh.py is terminating the ssh process that does not finish its computation in a given timeout (when timeout is passed), the SSHCall function is returning the process error code. The Openssl ssh before version 8.6_p1 is returning 0 when it is terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda onwards (version >= 8.6_p1) ssh is returning 255 instead. So for version of ssh older than 8.6_p1 when the SSHCall time out, the return code will be 0, meaning success, which is wrong. Fix this issue checking if the process has timeout (hence it's been terminated) and checking if the returned code is 0, in that case set it to 255 to advertise that an error occurred. Add a test case excercising the timeout in the SSHTest, test_ssh test function. (From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00) Signed-off-by: Luca Fancellu <luca.fancellu@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases/parselogs: remove "edgerouter" caseMichael Opdenacker2023-10-261-6/+0
| | | | | | | | | | The "edgerouter" machine has been removed since https://git.yoctoproject.org/poky/commit/?id=0c64d0e4317e3749f7f7ed9ecd5d08bbb0cedc9e (From OE-Core rev: b82514c821add181e141d12b0c1723760b445fea) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/_qemutiny: rewrite test to be functionalRoss Burton2023-10-111-3/+8
| | | | | | | | | | | | The _qemutiny is a small test case that was explicitly designed to do a minimal level of testing for poky-tiny images. These typically don't have SSH servers so we need to assume that qemu is being used and access the serial console directly. (From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: parse the logs with Python, not grepRoss Burton2023-09-261-50/+41
| | | | | | | | | | | | | | | | | Instead of constructing huge grep statements, we can simply open the logs in Python and do the relevant string operations directly. The trick is to remember to casefold() all of the strings, so that the "in" operator can be used. Just one of the ignores needs to be adjusted because it uses a regular expression and the new logic doesn't support that. This is handled by simply reducing the size of the ignore match. (From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: select the correct machine-specific ignores earlyRoss Burton2023-09-261-7/+6
| | | | | | | | | | This has no impact to the execution, but makes the following changes neater. (From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: move some variables out of global scopeRoss Burton2023-09-261-10/+9
| | | | | | | | | | errors and log_locations can be trivially set in the class directly, instead of being defined in the module and then copied into the class. (From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: don't pass around membersRoss Burton2023-09-261-10/+14
| | | | | | | | | | There's no point in passing around member fields, just access them directly. (From OE-Core rev: a24d6eda061363cdcfa95980cd2698a674737d23) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: improve find callRoss Burton2023-09-261-5/+5
| | | | | | | | | | | | | | | getLogList() uses remote find invocations to find the logs. Instead of relying on shell expansion of wildcards and redundant use of -maxdepth (pointless as the shell expansion means the find is passed the files to return), invoke find idiomatically by telling it what directory to search for and escape the glob so find processes it. Also remove many pointless str() calls. (From OE-Core rev: 03bb14cebf5879472a8da78d892ecd5c5df5c3cf) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Use 2.14 release of cpio instead of 2.13Khem Raj2023-09-261-1/+1
| | | | | | | | | 2.13 may not be buildable with latest compilers without patching (From OE-Core rev: 406a33f896accc35a9cb6ab156f1e0f42dda67d8) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: inline single-caller functionsRoss Burton2023-09-221-8/+2
| | | | | | | | | | There's no need to have one-liner functions to get the MACHINE or WORKDIR when they're only called once. (From OE-Core rev: 9478a665641f55cdc14f12a4409121ef95883d74) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: remove obsolete LSB testing supportRoss Burton2023-09-221-8/+0
| | | | | | | | | | | The LSB compliance tests were removed in 2019[1], so this is obsolete. [1] oe-core fb064356 (From OE-Core rev: a465b61bc3a16152f90032d5d5a041f87bf30108) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: don't bother to show target hardware informationRoss Burton2023-09-221-29/+0
| | | | | | | | | | This information is pretty useless as we know what the target is from the build configuration (be it a qemu machine or real hardware). (From OE-Core rev: 7821c42172191e3c16ab8662a9e4f7e921830db0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: remove unused importsRoss Burton2023-09-221-2/+0
| | | | | | | (From OE-Core rev: 5c2b6ce30b3cbec6236b99ce4019dbd443ccccae) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* go-helloworld: update to latest revisionAlexander Kanavin2023-09-181-1/+1
| | | | | | | | | Fix up test case to match what binary prints. (From OE-Core rev: 4683804b9669c71d31ea6a8a300e6e87e817ee12) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ptest: report tests that were killed on timeoutAlexander Kanavin2023-09-181-1/+1
| | | | | | | | | | I'm not sure if this was reported correctly before, but it currently is not. Test that is stuck is an error in itself. (From OE-Core rev: 002e27c9932a83e46be0b03a5232594cfba7212c) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "oeqa/runtime/parselogs: Exclude preempt-rt error for now"Luca Ceresoli2023-08-221-1/+0
| | | | | | | | | | | There's now a real fix in commits "linux-yocto/6.*: fix IRQ-80 warnings". This reverts commit 1451df346a0e2433714774421ce8f339a37a844a. (From OE-Core rev: f5c1b0f2b17814315bc2a09acc3aaa20f2522cff) Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rust: correct rust testFrederic Martinsons2023-08-211-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Since setUp of RustCompileTest use cls.tc.copyTo, those tests needs the scp command: NOTE: ====================================================================== NOTE: FAIL: test_cargo_compile (rust.RustCompileTest) NOTE: ---------------------------------------------------------------------- NOTE: Traceback (most recent call last): File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/case.py", line 53, in _oeSetUp self.setUpMethod() File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/runtime/cases/rust.py", line 17, in setUp cls.tc.target.copyTo(src, dst) File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 132, in copyTo return self._run(scpCmd, ignore_status=False) File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 81, in _run raise AssertionError("Command '%s' returned non-zero exit " AssertionError: Command '['scp', '-o', 'ServerAliveCountMax=2', '-o', 'ServerAliveInterval=30', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '-r', '-P', '2222', '/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/files/test.rs', 'root@127.0.0.1:/tmp/']' returned non-zero exit status 1: sh: scp: not found lost connection (From OE-Core rev: 1bf24df9be44c73e5d8e90feb446ecfcd542228c) Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: provide examples for C library generation in rustFrederic Martinsons2023-08-211-0/+12
| | | | | | | | | | | | | | | | | | | | rust-c-lib-example is a little rust code which provide a single function to print a formatted date (via the chrono crate) from an input timestamp in millisecond. It has the necessary FFI annotation and inherit cargo_c class for the C ABI compatible library generation. rust-c-lib-example is meson project for the C code which will call the print_date function from rust-c-lib-example if no argument is provided, if any argument is provided it will print "Hello world in rust from C!" add a runtime test case to check if all went well. (From OE-Core rev: bb177c7764b1bc47157d57d7a34930e59a7acef3) Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: Exclude preempt-rt error for nowRichard Purdie2023-08-151-0/+1
| | | | | | | | | | | With the new 6.4 kernel we see this preempt-rt error. It is blocking changing to the new kernel and has sat on mailing lists unresolved for a long time. Ignore it in testing for now and allow upgrading until we can better understand the issues. (From OE-Core rev: 1451df346a0e2433714774421ce8f339a37a844a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-04-22 01:13:35 +0000