summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
* oeqa ping.py: fail test if target IP address has not been setMikko Rapeli2023-04-281-0/+1
| | | | | | | | | | | | | It is possible to call exported tests with --target-ip set to ":22" where IP address is not set at all. Detect this case and fail the test instead of calling ping without an IP address. (From OE-Core rev: b14b8f38da9195852e9a2fd6b2e446ed8a1fa0b8) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 17c995c53775b8cee279ca4ced916092067e1195) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa ping.py: avoid busylooping failing ping commandMikko Rapeli2023-04-281-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use a sleep on error path before trying again. For example when oeqa runtime tests are executed without setting target IP address correctly, the tests are drowning logs with messages: 2023-04-04 07:19:24,985 - runtime - INFO - test_ping (ping.PingTest.test_ping) ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ping: usage error: Destination address required ... 2023-04-04 07:19:55,002 - runtime - INFO - ... FAIL 2023-04-04 07:19:55,002 - runtime - INFO - Traceback (most recent call last): File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/runtime/cases/ping.py\", line 23, in test_ping output += proc.communicate()[0].decode('utf-8') ^^^^^^^^^^^^^^^^^^ File \"/usr/lib/python3.11/subprocess.py\", line 1194, in communicate stdout = self.stdout.read() ^^^^^^^^^^^^^^^^^^ File \"/lava-62618/3/tests/3_oeqa-runtime-tests/image/lib/oeqa/core/decorator/oetimeout.py\", line 18, in _timeoutHandler raise OEQATimeoutError(\"Timed out after %s \" oeqa.core.exception.OEQATimeoutError: Timed out after 30 seconds of execution (From OE-Core rev: 5e5975b199e161936471824aabc43da5097219fa) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 87ec75710b5cd7b3f35d886003844d62d3182b54) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/runtime: clean up deprecated backslash expansionRoss Burton2023-04-143-5/+8
| | | | | | | | | (From OE-Core rev: bed045277b835d38048021e7ad158f27dd3ae9bb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 77085a05240c3f8226b9f2199c977f2555807789) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* selftest: runqemu: use better error message when asserts failMartin Jansa2023-04-141-3/+6
| | | | | | | | | | | | | | * It was showing whole log and that the runqemu command failed, but not where the log file is, nor why it thinks the runqemu failed [YOCTO #12937] (From OE-Core rev: c79334f7bd52db6edf05f05616c5a3398d32a55c) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ad4b72e6524114a3bdee10cab71f194ea143cd24) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* selftest: runqemu: better check for ROOTFS: in the logMartin Jansa2023-04-141-1/+2
| | | | | | | | | | | | | | | | | | | | * it was searching for line like this: ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4] but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs" or with my pending changes the line looks like this: ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4] and test was failing. * Check for whole line starting with ROOTFS: and ending just with .ext4 [YOCTO #12937] (From OE-Core rev: 1079fade975c54b62b0187b1acd9c9475a5bbe7e) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7a0f622f23aff2c4eeca0606e7682931eb53287a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/utils/qemurunner: replace hard-coded user 'root' in debug outputEnrico Jörns2023-04-141-2/+2
| | | | | | | | | | | | | | Since login user is configurable with boot_patterns['send_login_user'], unconditionally using 'root' in the debug message can be confusing. Also fix the debug message to say 'Logged in' instead of 'Logged'. (From OE-Core rev: 386f55d2ba5783038c810dfde98b421191fa00de) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 900e3d42b918b5a33d8d952b3a8078fbe72ba98f) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/utils/commands: remove unused importsEnrico Jörns2023-04-141-3/+0
| | | | | | | | | (From OE-Core rev: 9faa5ad7ea61a2edd121b526c688f964e0b656ac) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0c00b5cdd57c3d9c47d4780d6627c74221911b72) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/utils/commands: fix usage of undefined EPIPEEnrico Jörns2023-04-141-1/+2
| | | | | | | | | (From OE-Core rev: d0eeb32cb0d990698287955dd7cac643bcebdafc) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1cb17e37e275794e26debed2d171c2394d8bb945) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/targetcontrol: remove unused importsEnrico Jörns2023-04-141-4/+0
| | | | | | | | | (From OE-Core rev: 6659360828b5d022b90e71883159a75f4def7382) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 46b42edbf814d15be3a18283018ede2d4f817bc6) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/targetcontrol: do not set dump_host_cmds redundantlyEnrico Jörns2023-04-141-1/+1
| | | | | | | | | | | This is already set above, thus simply use the set value from there. (From OE-Core rev: ddcd9061ec11a53bbb4b7c9ffc1b7880f45a17d5) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2f6571ffb23b1a0b5bd6a1212fc108e536f04539) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/targetcontrol: fix misspelled RuntimeErrorEnrico Jörns2023-04-141-1/+1
| | | | | | | | | (From OE-Core rev: 21c513fccbf743f9000c3b7b13fc4962c3ae5c7d) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 92ffc35052768c753a89b4839c70db87072437a2) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/selftest/cases/runqemu: update importsEnrico Jörns2023-04-141-2/+2
| | | | | | | | | | | | 'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is actually used. (From OE-Core rev: 21a31082b7100f82280c1e0b225e6d82b9f6e4f9) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 50f766233e83528eebeca92877277374c7193530) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* lib/oe/gpg_sign.py: Avoid race when creating .sig files in detach_signTobias Hagelborn2023-04-141-10/+17
| | | | | | | | | | | | | | Move the signature file into place only after it is successfully signed. This to avoid race and corrupted .sig files in cases multiple onging builds write to a shared sstate-cache dir. (From OE-Core rev: bbb20a6d74aebff830fcb0c69739a10fd64d9cf3) Signed-off-by: Tobias Hagelborn <tobiasha@axis.com> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b4ec08ea9efebac262d43f47d95a356fe2829de9) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/selftest/cases/package.py: adding unittest for package rename conflictsFawzi KHABER2023-04-141-0/+7
| | | | | | | | | | | | | This Unittest tries to rename a package, using an already used name and fails on do_package. Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr> (From OE-Core rev: efcb5c2b6009632c1ca1c7b401944ccf9fe2eb44) Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c3fe173d5196506d89aa464ba56aabcf581a60db) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/selftest/reproducible: Split different packages from missing packages ↵Jose Quaresma2023-04-141-3/+7
| | | | | | | | | | | | | | output When we provide sstate_targets recipes they are reported as missing which is not quite correct because they are proveided to cut build/debugging time. (From OE-Core rev: a623217aa3297eb6c0616723901788dfd706e1d7) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit fd70a564d6946fa460638dd04ce2daecf4566cf3) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/selftest: OESelftestTestContext: convert relative to full path when ↵Jose Quaresma2023-04-141-0/+10
| | | | | | | | | | | | | | | | | | | | | | newbuilddir is provided Relative paths in BBLAYERS only works when the new build dir are on the same ascending directory node: . ├── build ├── build-st It works because they share the same ascending relative directory node. So use the full path when the argument newbuilddir is provided to make the oe-selftest work everywere regardless of the location chosen. (From OE-Core rev: 41f169ea23078cc8f4a6b6fc6b59230f05cb758b) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 2e022c1977bc1006c00a87e08a2dca5b69db4801) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqs/selftest: OESelftestTestContext: replace the os.environ after ↵Jose Quaresma2023-04-141-5/+5
| | | | | | | | | | | | | subprocess.check_output No function change but is needed to simplify the next patch. (From OE-Core rev: 273cc6d6a11c4f11f80830f23489a287f38c1a17) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 193d8f0d6383f54c61f90d58eeb8f41580c2360a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting SSTATE_MIRRORMartin Jansa2023-04-041-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * with my build/conf/local.conf: SSTATE_DIR = "/OE/build/poky/build/sstate-cache" these devtool tests will first set own SSTATE_DIR and the original one set as SSTATE_MIRROR: 2023-03-11 11:51:46,837 - oe-selftest - INFO - test_devtool_update_recipe_append (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append) 2023-03-11 11:51:46,846 - oe-selftest - DEBUG - Appending to: /OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/conf/selftest.inc SSTATE_DIR = "/OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/sstate_devtool" SSTATE_MIRRORS += "file://.* file:////OE/build/poky/build/sstate-cache/PATH" * but that unfortunately leads to a warning from sanity.bbclass about SSTATE_MIRRORS without matching BB_HASHSERVE, because BB_HASHSERVE is set to "auto" by default these tests failing with: 2023-03-11 11:55:39,610 - oe-selftest - INFO - ====================================================================== 2023-03-11 11:55:39,610 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_append_git (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append_git) 2023-03-11 11:55:39,610 - oe-selftest - INFO - ---------------------------------------------------------------------- 2023-03-11 11:55:39,611 - oe-selftest - INFO - Traceback (most recent call last): File "/OE/build/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1118, in test_devtool_update_recipe_append_git self.assertNotIn('WARNING:', result.output) AssertionError: 'WARNING:' unexpectedly found in 'NOTE: Starting bitbake server...\nWARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.\nLoading cache...done.\nLoaded 0 entries from dependency cache.\nParsing recipes...done.\nParsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 52 skipped, 0 masked, 0 errors.\n\nSummary: There was 1 WARNING message.\nINFO: Updating SRCREV in recipe mtd-utils-selftest_git.bb\nNOTE: Writing append file /tmp/devtoolqa1m2lh02v/layer/recipes-devtools/mtd/mtd-utils-selftest_git.bbappend' ---------------------------------------------------------------------- * just setting BB_HASHSERVE to empty doesn't work, because then we would need to disable OEEquivHash as well as it fails with: ERROR: OEEquivHash requires BB_HASHSERVE to be set (From OE-Core rev: 14c727e4cafd7e1a945913bcc69747bb2efe3c71) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 96d4392ee9c5c3674e5c4c4512f527a2ca6765e4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Fix false negative version issueGeoffrey GIRY2023-04-042-0/+58
| | | | | | | | | | | | | | | | | NVD DB store version and update in the same value, separated by '_'. The proposed patch check if the version from NVD DB contains a "_", ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison. [YOCTO #14127] Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr> (From OE-Core rev: 423ba02b0c2a15bf771db2271df17e12c5adabb2) Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 7d00f6ec578084a0a0e5caf36241d53036d996c4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa rtc.py: skip if read-only-rootfsMikko Rapeli2023-03-251-3/+5
| | | | | | | | | | | | | | hwclock command fails on read-only-rootfs: AssertionError: 1 != 0 : Failed to reset RTC time, output: hwclock: cannot open /etc/adjtime: Read-only file system (From OE-Core rev: 0bb93dce6abdaf09d73d2f9f71781ca20d111601) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 017bf8c160f6ab67d9f8e8d9e30b15bf84f73807) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/recipetool: Stop test corrupting tinfoil classRichard Purdie2023-03-211-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Running recipetool.RecipetoolTests.test_recipetool_handle_license_vars followed by wic.Wic2.test_biosplusefi_plugin_qemu would show a failure of: File "/media/build/poky/meta/lib/oeqa/utils/commands.py", line 351, in runqemu qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype) File "/media/build/poky/meta/lib/oeqa/targetcontrol.py", line 116, in __init__ use_kvm = oe.types.qemu_use_kvm(d.getVar('QEMU_USE_KVM'), d.getVar('TARGET_ARCH')) File "/media/build/poky/meta/lib/oe/types.py", line 182, in qemu_use_kvm if kvm and boolean(kvm): File "/media/build/poky/meta/lib/oe/types.py", line 128, in boolean raise ValueError("Invalid boolean value '%s'" % value) ValueError: Invalid boolean value '/media/build/poky/meta/files/common-licenses' which made no sense until you realise the recipetool test is corrupting the tinfoil class. Work on a copy instead to avoid this. (From OE-Core rev: 76930f40ffeabea07f495c0011a3e6231ea3eddc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7fe76fe17f67c1bbd108d02836692fed20d24771) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/runtime_test/virgl: Disable for all Rocky LinuxMichael Halstead2023-03-211-1/+2
| | | | | | | | | | | | | RHEL compatible kernels do not support vgem so we disable virgl headless testing. (From OE-Core rev: 21220564ab2c6966959d6e97a37f7fe22d48c65b) Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 78bc1ba2361249ec845d1b9e28d429a37dd83910) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: Improve Meson testTom Hochstein2023-03-141-1/+1
| | | | | | | | | | | | | | | | | The meson wrapper setup command detection is broken in the case of an implicit setup command with an option with a space-separated argument, but the test was not detecting it since the case was not covered. Add the option `--warnlevel 1` to the meson command line to cover this case. (From OE-Core rev: 25d48a686f710655241e9c42acf5c31063b2af4b) Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 54e9ee8a0c6c9fc89cbb743f0e4fc18607d503cf) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/prservice: Improve debug output for failureRichard Purdie2023-03-141-1/+1
| | | | | | | | | | | | | We keep seeing this failure on the autobuilder but the output amounts to "False is not True". Improve the debug message on the chance it may make the issue clearer. (From OE-Core rev: 8547304ca874e00714701f647c2a5229a730dfb1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d03f4cf19c2cc96e9d942252a451521dfec42ebc) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa ssh.py: fix hangs in run()Mikko Rapeli2023-03-061-9/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When qemu machine hangs, the ssh commands done by tests are not timing out. do_testimage() task has last logs like this: DEBUG: time: 1673531086.3155053, endtime: 1673531686.315502 The test process is stuck for hours, or for ever if the executing command or test case did not set a timeout correctly. The default 300 second timeout is not working when target hangs. Note that timeout is really a "inactive timeout" since data returned by the process will reset the timeout. Make the process stdout non-blocking so read() will always return right away using os.set_blocking() available in python 3.5 and later. Then change from python codec reader to plain read() and make the ssh subprocess stdout non-blocking. Even with select() making sure the file had input to be read, the codec reader was trying to find more stuff and blocking for ever when process hangs. While at it, add a small timeout to read data in larger chunks if possible. This avoids reading data one or few characters at a time and makes the debug logs more readable. close() the stdout file in all cases after read loop is complete. Then make sure to wait or kill the ssh subprocess in all cases. Just reading the output stream and receiving EOF there does not mean that the process exited, and wait() needs a timeout if the process is hanging. In the end kill the process and return the return value and captured output utf-8 encoded, just like before these changes. This fixes ssh run() related deadlocks when a qemu target hangs completely. (From OE-Core rev: 3e1a4d572922eadc85ff6ac169722ad7ab118cf4) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 9c63970fce3a3d6029745252a6ec2bf9b9da862d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/resulttooltests: fix minor typoAlexis Lothoré2023-03-061-1/+1
| | | | | | | | | | (From OE-Core rev: fdabdb6bd5d190446d15bdefd1032add81c2ebf3) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 80cfa56d133bd3abbb1f37272607d8e15ce70861) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Improve output hash calculationMateusz Marciniec2023-03-061-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Symbolic links to the files are included during the output hash calculation but symlinks to the directories are missed. So if the new symlink to a directory was the only change made, then the output hash won't change, and the Hash Equivalence server may change unihash. In the next run bitbake may use an older package from sstate-cache. To fix this followlinks=True flag could be set for os.walk but it can lead to infinite recursion if link points to a parent directory of itself. Also, all files from a directory to which symlink points would be included in depsig file. Therefore another solution was applied, I added code that will loop through directories and process those that are symlinks. (From OE-Core rev: f1f24e9dc4d4016702d40ec5567cf65d9c04000d) Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ee729163f31f26b1462a47e1e53f7a0f9de9b464) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bblayers/makesetup: skip git repos that are submodulesAlexander Kanavin2023-03-061-0/+10
| | | | | | | | | | | (From OE-Core rev: 0bbcc17f68943655a95913b0d4c214c0227d24c0) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d8bc9cd4ca8ae268a61024f8ac5083a2bbdc432f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bblayers/setupwriters/oe-setup-layers: create dir if not existsAdrian Freihofer2023-03-061-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Without this patch: $ bitbake-layers create-layers-setup /home/adrian/temp/poky-clone NOTE: Starting bitbake server... Traceback (most recent call last): File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 169, in run pipe = Popen(cmd, **options) File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 73, in __init__ subprocess.Popen.__init__(self, *args, **options) File "/usr/lib64/python3.10/subprocess.py", line 971, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib64/python3.10/subprocess.py", line 1847, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/adrian/temp/poky-clone' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/adrian/projects/poky/bitbake/bin/bitbake-layers", line 95, in <module> ret = main() File "/home/adrian/projects/poky/bitbake/bin/bitbake-layers", line 88, in main return args.func(args) File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 90, in do_make_setup p.do_write(self, args) File "/home/adrian/projects/poky/meta/lib/bblayers/setupwriters/oe-setup-layers.py", line 36, in do_write repos = parent.make_repo_config(args.destdir, args.include_layer_repo) File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 55, in make_repo_config destdir_repo = self._get_repo_path(destdir) File "/home/adrian/projects/poky/meta/lib/bblayers/makesetup.py", line 30, in _get_repo_path repo_path, _ = bb.process.run('git rev-parse --show-toplevel', cwd=layer_path) File "/home/adrian/projects/poky/bitbake/lib/bb/process.py", line 172, in run raise NotFoundError(cmd) bb.process.NotFoundError: Execution of 'git rev-parse --show-toplevel' failed: command not found with this patch: $ bitbake-layers create-layers-setup /home/adrian/temp/poky-clone NOTE: Starting bitbake server... NOTE: Created /home/adrian/temp/poky-clone/setup-layers.json NOTE: Created /home/adrian/temp/poky-clone/setup-layers (From OE-Core rev: bd8920b3a93167ad80f41572b84d120538da22e4) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2da12ccada46443d58dd8fab463156fa763b84cc) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa qemurunner.py: try to avoid reading one character at a timeMikko Rapeli2023-02-241-0/+2
| | | | | | | | | | | | | | Read from serial console with a small delay to bundle data to e.g. full lines. Reading one character at a time is not needed and causes busy looping. (From OE-Core rev: ab1e3000cee9f5f3496a7e67cc59b2e08a681a89) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit 0049f6757f6f956fb4cc77b3df6a672c20b53cf4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa qemurunner.py: add timeout to QMP callsMikko Rapeli2023-02-241-0/+3
| | | | | | | | | | | | | | | | | | When a qemu machine hangs, the QMP calls can hang for ever too, and when this happens any failing test commands from ssh runner may be followed by dump_monitor() calls which then also hang. Hangs followed by hangs. Use runqemutime at setup and run_monitor() specific timeout for later calls. (From OE-Core rev: cd6bb88d98b1dc8d751cf75b9ddcca39c84738c6) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit 3a07bdf77dc6ecbf4c620b051dd032abaaf1e4ff) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa qemurunner: read more data at a time from serialMikko Rapeli2023-02-241-1/+3
| | | | | | | | | | | | | | Use a short sleep to bundle serial console reads so that we are not reading one character at a time which reduces busy looping. (From OE-Core rev: 62df87d3544b7fc3717743b131a62c6092359ec5) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit cafe65d8cf7544edbd387f7f5f6d77c64c6b18fa) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa dump.py: add error counter and stop after 5 failuresMikko Rapeli2023-02-241-2/+21
| | | | | | | | | | | | | | | | | | | | | | If test target qemu machine hangs completely, dump_target() calls over serial console are taking a long time to time out, possibly for every failing ssh command execution and a lot of test cases, and same with dump_monitor(). Instead of trying for ever, count errors and after 5 stop trying to dump_target() and dump_monitor() completely. These help to end testing earlier when a test target is completely deadlocked and all ssh, serial and QMP communication with it are failing. (From OE-Core rev: d570e8f3d6ac05be200f5f4acbbd1a15a95fc122) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit d9ad0a055abba983c6cee1dca4d2f0a8a3c48782) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa ssh.py: add connection keep alive options to ssh clientMikko Rapeli2023-02-241-0/+2
| | | | | | | | | | | | | | | | | | | | Configure ssh client to test that connection with server is up. If the server does not respond within a minute then the connection, target machine or sshd daemon are stuck and it's better to exit the command execution with errors. Some tests can execute a long time without returning stdout/stderror data and it's difficult to adjust timers for those cases if connection to target machine or the target machine itself hangs and output is not expected in minutes or even hours. (From OE-Core rev: 07db4a45918eb82503b0bbc7003258c0a03bd5fe) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit ba68ff04c5786eca7cd8dd44056705867dea8ac4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa ssh.py: move output prints to new lineMikko Rapeli2023-02-241-3/+3
| | | | | | | | | | | | | The output from is garbled otherwise and it's not easy to remove debug output form real command output on target. (From OE-Core rev: 5346b927a9fba57b2bb41406fcac09cd6369f26c) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit 917a70cbc43ac1c70c477b220c4115735457ef04) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa context.py: fix --target-ip comment to include ssh port numberMikko Rapeli2023-02-151-2/+2
| | | | | | | | | | | | | Providing ssh port number is supported too with "--target-ip 192.168.0.10:22". (From OE-Core rev: 62b894d75fb3361ed4d5473ae69188bce390980a) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 637919b9df0abc06da5b2f9b389cf25376bd6b7c) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/locales: Add selftest for locale generation/presenceLouis Rannou2023-02-151-0/+45
| | | | | | | | | | | | | | | | [YOCTO #9070] Add a new selftest to validate locale generation. This selftest builds a complete target with GLIBC_GENERATE_LOCALES, IMAGE_LINGUAS, ENABLE_BINARY_LOCALE_GENERATION set. (From OE-Core rev: cbb6344c583a4aaef6514120b4f38f376859368d) Signed-off-by: Louis Rannou <lrannou@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 53258fd810bea6475af9f908f7b712a13a02b628) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: remove True option to getVar and getVarFlag calls (again)Martin Jansa2023-02-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | * True is default since 2016 and most layers were already updated not to pass this parameter where not necessary, e.g. oe-core was updated couple times, first in: https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f Updated with the same regexp as later oe-core update: https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba with small modification to replace not only d.getVar, but also data.getVar as in e.g.: e.data.getVar('ERR_REPORT_USERNAME', True) and for getVarFlag: sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \ -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \ | cut -d':' -f1 \ | sort -u) (From OE-Core rev: f12fcb8a3f3d11c981c223569f8f40ec705c837a) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 26c74fd10614582e177437608908eb43688ab510) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* httpserver: add error handler that write to the loggerRoss Burton2023-02-151-0/+6
| | | | | | | | | | (From OE-Core rev: d3656cf1f749fea837ac2e263f505fcb18965839) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a4bcca3123685f410fc99e5cc23f2b8f39fd0a63) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdkext/cases/devtool: pass a logger to HTTPServiceRoss Burton2023-02-151-1/+1
| | | | | | | | | | | | | Pass our logger to the HTTPService instance so we can see the requests and any errors. (From OE-Core rev: 66b2875e080e9166d176cbe358906523f619ab5e) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3485df2ff61143aac03d92300b7bac4e5d6b2427) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: do not use Popen.poll() when terminating runqemu with a signalAlexander Kanavin2023-02-151-4/+7
| | | | | | | | | | | | | | | | | | This does not actually guarantee that the child runqemu process has completely exited: poll() may return prematurely while the SIGTERM handler in runqemu is still running. This thwarts the rest of the processing, and may terminate the handler before it completes. Use Popen.communicate() instead: this is what python documentation recommends as well: https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate (From OE-Core rev: b228cdf9d4b00411d772a7b605566d33a3a1b82b) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit cd3e55606c427287f37585c5d7cde936471e52f4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/virgl: use pkg-config from the hostAlexander Kanavin2023-01-261-1/+1
| | | | | | | | | | | | | | | | | | | | | The check needs to report dri location on the host machine, so pkg-config binary needs to be capable of finding the needed dri.pc file on the host, and therefore needs to know where host .pc files are located. This may not be the case when using pkg-config from buildtools, so this forces usage of host pkg-config. runqemu already does the same PATH tweak, so this simply brings the two in sync. (From OE-Core rev: cfd9f549a0d5dd44f4c71bd077b5fd17d3683550) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit f0521f8a3ba7e15482756529ee7b0a95b3d53e7d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/reproducible: Use git log without gpg signatureBenoît Mauduit2023-01-261-1/+2
| | | | | | | | | | | | | | | Previously, if "showSignature" is present in user gitconfig, parsing of the timestamp will fail. Ideally we should replace this command with a git plumbing command. (From OE-Core rev: 1075cdeae6af392acddb8c975ac4facc91a5ccde) Signed-off-by: Benoît Mauduit <bmauduit@beneth.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 3bd6f78f79b3d3e87d8db1e11f58d8021f929843) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/debuginfod: improve testcaseRoss Burton2023-01-261-8/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | Primarily, before running the debuginfod-find tool, check that the debuginfod server has finished sweeping the deploy directory. If we make the request too soon then there's a rare chance that we run the client before it has scanned the right packages, and the log gets swamped with warnings from sqlite due to a race. Also: - unset DEBUGINFOD_URLS so the debuginfod doesn't proxy to an upstream server provided by the host distro - Lower concurrency to reduce system load and handle systems with lower maximum open file counts but lots of cores (as the concurrency means cores*2*2 open files) - Set the refresh times to 0 so we never rescan during the test - Only scan the packages for the format which the image is using - Log the commands that are being invoked (From OE-Core rev: fdce8ecdc27796955156b9c4a91ccaca80459ec5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d65729748253eaa640333198ca8aec05946cb9e8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: fix a typoEnguerrand de Ribaucourt2023-01-261-1/+1
| | | | | | | | | | (From OE-Core rev: adee6183096c15b1743ae3e33ebbbbe6218a668f) Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fab3aa7b1b7003bb03678c5cfc62ad2b597f0932) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/rpm.py: Increase timeout and add debug outputPavel Zhukov2023-01-151-12/+11
| | | | | | | | | | | | | | | | | | | | | [Yocto #14346] Systemd may be slow in killing pam session sometimes [1][2]. It may cause rpm test to fail because there's process (sd_pam) running and own by "test1" user after timeout. Increasing timeout to 2 mins and assert earlier with debug output if there's such process(es). If increasing of timeout doesn't help we may want to force deletion of the user as [2] suggests. [1] https://github.com/systemd/systemd/issues/8598 [2] https://access.redhat.com/solutions/6969188 (From OE-Core rev: ed2c94ae744331a1e977dd4e02005390a84cd5be) Signed-off-by: Pavel Zhukov <pavel@zhukoff.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 972fcc0ed1e0d36c3470071a9c667c5327c1ef78) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/concurrencytest: Add number of failures to summary outputRichard Purdie2023-01-061-1/+3
| | | | | | | | | | | | | | | When running oe-selftest and seeing the end of a running log, it is extremely helpful to know if there have been failures or not to save looking at the rest of the log. Add the number of failures to the summary line so that people monitoring builds have an easier time before the end totals are printed. (From OE-Core rev: 80ac9e2eddd4b8b87c2978b3238ac16db2c55e43) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6b23996911d91f7f99774646c6db9f3490b4cb62) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/externalsrc: add test for srctree_hash_filesPeter Marko2022-12-231-0/+44
| | | | | | | | | | (From OE-Core rev: 3cdcd2ce80daaa4a49471e483bd7863803fa6388) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 7b9728e5b8bdf1193c1304ec3beeca4b5bf8d2da) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/tinfoil: Add test for separate config_data with ↵Richard Purdie2022-12-231-0/+14
| | | | | | | | | | | | | | | | recipe_parse_file() We've seen two different regressions in this API since it is used by layer-index but not be the core code. Add a test for it to try and ensure we don't break it again. (From OE-Core rev: 82e2b21f4a91c03203178a4d927c7c7feb930785) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit b07de5de43ec9c9a2c5d496a64940ccdc5b47cf8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: emit more helpful error message when not finding sstate manifestEnrico Jörns2022-12-071-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since oe-core commit 64b89f3c8fc31842256c482a3039d90d3f12c1cc ("sstatesig.py: make it fatal error when sstate manifest isn't found") errors like: | Manifest [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot not found in imx8mm_dummy cortexa53-mx8mm cortexa53 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')? are fatal now and cannot be ignored but must be debugged. Unfortunately, the currently emitted error message is a bit imprecise with telling the reader what has actually gone wrong. This commit: * adds the word 'sstate' to the error message to clarify the scope we are dealing with ('sstate manifests', since there are other manifests, too) * does not randomly print the last manifest file searched for as THE manifest file that could not be found Instead, we print the name of the task the sstate was searched for * adds the word 'multilib' to variant to make clear which variant we are talking about * adds a separate line noting the searched pkgarchs and adds explicitly mentions this word ('pkgarchs') * prints a list of ALL manifest file locations attempted * removes the '?' at the end of the message since such errors indeed leave the question of what is the cause but the error message itself is more like a statement. The result for the exact same issue as noted above then looks as follows: | The sstate manifest for task 'dbus:populate_sysroot' (multilib variant '') could not be found. | The pkgarchs considered were: imx8mm_dummy, cortexa53-mx8mm, cortexa53, armv8a-crc, armv8a, aarch64, allarch, x86_64_x86_64-nativesdk. | But none of these manifests exists: | [..]/tmp/sstate-control/manifest-imx8mm_dummy-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-mx8mm-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-crc-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-aarch64-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-allarch-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot (From OE-Core rev: 18f1a5554063b0cecd206abed6e57f397f7d1346) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 735ec126ec219c7cb89cb05b0e433201bb7f59eb) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>