summaryrefslogtreecommitdiffstats
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* devtool: deploy-target: Remove stripped binaries in pseudo contextFlorian Amstutz2022-02-231-1/+1
| | | | | | | | | | | | | | deploy-target may fail the second time with "pseudo abort" because devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot context. (From OE-Core rev: b293bcf4c312156cb57814dd71f13f8ab2a8ae80) Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2338a33b690b0bbe279cde3f73764911b239cb50) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Fix circular reference in SRC_URISaul Wold2022-02-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating a new recipe.bb file for a binary, don't use BP which includes the version information, instead use BPN which is just the name base Package Name. Since PB is not specified, it takes the default: PV = "1.0+git${SRCPV}" But SRCPV is defined in terms of the SRC_URI, which leads to infinite recursion (traceback below). Here are the pertinent variables which cause the recursion: SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}" BP = "${BPN}-${PV}" PV = "1.0+git${SRCPV}" SRCPV = "${@bb.fetch2.get_srcrev(d)}" def get_srcrev(d, method_name='sortable_revision'): # ... trimmed scms = [] fetcher = Fetch(d.getVar('SRC_URI').split(), d) # ... trimmed [YOCTO #14040] (From OE-Core rev: 64434ffd6d406e4c3d1e9ca2fb72fa97990346d8) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use shutil.whichMingli Yu2022-01-223-7/+7
| | | | | | | | | | | | | | | | | | | | Use shutil.which to find the executable instead to silence the below warning: $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic [snip] DEBUG: Executing shell function do_image_wic /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import spawn INFO: Creating image(s)... [snip] [RP: Added conversion for missed function reference] (From OE-Core rev: 488815681466d5e4c7640df5281fa6e1f9b4c75e) Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: misc: Do not find for executables in ASSUME_PROVIDEDRicardo Ribalda Delgado2022-01-221-2/+12
| | | | | | | | | | | | | | Executables like tar won't be available on the native sysroot, as they are part of the ASSUME_PROVIDED variable. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: eab485069809c991433b9e1b8e4223a555e4d7f9) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2f574d535f8665b26dab65c14668cf8fc7b751c0) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Update to use exec_module() instead of load_module()Richard Purdie2022-01-112-4/+11
| | | | | | | | | | | | This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. (From OE-Core rev: c0710d4ffb6c11083daf77fe24129d879f270591) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 655cd3f614d736416eab0d708b7c49674bf5c977) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/buildhistory-diff: drop use of distutilsTim Orling2022-01-111-5/+0
| | | | | | | | | | | | | | | | | | | | The use of distutils.version.LooseVersion to check for GitPython > 0.3.1 is not really needed anymore since any supported distribution has at least 1.0.0 (centos-7 via epel7, debian-9, ubuntu-16.04) If we want to reinstate this check, alternatives would be to require python3-packaging on all hosts and use packaging.version.Version or use an imported LooseVersion in bb.version. [YOCTO #14610] (From OE-Core rev: 15b1a82b4cdfadeaf89489929e10aaf9a0edd13d) Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bc90dcae9f53ddc246942f4d9b8ae8943e3b9754) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Set master branch only as fallbackStefan Herbrechtsmeier2021-12-141-8/+7
| | | | | | | | | | | | | | | | The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)' sets the branch= parameter too early to master and thereby breaks the -B/--srcbranch option. ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other (From OE-Core rev: 2bc6deb34aee7542d7086c24b166083c562a6a01) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: check the qemu PID has been set before kill()ing itRoss Burton2021-12-141-1/+2
| | | | | | | | | | | | | | | If runqemu is killed, check that we have a valid PID for the qemu before sending a kill() to it. [ YOCTO #14651 ] (From OE-Core rev: 7f95eb486a2d07cf0c821d5301fd670de34fe2d1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0f3afbd3a6a6bef668612f818517df7543c0a683) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic:direct.py: ignore invalid mountpoints during fstab updateMarkus Volk2021-12-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | wic fstab-update creates invalid entries for partitons that are not supposed to be mounted from userspace eg u-boot partitions. The following lines were added to fstab on a rock-pi-4: /dev/mmcblk1p1 loader1 vfat defaults 0 0 /dev/mmcblk1p2 reserved1 vfat defaults 0 0 /dev/mmcblk1p3 reserved2 vfat defaults 0 0 /dev/mmcblk1p4 loader2 vfat defaults 0 0 /dev/mmcblk1p5 atf vfat defaults 0 0 /dev/mmcblk1p6 /boot vfat defaults 0 0 With this patch only valid entries should be added /dev/mmcblk1p6 /boot vfat defaults 0 0 (From OE-Core rev: 3c8e50b073a1505cc3586ca51dfe1e75338feafc) Signed-off-by: MarkusVolk <f_l_k@t-online.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7aa678ce804c21dc1dc51b9be442671bc33c4041) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/checklayer/common.py: Fixed a minor grammatical errorDhruva Gole2021-12-081-1/+1
| | | | | | | | | | (From OE-Core rev: eee2103640d15011cb40d719769ecb4ce42095fb) Signed-off-by: Dhruva Gole <goledhruva@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8ea17456ae5318ed7a3b4c8f75c8441456d8b979) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/help.py: Update Fedora Kickstart URLsJon Mason2021-12-021-2/+2
| | | | | | | | | | | | | The URLs describing Kickstart are no longer valid and do not redirect to the correct location. Update them with the correct location. (From OE-Core rev: f01e3e8ece228a355cb91cbd043e0fe7fa951ebf) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e5ac75f93c8128b0761af5fee99e8603ddd1657d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-package-browser: Handle no packages being builtRichard Purdie2021-12-021-0/+2
| | | | | | | | | | | | | | Give the user a proper error message if there aren't packages built, rather than a less friendly traceback. [YOCTO #14619] (From OE-Core rev: 4b6d3564c09dbea3794f2915b65cbd2314ccffb6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b14c176b7dd74b7d63ca0f72e6e00fbf209f5a0b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Manual git url branch additionsRichard Purdie2021-11-111-0/+3
| | | | | | | | | | | | | Following the scripted conversion adding branches to git:// SRC_URI entries, add the remaining references, mainly in the selftests and recipetool. (From OE-Core rev: 38fc0807eea14dc12610da4ba73c082d5a4b0744) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5340c0d688036c1be6c938f05d8a8c1e3b49ec38) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/convert-srcuri: Backport SRC_URI conversion script from master branchRichard Purdie2021-11-111-0/+77
| | | | | | | | | | | | | | | | | | This script handles two emerging issues: 1. There is uncertainty about the default branch name in git going forward. To try and cover the different possible outcomes, add branch names to all git:// and gitsm:// SRC_URI entries. 2. Github are dropping support for git:// protocol fetching, so remap github urls as needed. For more details see: https://github.blog/2021-09-01-improving-git-protocol-security-github/ (From OE-Core rev: 904bdbab712e2f3c332c4d85d08ed83c957e249f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scriptutils.py: Add check before deleting pathChandana kalluri2021-10-231-1/+2
| | | | | | | | | | | | | | | | | | | | | | Add a check before deleting path when using recipetool commands to avoid the following type of errors: Traceback (most recent call last): File "<workdir>/sources/core/scripts/lib/scriptutils.py", line 218, in fetch_url shutil.rmtree(path) File "/usr/local/lib/python3.7/shutil.py", line 476, in rmtree onerror(os.lstat, path, sys.exc_info()) File "/usr/local/lib/python3.7/shutil.py", line 474, in rmtree orig_st = os.lstat(path) FileNotFoundError: [Errno 2] No such file or directory: '<workdir>/build/tmp/work/recipetool-usg7o81n/work/recipe-sysroot' ERROR: Command 'script -e -q -c "recipetool --color=always create --devtool -o /tmp/devtool5sq_op37 'file:///<SRCTREE>' -x <workdir>/build/workspace/sources/devtoolsrcxc1b9zjq -N test" /dev/null' failed (From OE-Core rev: a21d3d8f6da9fbb71f484343244cc3a63a346337) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b6aa8b47e023004ffd6958d1cec18c2d9c95d77b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchart: Avoid divide by zeroRichard Purdie2021-10-071-1/+4
| | | | | | | | | | | | | Avoid a rare divide by zero error if there isn't data point spread. [YOCTO #14547] (From OE-Core rev: 6b0e7d478bf23bcae774422160be51419c45c386) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d7e36d01e87ddf89f76f164a0b7d98f597a53fa5) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: keep rootfs_size as integerAlexander Kanavin2021-09-301-1/+1
| | | | | | | | | | | | | | | | | The corrected line accidentally converted it to float, which causes problems later on with python 3.10: | File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext | os.ftruncate(sparse.fileno(), rootfs_size * 1024) | TypeError: 'float' object cannot be interpreted as an integer (From OE-Core rev: 098195ecefafa4082bb5d27ff1c8b1b7b8e222d5) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d1d260dd2d196d10379ed9e238bcb34f39f3a3b7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use the label provided when formating a dos partitionjbouchard2021-09-011-2/+4
| | | | | | | | | | | | | | | | | Previously the bootimg-pcbios wic plugin was not respecting the --label option provided from the wks file. The plugin was setting the label to 'boot'. With this fix, the --label option is use. If no option are specified, then the default is 'boot'. (From OE-Core rev: c9617934a20d30198a8ddcf10fc02de4885fd433) Signed-off-by: jbouchard <jeanbouch418@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: don't forcibly pass -T defaultRoss Burton2021-08-183-3/+3
| | | | | | | | | | | | | | | | | This reverts part of oe-core eecbe62555, which was a previous attempt to solve the Y2038 problem. This is now solved centrally in e2fsprogs, so doesn't need to be dealt with in wic. We don't revert the commit entirely, to retain the warning if a filesystem has small inodes. (From OE-Core rev: 83e53de4c99578a686cd7885610a978d995996b7) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7e8017208bed98b6c90735cb641fc9d7aedf9140) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: Fix typo in error messageMatthias Klein2021-08-181-1/+1
| | | | | | | | | | (From OE-Core rev: 0d14ade2ba75e6e918a5d7e84ac27830a8548a42) Signed-off-by: Matthias Klein <matthias@extraklein.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5cc0051d50974e198313f9513b24fd7ae9a96dd4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-check-layer: ensure that all layer dependencies are tested tooNicolas Dechesne2021-08-101-1/+18
| | | | | | | | | | | | | | | | | | | | | | In order to be compliant with the YP compatible status, a layer also needs to ensure that all its dependencies are compatible too. Currently yocto-check-layer only checks the requested layer, without testing any dependencies. With this change, all dependencies are also checked by default, so the summary printed at the end will give a clear picture whether all dependencies pass the script or not. Using --no-auto-dependency can be used to skip that. (From OE-Core rev: bec38becf8a489d69aca0917a2ce1dfdc96d8ab3) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 45d59b774b95c91193a8376b83c05291d555e5c8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: rename _find_layer_dependsNicolas Dechesne2021-08-101-2/+2
| | | | | | | | | | | | | | | What this function does is really to find a layer, not a 'depends'. We are using this function to find a dependent layer, but the name is confusing. (From OE-Core rev: d6618f92fa3f589c1d155081e85905ffe5e39a6c) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e9b7690ab30d0e7c07471034f6cb89ccc3168a11) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: new function get_layer_dependencies()Nicolas Dechesne2021-08-101-1/+6
| | | | | | | | | | | | | | | Split add_layer_dependencies() into 2 parts. First search for layer dependencies, and then add them to the config. That allows us to call get_layer_dependencies() independently. (From OE-Core rev: e11c7ceed212c0d38e3d38de3099389a4e397818) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 08edf928aac3f2daaa0c256d4c21e56e2db72bff) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-check-layer: improve missed dependenciesNicolas Dechesne2021-08-101-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The first 2 calls to add_layer_dependencies() are here to add all dependencies for the 'layer under test' and the additional layers provided by the users. In both cases, we use misssing_dependencies boolean to indicate if any dependency is missing. But we then never really use missing_dependencies. Instead the script is calling add_layer_dependencies() again (for both the layer under test, and the additional layers) to detect if there are any missing dependency. As a result, we are trying to add again all dependencies, and we can see that from the traces: INFO: Detected layers: INFO: meta-aws: LayerType.SOFTWARE, /work/oe/sources/meta-aws INFO: checklayer: Doesn't have conf/layer.conf file, so ignoring INFO: INFO: Setting up for meta-aws(LayerType.SOFTWARE), /work/oe/sources/meta-aws INFO: Adding layer meta-python INFO: Adding layer meta-oe INFO: Adding layer meta-networking --> INFO: Adding layer meta-python INFO: meta-python is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf INFO: Adding layer meta-oe INFO: meta-oe is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf INFO: Adding layer meta-networking INFO: meta-networking is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf <-- INFO: Getting initial bitbake variables ... The code appears more complex than it should, and we can simply replace the complex if statement by using missing_dependencies, and avoid duplicating the call to add_layer_dependencies(). (From OE-Core rev: 84e63d179d935a071730e89a0963bb9ae867c93b) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fceb84f7bc472731b8f96ee1ebf0f4485943226c) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-setup-builddir: update YP docs and OE URLsMichael Opdenacker2021-08-101-2/+2
| | | | | | | | | | | | | This updates the link to the YP docs and proposes to access the OE website through https (From OE-Core rev: a73c10667b3ef2d0606dc5b2e4ff9605e2cd52f6) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 87686233aeffc639c3f412fd5c4898b32b15013b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-check-layer: Remove duplicated codeRichard Purdie2021-08-101-2/+0
| | | | | | | | | | | | | | | Nicolas Dechesne spotted there was duplicate code I had introduced with a previous fix. Remove the second statement since the earlier one is correct all that is needed. (From OE-Core rev: aa8e0928bda4a01623552084fc0ac0658e495b53) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f3e8d9f0e53e73de5498fccce81d049a88f6473b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Fix preserving attributes when using --stripFlorian Amstutz2021-07-101-1/+1
| | | | | | | | | | | | | | | | | Commit a2db4fa127a3347fc6df31f895fb0b552669119e added ${WORKDIR}/deploy-* to PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to deploy-target-stripped. Use the directory devtool-deploy-target-stripped instead. [YOCTO #14451] (From OE-Core rev: 9e2c7750f57bd1baa429d2f28a4c836ee57d1bfc) Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchart/draw: Avoid divide by zero errorRichard Purdie2021-05-131-1/+1
| | | | | | | | | | | | | | | When disk stats don't run frequenctly enough, we see divide by zero errors. The code already has a fallback path so ensure we use it for this case too. [YOCTO #14360] (From OE-Core rev: 6c4355cd13ae47c327ddc5e9b66623e44ba5b118) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b71d30aef5dc2c360432c0dd4147859dd303ea48) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-check-layer: Avoid bug when iterating and autoadding dependenciesRichard Purdie2021-05-041-0/+3
| | | | | | | | | | | | | | | | | | | If iterating a layer with multiple components and auto-adding dependencies the tests can break since layers are never removed and order isn't guaranteed to account for that. Fix this by resetting the layer list back to the original list each time before auto-adding the dependencies in each case. This fixes scanning of meta-openembedded in particular where the sublayers may not be added in order of minimal dependency. (From OE-Core rev: 886752b6877438c16b1f27b3b05798498026e645) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bf1b467dacf345379cd5d84a1c9b3b0d844d5c91) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* documentation-audit.sh: Fix typo in specifying LICENSE_FLAGS_WHITELISTKhem Raj2021-04-061-1/+1
| | | | | | | | | | (From OE-Core rev: ca95497a2d29620f84849a8f92b8bd05aae107f8) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 410a45639d84a3d69a65133593da32062196dd59) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/verify-bashisms: Update checkbashisms.pl URLPetr Vorel2021-03-281-1/+1
| | | | | | | | | | | | anonscm.debian.org is not used any more, update the URL. (From OE-Core rev: 34e1adf61a84ad2e697112a7582d2d5dba3cd4cb) Signed-off-by: Petr Vorel <petr.vorel@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3c89071a2c19a522c1df5919f500b88339ec6294) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-whatchanged: change ending quote to proper periodRobert P. J. Day2021-03-281-1/+1
| | | | | | | | | | | | | Pretty sure that trailing quote should be a period; it appears to work properly. (From OE-Core rev: f63ae3256cb880564f1b786e4c381ffae5cb58e8) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 579f9ada19bd174bff0002cd6a731d12a1868252) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix do_kernel_configme taskAlejandro Hernandez Samaniego2021-03-281-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | The do_kernel_configme task is no longer part of SRCTREECOVEREDTASKS, its been removed from the kernel-yocto.bbclass since b72dbb2e4, but there wasnt a matching patch for devtool for those changes. This patch enables us to invoke the do_kernel_configme task when using a devtool workspace, it also prepends a check for an existing .config file in the source directory and moves it if thats the case, since when using devtool modify a .config is created and do_kernel_configme complains about it, this is not the case when using bitbake since the .config file would be on B instead. Alowing do_kernel_configme to run also fixes the flow where testing a new config fragment from devtool workspace isnt added properly (config queue shows it as ///frg.cfg) and as a side effect it never gets merged into the final config. (From OE-Core rev: 5f27d66b5e3a7bffc23c140edf76fc8fd4cd5b23) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 08dcc0e68095dcf2a159546a48b29d40c9aabc0b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: use "raw" instead of "bin" for ovmfJon Mason2021-03-181-0/+2
| | | | | | | | | | | | | | bin is not an acceptable format for qemu and will throw an error is set. The correct format in this case is raw. (From OE-Core rev: b3cbde17aff693e2d5ba6d61922fca54e75b9f6e) Signed-off-by: Jon Mason <jon.mason@arm.com> Change-Id: I9d78e356d39ecb15bef886c19870b029d9badd65 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit cd279aeb986b4676ea8ecb1e7b12c7e29e83460a) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Warn if an ext filesystem affected by the Y2038 problem is usedFlorian Bezdeka2021-03-185-3/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 filesystems are always affected by the time overflow, so let's warn the user if these filesystems are still being used. If ext4 is affected depends on the inode size chosen during filesystem creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Some filesystems created during CI runs were already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to auto-detect the usage type based on the filesystem size. mke2fs will use the default values for tuning parameters instead. The inode size is one of these parameters. (From OE-Core rev: 280bf295a12b49a1d5199d86da6641516df0ce6a) Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit eecbe625558406680121d2a7e84917fea45ea9dc) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-pkgdata-util: Check if environment script is initializedDorinda2021-02-181-0/+3
| | | | | | | | | | | | | | Tinfoil doesn't behave well if environment is not initialized, this check ensures a proper error log if environment is not initialized. [YOCTO #12096] (From OE-Core rev: dfa4b0e6cabb870a33627ff5a0b5f413f6edb1e2) Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e88073e16f1b4cfd0f97c81a988640a84adad674) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix file:// fetcher symlink directory structureTomasz Dziendzielski2021-02-051-1/+5
| | | | | | | | | | | | | | Add relative path if file is under directory. [YOCTO #13738] (From OE-Core rev: cd9cee29b656e4c8642971cb87154cc4d5fda599) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 19ddacc1b38f9ebb86a9359963ccc3c707f7125e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix source extraction for gcc shared sourceTomasz Dziendzielski2021-02-051-4/+26
| | | | | | | | | | | | | | | If do_patch task is disabled then prepare do_configure dependencies to fetch external sources and create symlink to ${S} in devtool workspace. [YOCTO #13036] (From OE-Core rev: 1685e470f0dc49c1b857ffd869407f3396c55d31) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Optimise fstab modification for ext2/3/4 and msdos partitionsPaul Barker2021-02-032-8/+28
| | | | | | | | | | | | | | | | | | | | | The fix for [Yocto #13994] required the rootfs directory to be copied (using hardlinks if possible) when modifying the fstab file under wic. We can optimise this copy away for filesystems where we have the tools to modify the contents of the partition image after it is created. For ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat filesystems we have the mcopy tool. So for any of these filesystems we skip the modification of the fstab file in the rootfs directory (and skip the associated copy unless it is otherwise necessary) and update the contents of fstab directly in the partition image. (From OE-Core rev: 1988d07b65ad38bdf8fac8615f11fb6536a75806) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5fb8ae0e9159597d7eaa9307a3a8543800bf9405) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Copy rootfs dir if fstab needs updatingPaul Barker2021-02-033-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, wic updates the /etc/fstab in the rootfs to include details of additional partitions described in the selected wks file. If this modification is performed in place, other tasks which create an image file from the rootfs directory (e.g. do_image_tar and do_image_ext4) will pick up the modified fstab file which would not be appropriate for those images as they do not include the additional partitions described in the wks file. wic does undo modifications to the fstab file once it has finished creating the filesystem image, however this leaves open a race condition if one of the other tasks reads the contents of the fstab file from the rootfs directory between the point where wic modifies the fstab file and the point where wic restores the files original content. This could be solved by adding a lockfile for tasks which use the rootfs directory to ensure that no other such task is reading the rootfs directory while do_image_wic is running. This would serialize several do_image_* tasks and result in slower builds, especially for large images. Another drawback of this solution is that it is hard to selectively optimise - adding lockfiles to do_image_* tasks would result in these tasks always being serialized even if no fstab modification will take place. An alternative solution is to copy the rootfs directory when fstab needs to be modified. The code to do this in wic already exists as it is needed when including or excluding content in the rootfs. This still results in an impact on build times but the copy uses hardlinks if possible (so little data is actually copied) and we can make selective optimisations to improve things. The rootfs copy will only take place if fstab modification is required (or if it was already needed to include or exclude rootfs content). We can also follow up with further optimisations after this commit. So this second solution is chosen. Fixes [Yocto #13994] (From OE-Core rev: 9414007dd73ffd41d1e9c68bae152e8cbb3c28a2) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ce682a73b7447652f898ce1d1d0416a456df5416) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add --change-directory argumentRicardo Ribalda Delgado2021-02-034-3/+26
| | | | | | | | | | | | | | | | | | | | | | This option allows to specify which part of a rootfs is going to be included, the same way the -C argument on tar. Thanks to this option we can make sure the permissions and usernames on the target partition are respected, and also simplify the creation of splitted partitons, not neeting to invoke external vars or using .wks.in files. Eg: part / --source rootfs --ondisk sda --fstype=ext4 --exclude-path=etc/    part /etc --source rootfs --fstype=ext4 --change-directory=etc Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: d17b03e49b09f1a2db3642823522ac39e7c3c301) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2265d089a58e1f78f26d623ee667c420cb1c3bd4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Update pseudo db when excluding content from rootfsPaul Barker2021-02-031-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. (From OE-Core rev: 5bfdebe05a4eed1df29b2ad7c3871d323646cc50) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d5db7e268947f0392c2126137571a44acd29ccd6) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Ensure internal workdir is not reusedPaul Barker2021-02-031-1/+11
| | | | | | | | | | | | | | | | | If a path is specified for the internal wic working directory using the -w/--workdir argument then it must not already exist. Re-using a previous workdir could easily result in rootfs and intermediate files from a previous build being added to the current image. (From OE-Core rev: fb884fb9e5eef1c322ae7c196aa19bd32dcf6897) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2e40c8d4109024ff704c5ce40d98050ca7f34dd5) Signed-off-by: Steve Sakoman <steve@sakoman.com> (cherry picked from commit 5dcdb76f89c2e5bbc383e2921589482cd1c51491) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Allow exec_native_cmd to run HOSTTOOLSPaul Barker2021-02-031-2/+5
| | | | | | | | | | | | | | This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to be more easily executed by wic. Without this change only programs from an actual *-native recipe built by bitbake can be executed by wic. (From OE-Core rev: 7acb39f02b8160600d1508da114e98dd1c3e76ed) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8eb186acdecfbb3151c9a0ab148358e3fe5cce39) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add workdir argumentPaul Barker2021-02-032-1/+3
| | | | | | | | | | | | | | | This allows the path for the temporary workdir used by wic to be set when running wic from bitbake or directly from the command line. (From OE-Core rev: 447643eebff00715bd9717ebcb75edb20fc69778) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2e7314ac3a3cab89de93c932e8efc75de0feb0a7) Signed-off-by: Steve Sakoman <steve@sakoman.com> (cherry picked from commit a2308800b7eb5215982db7269d74d6292075f19f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHSPeter Kjellerstedt2021-02-031-1/+11
| | | | | | | | | | (From OE-Core rev: 4117e497650a0d8e7709439385c926d58a7d9aca) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 55618cbd58d6784a82e773f323723be6f722232f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Handle new PSEUDO_IGNORE_PATHS variableRichard Purdie2021-02-031-5/+7
| | | | | | | | | | | | Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid inode corruption issues. (From OE-Core rev: 16fbe45f2e0e7621139ae26ace59889a91fceda2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 13500f5234361385c365c7c35e83f99435500481) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Avoid creating invalid pseudo directoryRicardo Ribalda Delgado2021-02-031-7/+8
| | | | | | | | | | | | | | | | | | | If the source of the rootfs is not a bitbake cooked image, or it is not pointing to the root of one, we call pseudo again, which will produce a new pseudo folder at rootfs/../pseudo Eg: part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4 Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: b4393f56069801c014e736100dcdeab77e549f08) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Fix multi images .wks with bitbakeRicardo Ribalda Delgado2021-02-031-6/+3
| | | | | | | | | | | | | | | | | | | | | | | | In order to support .wks files with multiple images inside bitbake we need to explicitly set the pseudo database in use. Eg: If we try this .mks: part / --source rootfs --ondisk sda --fstype=ext4 part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4 The username for all the files under /export will be set to the runner of bitbake (usually UID 1000). Before we run wic, we need to make sure that the pseudo database will be flushed, and contains all the data needed. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 63bebeaf60757967b47cae1e24aa61ec5bcbd61b) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dde90a5dd2b22a539095d1bac82acc15c6380ac8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Fix permissions when using exclude or include pathRicardo Ribalda Delgado2021-02-032-6/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When parameters include_path or exclude_path are passed to the rootfs plugin, it will copy the partition content into a folder and make all the modifications there. This is done using copyhardlinktree(), which does not take into consideration the content of the pseudo folder, which contains the information about the right permissions and ownership of the folders. This results in a rootfs owned by the user that is running the wic command (usually UID 1000), which makes some rootfs unbootable. This bug can be easily triggerd with the following .wks part / --source rootfs --fstype=ext4 --exclude-path=home And this sequence: $ wic create test-permissions -e core-image-minimal -o test/ $ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt $ ls -la /mnt/etc/shadow To fix this we copy the content of the pseudo folders to the new folder and modify the pseudo database using the "pseudo -B" command. If the rootfs is not a rootfs generated by bitbake a warning is shown making the user aware that the permissions on the target might not match what he expects. WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 9ea7162e9efc29beacdf1c8f4ec98f4895dd5df6) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fd739c15cdba221d9d497d3402b7d929c0be2ca4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-14 15:34:35 +0000