summaryrefslogtreecommitdiffstats
path: root/scripts/lib
Commit message (Collapse)AuthorAgeFilesLines
* recipetool: Fix circular reference in SRC_URISaul Wold2022-02-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating a new recipe.bb file for a binary, don't use BP which includes the version information, instead use BPN which is just the name base Package Name. Since PB is not specified, it takes the default: PV = "1.0+git${SRCPV}" But SRCPV is defined in terms of the SRC_URI, which leads to infinite recursion (traceback below). Here are the pertinent variables which cause the recursion: SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}" BP = "${BPN}-${PV}" PV = "1.0+git${SRCPV}" SRCPV = "${@bb.fetch2.get_srcrev(d)}" def get_srcrev(d, method_name='sortable_revision'): # ... trimmed scms = [] fetcher = Fetch(d.getVar('SRC_URI').split(), d) # ... trimmed [YOCTO #14040] (From OE-Core rev: 5e5874d9b8f954771edb4ee3725bb31c1c5a70f2) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Update to use exec_module() instead of load_module()Richard Purdie2022-01-252-4/+11
| | | | | | | | | | | | This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. (From OE-Core rev: 85194f3264551adfffd168dd55e7f2a77f517a55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 655cd3f614d736416eab0d708b7c49674bf5c977) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Set master branch only as fallbackStefan Herbrechtsmeier2021-12-151-8/+7
| | | | | | | | | | | | | | | | The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)' sets the branch= parameter too early to master and thereby breaks the -B/--srcbranch option. ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other (From OE-Core rev: 15919898fa8dc40f4fe9c668d56e1d2afd52c894) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: extend curl detection when creating recipesRoss Burton2021-12-151-1/+2
| | | | | | | | | | | | If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl. (From OE-Core rev: 39e5b3b37c3a0116fb01be6183b320d9afea3bfe) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 16e830ca1352cee61e4ae4b98b1203f849bf71f3) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: handle GitLab URLs like we do GitHubRoss Burton2021-12-151-1/+1
| | | | | | | | | | | | | GitHub URLs are automatically transformed to git: fetches, so handle GitLab URLs too. (From OE-Core rev: 7cc06b578612e4965add12181d76dadc81a9e873) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 651fb951819840fe122458ddbd852ee6c7ec0455) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/checklayer/common.py: Fixed a minor grammatical errorDhruva Gole2021-12-151-1/+1
| | | | | | | | | | (From OE-Core rev: f4b21b2de13834b8308ef5a247cd632ef67dc591) Signed-off-by: Dhruva Gole <goledhruva@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8ea17456ae5318ed7a3b4c8f75c8441456d8b979) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: support rootdev identified by partition labelSchmidt, Adriaan2021-12-151-0/+2
| | | | | | | | | | | | | We already support specifying the rootfs by PARTUUID. This adds general support for letting the kernel find the rootfs by PARTLABEL. (From OE-Core rev: e10d5f31566285d66bbab29f5e2d9d8a79ee8e85) Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2fb247c5ecf057bb96649a3c0234794b4991c050) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use shutil.whichMingli Yu2021-12-052-5/+5
| | | | | | | | | | | | | | | | | | | | Use shutil.which to find the executable instead to silence the below warning: $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic [snip] DEBUG: Executing shell function do_image_wic /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import spawn INFO: Creating image(s)... [snip] [RP: Added conversion for missed function reference] (From OE-Core rev: 17e2839001076f7fc56a7cf62196b087351083f1) Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/help.py: Update Fedora Kickstart URLsJon Mason2021-11-241-2/+2
| | | | | | | | | | | | | The URLs describing Kickstart are no longer valid and do not redirect to the correct location. Update them with the correct location. (From OE-Core rev: cb00474c1a3074cd36b9bd69f3563180784f5513) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e5ac75f93c8128b0761af5fee99e8603ddd1657d) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Manual git url branch additionsRichard Purdie2021-11-211-0/+3
| | | | | | | | | | | | | Following the scripted conversion adding branches to git:// SRC_URI entries, add the remaining references, mainly in the selftests and recipetool. (From OE-Core rev: 87ac0c0a643cca436784dfcd86472adcf2e04130) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5340c0d688036c1be6c938f05d8a8c1e3b49ec38) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scriptutils.py: Add check before deleting pathChandana kalluri2021-10-071-1/+2
| | | | | | | | | | | | | | | | | | | | | | Add a check before deleting path when using recipetool commands to avoid the following type of errors: Traceback (most recent call last): File "<workdir>/sources/core/scripts/lib/scriptutils.py", line 218, in fetch_url shutil.rmtree(path) File "/usr/local/lib/python3.7/shutil.py", line 476, in rmtree onerror(os.lstat, path, sys.exc_info()) File "/usr/local/lib/python3.7/shutil.py", line 474, in rmtree orig_st = os.lstat(path) FileNotFoundError: [Errno 2] No such file or directory: '<workdir>/build/tmp/work/recipetool-usg7o81n/work/recipe-sysroot' ERROR: Command 'script -e -q -c "recipetool --color=always create --devtool -o /tmp/devtool5sq_op37 'file:///<SRCTREE>' -x <workdir>/build/workspace/sources/devtoolsrcxc1b9zjq -N test" /dev/null' failed (From OE-Core rev: 366070c476405a1f49e22a02c21fd99fc0ec76d9) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b6aa8b47e023004ffd6958d1cec18c2d9c95d77b) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: keep rootfs_size as integerAlexander Kanavin2021-10-071-1/+1
| | | | | | | | | | | | | | | | | The corrected line accidentally converted it to float, which causes problems later on with python 3.10: | File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext | os.ftruncate(sparse.fileno(), rootfs_size * 1024) | TypeError: 'float' object cannot be interpreted as an integer (From OE-Core rev: 5b6cd58a000c4a1eaac95e37a047838df6478305) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d1d260dd2d196d10379ed9e238bcb34f39f3a3b7) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: don't forcibly pass -T defaultRoss Burton2021-08-263-3/+3
| | | | | | | | | | | | | | | | | This reverts part of oe-core eecbe62555, which was a previous attempt to solve the Y2038 problem. This is now solved centrally in e2fsprogs, so doesn't need to be dealt with in wic. We don't revert the commit entirely, to retain the warning if a filesystem has small inodes. (From OE-Core rev: c727397e2544dffe3cccb8be3c87cf49227560f6) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7e8017208bed98b6c90735cb641fc9d7aedf9140) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: print a warning on upgrades if PREFERRED_VERSION is setAlexander Kanavin2021-08-141-0/+3
| | | | | | | | | | (From OE-Core rev: 5fb2fb70d14b0e9f6235e5b940e9a3e1145750bb) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0b0f53eed0aadbf45d9eead96ebf7725cc7447e6) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: rename _find_layer_dependsNicolas Dechesne2021-08-061-2/+2
| | | | | | | | | | | | | | | What this function does is really to find a layer, not a 'depends'. We are using this function to find a dependent layer, but the name is confusing. (From OE-Core rev: fa52d417ed578194274003c12e14cf93bdb8f7fb) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e9b7690ab30d0e7c07471034f6cb89ccc3168a11) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: new function get_layer_dependencies()Nicolas Dechesne2021-08-061-1/+6
| | | | | | | | | | | | | | | Split add_layer_dependencies() into 2 parts. First search for layer dependencies, and then add them to the config. That allows us to call get_layer_dependencies() independently. (From OE-Core rev: a3d5e108b6982771c99a75abf09a8210788c5be1) Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 08edf928aac3f2daaa0c256d4c21e56e2db72bff) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: correctly handle non-standard source tree locations in upgradesAlexander Kanavin2021-08-061-3/+12
| | | | | | | | | | | | | | | | | | When S is set to a sub-directory of upstream source, the license checks and the bbappend writing (specifically, setting EXTERNALSRC) need to operate on that sub-directory. 'devtool modify' already has similar logic, and it was copied from there and adjusted. (From OE-Core rev: ddf5a7f3b644915d360f725eca8322187226f171) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fd77e356d4507405fde352c8bba7d3842518bbdd) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Fix preserving attributes when using --stripFlorian Amstutz2021-07-201-1/+1
| | | | | | | | | | | | | | | | | Commit a2db4fa127a3347fc6df31f895fb0b552669119e added ${WORKDIR}/deploy-* to PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to deploy-target-stripped. Use the directory devtool-deploy-target-stripped instead. [YOCTO #14451] (From OE-Core rev: 421a199ad31f7cd7fb50e3abf9cdd41fed5532dd) Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use the label provided when formating a dos partitionjbouchard2021-07-061-2/+4
| | | | | | | | | | | | | | | | | Previously the bootimg-pcbios wic plugin was not respecting the --label option provided from the wks file. The plugin was setting the label to 'boot'. With this fix, the --label option is use. If no option are specified, then the default is 'boot'. (From OE-Core rev: 2fc2d0874fcb51abda249218443a71a42d9f02e9) Signed-off-by: jbouchard <jeanbouch418@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool upgrade: rebase override-only patches as wellAlexander Kanavin2021-07-061-15/+14
| | | | | | | | | | | | | | | | | | | | | | | | There's a number of recipes (such as nettle or glib) which were upgraded incorrectly due to rebasing only the default set of patches in main SRC_URI. Native-only or musl-only patches were not handled, causing do_patch fails and overall misery. Unfortunately, this also necessitates aborting any incomplete rebases as devtool needs to be able to return to the original branch to complete the upgrade command. Also, do not add devtool-patched-%version tags, as they are not used anywhere, and it's unclear how to extend them to cover multiple override branches. (From OE-Core rev: 00131cbc969f6a73caffe068fcb983287cbe577f) (From OE-Core rev: a4a5bebcc06637e097f074be9e401aafa8685c38) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 588d04748dd89b5371ca9144666e97ec82dd6204) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix do_kernel_configme taskAlejandro Hernandez Samaniego2021-03-141-1/+6
| | | | | | | | | | | | | | | | | | | | | | | The do_kernel_configme task is no longer part of SRCTREECOVEREDTASKS, its been removed from the kernel-yocto.bbclass since b72dbb2e4, but there wasnt a matching patch for devtool for those changes. This patch enables us to invoke the do_kernel_configme task when using a devtool workspace, it also prepends a check for an existing .config file in the source directory and moves it if thats the case, since when using devtool modify a .config is created and do_kernel_configme complains about it, this is not the case when using bitbake since the .config file would be on B instead. Alowing do_kernel_configme to run also fixes the flow where testing a new config fragment from devtool workspace isnt added properly (config queue shows it as ///frg.cfg) and as a side effect it never gets merged into the final config. (From OE-Core rev: 08dcc0e68095dcf2a159546a48b29d40c9aabc0b) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/partition.py: do not set FAT sizeAlexander Kanavin2021-03-021-4/+0
| | | | | | | | | | | Modern dosfstools automatically determines the appropriate size and will error out if something that doesn't make sense is supplied on the command line. (From OE-Core rev: b85a09ea450a5e8f49418f4a930805fbb88dc83b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Warn if an ext filesystem affected by the Y2038 problem is usedFlorian Bezdeka2021-02-265-3/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 filesystems are always affected by the time overflow, so let's warn the user if these filesystems are still being used. If ext4 is affected depends on the inode size chosen during filesystem creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Some filesystems created during CI runs were already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to auto-detect the usage type based on the filesystem size. mke2fs will use the default values for tuning parameters instead. The inode size is one of these parameters. (From OE-Core rev: eecbe625558406680121d2a7e84917fea45ea9dc) Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: debug mode to keep tmp directoryLee Chee Yang2021-02-091-2/+4
| | | | | | | | | | | | | | | files in wic tmp directory can be usefull for debugging, so do not remove tmp directory when wic create run with debugging mode (-D or --debug). also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to check for tmp directory. [YOCTO#14216] (From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9) Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix file:// fetcher symlink directory structureTomasz Dziendzielski2021-01-231-1/+5
| | | | | | | | | | | Add relative path if file is under directory. [YOCTO #13738] (From OE-Core rev: 19ddacc1b38f9ebb86a9359963ccc3c707f7125e) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix source extraction for gcc shared sourceTomasz Dziendzielski2021-01-201-4/+26
| | | | | | | | | | | | If do_patch task is disabled then prepare do_configure dependencies to fetch external sources and create symlink to ${S} in devtool workspace. [YOCTO #13036] (From OE-Core rev: 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Optimise fstab modification for ext2/3/4 and msdos partitionsPaul Barker2021-01-202-8/+28
| | | | | | | | | | | | | | | | | | The fix for [Yocto #13994] required the rootfs directory to be copied (using hardlinks if possible) when modifying the fstab file under wic. We can optimise this copy away for filesystems where we have the tools to modify the contents of the partition image after it is created. For ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat filesystems we have the mcopy tool. So for any of these filesystems we skip the modification of the fstab file in the rootfs directory (and skip the associated copy unless it is otherwise necessary) and update the contents of fstab directly in the partition image. (From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Copy rootfs dir if fstab needs updatingPaul Barker2021-01-203-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, wic updates the /etc/fstab in the rootfs to include details of additional partitions described in the selected wks file. If this modification is performed in place, other tasks which create an image file from the rootfs directory (e.g. do_image_tar and do_image_ext4) will pick up the modified fstab file which would not be appropriate for those images as they do not include the additional partitions described in the wks file. wic does undo modifications to the fstab file once it has finished creating the filesystem image, however this leaves open a race condition if one of the other tasks reads the contents of the fstab file from the rootfs directory between the point where wic modifies the fstab file and the point where wic restores the files original content. This could be solved by adding a lockfile for tasks which use the rootfs directory to ensure that no other such task is reading the rootfs directory while do_image_wic is running. This would serialize several do_image_* tasks and result in slower builds, especially for large images. Another drawback of this solution is that it is hard to selectively optimise - adding lockfiles to do_image_* tasks would result in these tasks always being serialized even if no fstab modification will take place. An alternative solution is to copy the rootfs directory when fstab needs to be modified. The code to do this in wic already exists as it is needed when including or excluding content in the rootfs. This still results in an impact on build times but the copy uses hardlinks if possible (so little data is actually copied) and we can make selective optimisations to improve things. The rootfs copy will only take place if fstab modification is required (or if it was already needed to include or exclude rootfs content). We can also follow up with further optimisations after this commit. So this second solution is chosen. Fixes [Yocto #13994] (From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Update pseudo db when excluding content from rootfsPaul Barker2021-01-201-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. (From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Ensure internal workdir is not reusedPaul Barker2021-01-201-1/+11
| | | | | | | | | | | | If a path is specified for the internal wic working directory using the -w/--workdir argument then it must not already exist. Re-using a previous workdir could easily result in rootfs and intermediate files from a previous build being added to the current image. (From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: only add npmsw url if requiredKamel Bouhara2021-01-161-1/+5
| | | | | | | | | | | Before adding a npmsw fetcher to a recipe we should first check if the generated shrinkwrap file contains dependencies. (From OE-Core rev: ef153ad36d0299e83a03af8f207686d0d8a238b3) Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Allow exec_native_cmd to run HOSTTOOLSPaul Barker2021-01-131-2/+5
| | | | | | | | | | | This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to be more easily executed by wic. Without this change only programs from an actual *-native recipe built by bitbake can be executed by wic. (From OE-Core rev: 8eb186acdecfbb3151c9a0ab148358e3fe5cce39) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add workdir argumentPaul Barker2021-01-131-1/+1
| | | | | | | | | | This allows the path for the temporary workdir used by wic to be set when running wic from bitbake or directly from the command line. (From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/direct/kparser: ensure fsuuid for vfat and msdos align with formatLee Chee Yang2021-01-012-0/+24
| | | | | | | | | | | | | | | | | | | | | | vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x" in front follow with 8 hexadecimal number in uppercase. In wic, when using custom fsuuid for vfat/msdos partition in wks, it is able to set the value in any length, with or without leading "0x". This can cause fsuuid missaligned when fstab updates, fstab expect exactly 10 character fsuuid for vfat/msdos partition and all in uppercase. if custom fsuuid for vfat/msdos is set, check the length and format, error if it exceed the format size. Amend it so it is align with format 0xYYYYYYYY. This is done before image create and fstab update to ensure the fsuuid are same in all followup process. if custom fsuuid length less than expected, fill in "0". [YOCTO #14161] (From OE-Core rev: d9686ae511ef10a504becfd81bfe296b788e1456) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: gitsm:// should be handled same as git:// in upgradesAlexander Kanavin2020-12-311-2/+2
| | | | | | | (From OE-Core rev: e7076f1742bb83e76aa4e90aad861546536166cf) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHSPeter Kjellerstedt2020-12-201-1/+11
| | | | | | | (From OE-Core rev: 55618cbd58d6784a82e773f323723be6f722232f) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Introduce empty plugin to create unformatted empty partitionsDiego Sueiro2020-12-151-0/+32
| | | | | | | | | | | | | | | | | | The empty wic plugin is used to create unformatted empty partitions for wic images. To use it you must pass "empty" as argument for the "--source" parameter in the wks file. For example: part foo --source empty --ondisk sda --size="1024" --align 1024 Also adds a selftest for this plugin where the 'Fstype' column from 'wic ls' should be empty for the second partition as listed in test_empty_plugin.wks. (From OE-Core rev: 77d174fc80663403ef76c5b808aafc1117d3545c) Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* isoimage-isohybrid.py: Support adding files/dirsChaitanya Vadrevu2020-11-081-0/+14
| | | | | | | | | | Add "payload_dir" parameter so user can add other files/dirs to the ISO. (From OE-Core rev: bbe84a33c81066fee96dd28a4009d8404b51e642) Signed-off-by: Chaitanya Vadrevu <chaitanya.vadrevu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: make sure .git/info exists before writing to .git/info/excludesRoland Hieber2020-10-171-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | If nothing else is specified, 'git init' uses its default repository template from the install location (e.g. /usr/share/git-core/templates), which already includes an info/ subdirectory. However, when setting init.templateDir to a different template path in ~/.gitconfig, this isn't necessarily the case, and it can lead to setup_git_repo() failing with stack traces like: File: '.../scripts/lib/devtool/__init__.py', lineno: 234, function: setup_git_repo 0230: pass 0231: if 'singletask.lock\n' not in excludes: 0232: excludes.append('singletask.lock\n') 0233: bb.warn("try writing excludefile") *** 0234: with open(excludefile, 'w') as f: 0235: for line in excludes: 0236: f.write(line) 0237: 0238: bb.process.run('git checkout -b %s' % devbranch, cwd=repodir) Exception: FileNotFoundError: [Errno 2] No such file or directory: '.../devtooltmp-6m36b181/workdir/foobar-1.0.1/.git/info/exclude' Fix this edge case by creating the .git/info/ directory first. Fixes: 334ba846c795fc0d8c73 (2018-02-01, "devtool: set up git repos so that singletask.lock is ignored") (From OE-Core rev: 148a23e4d5ceaf655ccacb52deca4ba501f12975) Signed-off-by: Roland Hieber <rhi@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: remove unused variableRoss Burton2020-10-081-1/+0
| | | | | | | | | This variable isn't used at all. (From OE-Core rev: 9676ee24d190b144cb50be6ebde3990644a6df2a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Handle new PSEUDO_IGNORE_PATHS variableRichard Purdie2020-10-071-5/+7
| | | | | | | | | Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid inode corruption issues. (From OE-Core rev: 13500f5234361385c365c7c35e83f99435500481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi ↵Khairul Rohaizzat Jamaluddin2020-09-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | and bootimg-partition Due to recent changes in bootimg-efi to include IMAGE_BOOT_FILES, when both bootimg-partition and bootimg-efi occur in a single .wks and IMAGE_BOOT_FILES are defined, files listed in IMAGE_BOOT_FILES will be duplicated in both partition. Since IMAGE_BOOT_FILES are crucial for bootimg-partition, but optional for bootimg-efi, hence allowing bootimg-efi to have the option to ignore it. The new variable, IMAGE_EFI_BOOT_FILES, was added to help handle this issue. Its basic usage is the same as IMAGE_BOOT_FILES. Usage example: ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \ This commit is also squashed with the updated testcase to cover for this change. [YOCTO #14011] (From OE-Core rev: 945339e06b273df1935cfd784f548ef57e0b7f4c) Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Fix size calculation for hard linksMichael Tretter2020-09-021-1/+7
| | | | | | | | | | | | | | | | | | | | | | If a package contains hard links to a file, the file size is added for each hard link instead of once for the file. Therefore, the calculated size may be much larger than the actual package size. For example, the mesa-megadriver package contains several hard links to the same library. Keep track of the inode numbers when listing the files that are installed and use the actual size only for the first occurrence of an inode. All further hard links to the same inode are added to the file list, but accounted with size 0. All file names need to be added to the file list, because the list is used for preserving the files/hard links on the target. (From OE-Core rev: 17e92572278980d1a7f06de9d72c68baf57698f1) Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: misc: Add /bin to the list of searchpathsVijai Kumar K2020-09-021-2/+3
| | | | | | | | | | /bin is also a valid path where one can find executables. Add that to the search path. (From OE-Core rev: ca0a6025351cb2135e87cecf828633cf12aa34c6) Signed-off-by: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: expand SRC_URI when guessing recipe update modeMartin Jansa2020-08-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * I have recipes which use variable inside SRC_URI, e.g.: ROS_BRANCH ?= "branch=release/melodic/swri_nodelet" SRC_URI = "git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https" and devtool modify works fine, but devtool finish fails with: $ devtool finish --force-patch-refresh swri-nodelet meta-ros/meta-ros1-melodic/ ... Traceback (most recent call last): File "/OE/openembedded-core/scripts/devtool", line 334, in <module> ret = main() File "/OE/openembedded-core/scripts/devtool", line 321, in main ret = args.func(args, config, basepath, workspace) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 2082, in finish updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1737, in _update_recipe mode = _guess_recipe_update_mode(srctree, rd) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1721, in _guess_recipe_update_mode params = bb.fetch.decodeurl(uri)[5] File "/OE/bitbake/lib/bb/fetch2/__init__.py", line 390, in decodeurl raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s)) bb.fetch2.MalformedUrl: The URL: 'git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https' is invalid: parameter ${ROS_BRANCH} does not specify a value (missing '=') let it expand the SRC_URI before trying to decode it. (From OE-Core rev: 3535cfdbf3d77f550b804276f957acf859da484f) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add 512 Byte alignment to --offsetJoshua Watt2020-08-222-10/+18
| | | | | | | | | | | | | Allows the --offset argument to use the "s" or "S" suffix to specify that it is reporting the number of 512 byte sectors. This is required for some SoCs where the mask ROM looks for an item at a sector that isn't aligned to a 1KB boundary. (From OE-Core rev: 938595d1dc4abaf5f7f3a7900add3f0492b805d0) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: fix typoDaniel Ammann2020-08-061-1/+1
| | | | | | | (From OE-Core rev: 9ab4c4571d899d2eefcd2fc53af0851863f29008) Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: check layer in BBLAYERS before testLee Chee Yang2020-07-271-0/+14
| | | | | | | | | | | | | | | | | | layer under test should absent from BBLAYERS when running yocto-check-layer. This allow to get signatures before layer under test. There are existing steps to add the layer under test to BBLAYERS after getting initial signatures. add steps to check for layer under test in BBLAYERS before running any test, skip test for the layer if the layer under test exist in BBLAYERS. [YOCTO #13176] (From OE-Core rev: be02e8dbfb0d1decce125322f9f1e11a649756c0) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: Add support for IMAGE_BOOT_FILESKhasim Mohammed2020-07-251-0/+60
| | | | | | | | | | | | | | | List of files defined using IMAGE_BOOT_FILES are installed into the boot partition when preparing an image using the wic tool with the bootimg-efi source plugin. The code snippet introduced is taken as is from bootimg-partition.py Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782 Issue-Id: PLATFORMS-3134 (From OE-Core rev: a44ab3a4ee5b3c57812909c6194456f299d6ba7f) Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/filemap: Fall back to standard copy when no way to get the block mapKevin Hao2020-07-181-1/+27
| | | | | | | | | | | | | | For some filesystems, such as aufs which may be used by docker container, don't support either the SEEK_DATA/HOLE or FIEMAP to get the block map. So add a FileNobmap class to fall back to standard copy when there is no way to get the block map. [Yocto #12988] (From OE-Core rev: 7934ed49179242f15b413c0275040a3bb6b68876) Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-20 12:20:07 +0000