summaryrefslogtreecommitdiffstats
path: root/scripts/lib/wic
Commit message (Collapse)AuthorAgeFilesLines
* filemap.py: enforce maximum of 4kb block sizeAndrew Geissler2023-04-141-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | The logic in this script validates that the length of data sections are evenly divisible by the block size. On most systems the block size is 4KB and all is good. Some systems though, such as ppc64le, have a block size larger then 4KB. For example on a POWER9 based ppc64le system, the block size is 64KB. This results in this script failing with errors like this when building wic images: |440, in _do_get_mapped_ranges | assert extent_len % self.block_size == 0 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | AssertionError In this case the data section size was 268KB and the block size was 64KB, resulting in the above assert failure. Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075 (From OE-Core rev: 303cc9ce3a9d7ca85542f12ebfda27eeb449e73b) Signed-off-by: Andrew Geissler <geissonator@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1e23b803af6991fc20e4a4e88a0ef0541399e722) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* wic: Fix usage of fstype=none in wicPavel Zhukov2023-03-061-0/+5
| | | | | | | | | | | | | | | | | | | | | | | This allows to specify partition with fstype=none in the wks file to have partition created but without following mkfs. The none fstype is in the list already but the usage is not documented. Example; part /data --ondisk mmcblk0 --fstype=none --align 4096 --fixed-size 512 will create a partition, filesystem may be created manualy on the host or target and data will be preserved if the device is reflashed using same wks. Works with bmaptool and probably does not work with dd. Use case is persistent filesystem/data between reflashing of the image. (From OE-Core rev: 5f7239eebe61aabbd3dbf4e6e1cf703c71273a35) Signed-off-by: Pavel Zhukov <pavel@zhukoff.net> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 351cb64da37aa43113e5192605d04436652aa3b8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: make ext2/3/4 images reproducibleSergei Zhmylev2022-11-241-5/+24
| | | | | | | | | | | | | | | | Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime. Currently, all the files are being added into the rootfs image using mkfs -d functionality which affects all the timestamps excluding mtime. This patch ensures these timestamps inside the FS image equal to the SOURCE_DATE_EPOCH if it is set. (From OE-Core rev: e0dc0f4a153f75d14d50d5bc91edb93fac93cd25) Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 75d2dd0ea7790db2e8ee921784ca373abff2df65) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: swap partitions are not added to fstabciarancourtney2022-11-141-1/+1
| | | | | | | | | | | | - Regression in 7aa678ce804c21dc1dc51b9be442671bc33c4041 (From OE-Core rev: 8fdb75c0f0f7458305ccae657cf2722520e00572) Signed-off-by: Ciaran Courtney <ciaran.courtney@activeenergy.ie> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit f1243572ad6b6303fe562e4eb7a9826fd51ea3c3) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: honor the SOURCE_DATE_EPOCH in case of updated fstabSergei Zhmylev2022-11-093-2/+10
| | | | | | | | | | | | | | | In case user requested to build a binary repeatable package, it's required to honor the SOURCE_DATE_EPOCH environment variable. So forcefully set mtime inside all the routines which modify fstab in case it is updated. (From OE-Core rev: 4d3f43fe06186b6580395a161fdbc4470b8aab62) Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 99719a3712a88dce8450994d995803e126e49115) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "wic/plugins/images/direct: Allow changes in fstab on rootfs"Ross Burton2022-09-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | Commit 20d43a2 allows changes in fstab for the / mount. However, if the fstab contains / already then this results in two entries for / in the fstab, which is confusing and results in systemd producing errors on boot: systemd-fstab-generator[11101]: Failed to create unit file /run/systemd/generator/-.mount, as it already exists. Duplicate entry in /etc/fstab? Wic should be extended to support merging mount points in fstab, but as we're about to release revert the patch that introduced this change as it hasn't been in a release yet, and the issue which prompted the patch in the first place can be revisited post-release. [ YOCTO #14865 ] (From OE-Core rev: 4a7b3ef0a69b7e08030519b712a6fe4dd199c571) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add 'none' fstype for custom imageJeongBong Seo2022-08-122-1/+5
| | | | | | | | | | | | | | | | | It's not possible to set the label (of gpt entry) normally when I want to use non-listed fstype as a rawcopy. Example) part ? --source rawcopy --ondisk mmcblk0 --label mypart --sourceparams file=mypart.raw To resolve this problem, this patch addes a 'none' fstype and ignore do_image_label on rawcopy (that actually set the partition label.) (From OE-Core rev: 949cf797eb54f3d22a093910087e937633ce9b23) Signed-off-by: JeongBong Seo <jb.seo@lge.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Add copyright statements to files without oneRichard Purdie2022-08-124-0/+8
| | | | | | | | | | Where there isn't a copyright statement, add one to make it explicit. Also drop editor config lines where they were present and add license identifiers as MIT if there isn't one. (From OE-Core rev: deb3ccec53e0bd63bc4235cf2b0d3fc781687361) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: Add support for loading devicetree filesJan Kiszka2022-08-021-7/+31
| | | | | | | | | | | | | | | | | | | | For device tree using systems, add support to set a custom devices tree during UEFI boot. This requires to copy the DTB file to the boot partition and to add the respective loader entries to the configuration files. Both grub and systemd-boot support only loading a specific device tree. Therefore refuse to work if the 'dtb' parameter contains more than one entry. Out of scope for now are overlays (only supported by systemd-boot). (From OE-Core rev: c5a14f39a6717a99b510cb97aa2fb403d4b98d99) Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> [Luca: rebased on commit 98e9d435b278 ("wic/bootimg-efi: use cross objcopy when building unified kernel image")] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: Factor out some common bitsJan Kiszka2022-08-021-25/+21
| | | | | | | | | | | | | | | | The paths for configuring grub and systemd-boot have some common bits around copying the initrd files. This will even grow when adding dtb support. Factor this out into a class function. Along this, avoid evaluating 'create-unified-kernel-image' multiple times in do_configure_systemdboot and suppress a bogus warning about "Ignoring missing initrd" when it is turned on. (From OE-Core rev: c700cfd88473b9ed4e12a6620fb089f41bd95a9e) Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: use cross objcopy when building unified kernel imageRoss Burton2022-08-021-13/+12
| | | | | | | | | | | | | | We can't rely on the host objcopy knowing how to process target binaries, so use the cross objcopy in the sysroot instead. Also construct the command argument-by-argument as the format expression was getting unwieldy. (From OE-Core rev: 0264aeedbf21e9e7a104243c11b3b57f00e38bda) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add target tools to PATH when executing native commandsRoss Burton2022-08-021-3/+4
| | | | | | | | | | | | We might want to run a cross tool, such as objcopy, in wic. These are in a TARGET_SYS/ subdirectory under /usr/bin, so add that directory to the search path too. (From OE-Core rev: c523549141e5c31edc75281f581d97867b7d251d) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/plugins/rootfs: Fix NameError for 'orig_path'Mihai Lindner2022-07-231-1/+1
| | | | | | | | | | | Fix "NameError: name 'orig_path' is not defined". It's a typo from when this error was handled outside this function. (From OE-Core rev: 2124ec0d9f9de2da476f0024a0ccf70da987420f) Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/plugins/images/direct: Allow changes in fstab on rootfsTobias Schmidl2022-05-271-1/+1
| | | | | | | | | | | | | | Allow wic to also manipulate the rootfs entry in fstab, which it currently refuses to write. Reasons one might want to do that include using systemd-growfs via --fsoptions on / With this change / is now handled exactly the same as other mountpoints, the former exception seemingly was not even documented. (From OE-Core rev: 20d43a2599d7622b96e2fb0da87a886da1a3794a) Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: added fspassno parameter to partitionClaudius Heine2022-05-164-1/+10
| | | | | | | | | | | | | The `fspassno` parameter allows to overwrite the value of the last column (`fs_passno`) in the /etc/fstab of the target root file system. This allows to have periodic file system checks. (From OE-Core rev: b9b9f71e6f37bfbf954ade518391b242669481e3) Signed-off-by: Claudius Heine <ch@denx.de> Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/plugins/rootfs: Fix permissions when splitting rootfs folders across ↵Felix Moessbauer2022-05-081-1/+4
| | | | | | | | | | | | | | | | | | partitions This patches makes locating the file database containing the file and folder usernames and permissions more reliable. In addition to locating it relative to the partition directory, we also try to locate it relative to the IMAGE_ROOTFS. Prior to this patch, the database was not found when using --rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file permissions and ownership. (From OE-Core rev: 09e18ee246da8b56f446c4db548fb9c7e895142b) Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add dependencies for erofs-utilsSean Anderson2022-04-271-0/+1
| | | | | | | | | | | | In order to build erofs filesystems, wic must have the erofs-utils package installed into its sysroot. Fixes: 30375ce97 ("Add support for erofs filesystems") (From OE-Core rev: 68e364340c439a1341d37c3f7a2b0e6aad8e1e56) Signed-off-by: Sean Anderson <sean.anderson@seco.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: do not use PARTLABEL for msdos partition tablesHenning Schild2022-04-221-1/+1
| | | | | | | | | | | | | | | When using "msdos" partition tables and "--label" but not "--use-uuid" one can generate images which will not find their root, because PARTLABEL does not work for "msdos". Fix that by simply not going the PARTLABEL path in case of "msdos". Fixes: 2fb247c5ecf0 ("wic: support rootdev identified by partition label") (From OE-Core rev: 9ea1a838b946020e026edc032039552b723fcaa4) Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: rawcopy: Add support for packed imagesStefan Herbrechtsmeier2022-02-161-1/+28
| | | | | | | | | | | | | | | Add support for packed images to wic rawcopy handler do minimize disk usage in deploy directory and reuse of packed images between wic and swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz archives. Example: part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack" (From OE-Core rev: 4c97d25791389ece041565981ba3207ce9949a1a) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: partition: Support valueless keys in sourceparamsStefan Herbrechtsmeier2022-02-161-1/+1
| | | | | | | | | | Accept valueless keys in sourceparams without equals sign (=) to match the comment and support Boolean entries. (From OE-Core rev: 052a9b0f7d2ee5c284cf1b4f1f45caed0489221a) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Update to use exec_module() instead of load_module()Richard Purdie2022-01-051-2/+6
| | | | | | | | | This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. (From OE-Core rev: 655cd3f614d736416eab0d708b7c49674bf5c977) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: support rootdev identified by partition labelSchmidt, Adriaan2021-11-241-0/+2
| | | | | | | | | | We already support specifying the rootfs by PARTUUID. This adds general support for letting the kernel find the rootfs by PARTLABEL. (From OE-Core rev: 2fb247c5ecf057bb96649a3c0234794b4991c050) Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use shutil.whichMingli Yu2021-11-212-5/+5
| | | | | | | | | | | | | | | | | Use shutil.which to find the executable instead to silence the below warning: $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic [snip] DEBUG: Executing shell function do_image_wic /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import spawn INFO: Creating image(s)... [snip] [RP: Added conversion for missed function reference] (From OE-Core rev: 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87) Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/help.py: Update Fedora Kickstart URLsJon Mason2021-11-111-2/+2
| | | | | | | | | | The URLs describing Kickstart are no longer valid and do not redirect to the correct location. Update them with the correct location. (From OE-Core rev: e5ac75f93c8128b0761af5fee99e8603ddd1657d) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Use os.rename instead of bb.utils.renameKhem Raj2021-11-101-1/+1
| | | | | | | | | | | | | | | This is not invoked with bitbake context as a result bb.utils is not visible when this function is called during image creation and builds fail e.g. NameError: name 'bb' is not defined (From OE-Core rev: df9dca9fe4dd1abfe5f3986389a8e8ff524da5d7) Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Devendra Tewari <devendra.tewari@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: Add Unified Kernel Image optionKristian Klausen2021-10-041-10/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "A unified kernel image is a single EFI PE executable combining an EFI stub loader, a kernel image, an initramfs image, and the kernel command line. [...] Images of this type have the advantage that all metadata and payload that makes up the boot entry is monopolized in a single PE file that can be signed cryptographically as one for the purpose of EFI SecureBoot."[1] This commit adds a create-unified-kernel-image=true option to the bootimg-efi plugin for creating a Unified Kernel Image[1] and installing it into $BOOT/EFI/Linux/ with a .efi extension per the the Boot Loader Specification[1][2]. This is useful for implementing Secure Boot. systemd-boot is the only mainstream bootloader implementing the specification, but GRUB should be able to boot the EFI binary, this commit however doesn't implement the necessary changes to the GRUB config generation logic to boot the Unified Kernel Image. [1] https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images [2] https://systemd.io/BOOT_LOADER_SPECIFICATION/ (From OE-Core rev: b0573f240525df561ddef6e47cb285b217d38487) Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic:direct.py: ignore invalid mountpoints during fstab updateMarkus Volk2021-09-261-1/+1
| | | | | | | | | | | | | | | | | | | | | wic fstab-update creates invalid entries for partitons that are not supposed to be mounted from userspace eg u-boot partitions. The following lines were added to fstab on a rock-pi-4: /dev/mmcblk1p1 loader1 vfat defaults 0 0 /dev/mmcblk1p2 reserved1 vfat defaults 0 0 /dev/mmcblk1p3 reserved2 vfat defaults 0 0 /dev/mmcblk1p4 loader2 vfat defaults 0 0 /dev/mmcblk1p5 atf vfat defaults 0 0 /dev/mmcblk1p6 /boot vfat defaults 0 0 With this patch only valid entries should be added /dev/mmcblk1p6 /boot vfat defaults 0 0 (From OE-Core rev: 7aa678ce804c21dc1dc51b9be442671bc33c4041) Signed-off-by: MarkusVolk <f_l_k@t-online.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: keep rootfs_size as integerAlexander Kanavin2021-09-191-1/+1
| | | | | | | | | | | | | | The corrected line accidentally converted it to float, which causes problems later on with python 3.10: | File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext | os.ftruncate(sparse.fileno(), rootfs_size * 1024) | TypeError: 'float' object cannot be interpreted as an integer (From OE-Core rev: d1d260dd2d196d10379ed9e238bcb34f39f3a3b7) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add extra-space argumentKristian Klausen2021-09-061-2/+5
| | | | | | | | | | | | | This allows extra space to be added after the last partition and is especially useful when free space is needed for ex: adding partitions on first boot with ex: systemd-repart[1] and the image is tested in QEMU. [1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html (From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d) Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add --no-fstab-update part optionDaniel Gomez2021-08-204-3/+8
| | | | | | | | | | | | | | | | | | | | | | | When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we might want to keep the stock fstab for that image. In such a case, use this option to not update the fstab and use the stock one instead. This option allows you to specify which partitions get the fstab updated and which get the stock fstab. The option matches the argument you can pass to wic itself where the same action is performed but for all the partitions. Example: part /export --source rootfs --rootfs-dir=hockeycam-image --fstype=ext4 --label export --align 1024 --no-fstab-update part / --source rootfs --fstype=ext4 --label rootfs --align 1024 (From OE-Core rev: ab4c95af8ecd15dc136194ab761afae756db5803) Signed-off-by: Daniel Gomez <daniel@qtec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/help/py: "Redhat" -> "Red Hat"Robert P. J. Day2021-08-171-1/+1
| | | | | | | | | Company name is properly spelled as "Red Hat". (From OE-Core rev: cd60f20fe40bba5b80ab604ee34fbf4ec2b0b4a2) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: don't forcibly pass -T defaultRoss Burton2021-08-123-3/+3
| | | | | | | | | | | | | | This reverts part of oe-core eecbe62555, which was a previous attempt to solve the Y2038 problem. This is now solved centrally in e2fsprogs, so doesn't need to be dealt with in wic. We don't revert the commit entirely, to retain the warning if a filesystem has small inodes. (From OE-Core rev: 7e8017208bed98b6c90735cb641fc9d7aedf9140) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use the label provided when formating a dos partitionjbouchard2021-06-291-2/+4
| | | | | | | | | | | | | | Previously the bootimg-pcbios wic plugin was not respecting the --label option provided from the wks file. The plugin was setting the label to 'boot'. With this fix, the --label option is use. If no option are specified, then the default is 'boot'. (From OE-Core rev: 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141) Signed-off-by: jbouchard <jeanbouch418@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add support for erofs filesystemsRichard Weinberger2021-05-314-7/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Since Linux 5.4 a new read-only filesystem is available, erofs. Compared to squashfs it offers much better read performance with and without compression enabled. It suppports two optional compressors, lz4 and lz4hc. >From the mkfs.erofs man page: EROFS is a new enhanced lightweight linux read-only filesystem with modern designs (eg. no buffer head, reduced metadata, inline xattrs/data, etc.) for scenarios which need high-performance read-only requirements, e.g. Android OS for smartphones and LIVECDs. It also provides fixed-sized output compression support, which improves storage density, keeps relatively higher compression ratios, which is more useful to achieve high performance for embedded devices with limited memory since it has unnoticable memory overhead and page cache thrashing. This commit adds support for three new filesystem targets: erofs: erofs without compression erofs-lz4: erofs with lz4 compresssion enabled erofs-lz4hc: erofs with lz4hc compression enabled (From OE-Core rev: 41dead1ff8ccc49e6cd6e6f5d41a59d164693e0d) Signed-off-by: Richard Weinberger <richard@nod.at> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib/scripts: Use bb.utils.rename() instead of os.rename()Devendra Tewari2021-05-061-1/+1
| | | | | | | | | | | | | | | | | | Incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than adding fallback code to every call site, use a new wrapper in bitbake which detects this case and falls back to shutil.move which is slower but will handtle the overlay docker filesystems correctly. [YOCTO #14301] (From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/partition.py: do not set FAT sizeAlexander Kanavin2021-03-021-4/+0
| | | | | | | | | | | Modern dosfstools automatically determines the appropriate size and will error out if something that doesn't make sense is supplied on the command line. (From OE-Core rev: b85a09ea450a5e8f49418f4a930805fbb88dc83b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Warn if an ext filesystem affected by the Y2038 problem is usedFlorian Bezdeka2021-02-265-3/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 filesystems are always affected by the time overflow, so let's warn the user if these filesystems are still being used. If ext4 is affected depends on the inode size chosen during filesystem creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Some filesystems created during CI runs were already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to auto-detect the usage type based on the filesystem size. mke2fs will use the default values for tuning parameters instead. The inode size is one of these parameters. (From OE-Core rev: eecbe625558406680121d2a7e84917fea45ea9dc) Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: debug mode to keep tmp directoryLee Chee Yang2021-02-091-2/+4
| | | | | | | | | | | | | | | files in wic tmp directory can be usefull for debugging, so do not remove tmp directory when wic create run with debugging mode (-D or --debug). also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to check for tmp directory. [YOCTO#14216] (From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9) Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Optimise fstab modification for ext2/3/4 and msdos partitionsPaul Barker2021-01-202-8/+28
| | | | | | | | | | | | | | | | | | The fix for [Yocto #13994] required the rootfs directory to be copied (using hardlinks if possible) when modifying the fstab file under wic. We can optimise this copy away for filesystems where we have the tools to modify the contents of the partition image after it is created. For ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat filesystems we have the mcopy tool. So for any of these filesystems we skip the modification of the fstab file in the rootfs directory (and skip the associated copy unless it is otherwise necessary) and update the contents of fstab directly in the partition image. (From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Copy rootfs dir if fstab needs updatingPaul Barker2021-01-203-29/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, wic updates the /etc/fstab in the rootfs to include details of additional partitions described in the selected wks file. If this modification is performed in place, other tasks which create an image file from the rootfs directory (e.g. do_image_tar and do_image_ext4) will pick up the modified fstab file which would not be appropriate for those images as they do not include the additional partitions described in the wks file. wic does undo modifications to the fstab file once it has finished creating the filesystem image, however this leaves open a race condition if one of the other tasks reads the contents of the fstab file from the rootfs directory between the point where wic modifies the fstab file and the point where wic restores the files original content. This could be solved by adding a lockfile for tasks which use the rootfs directory to ensure that no other such task is reading the rootfs directory while do_image_wic is running. This would serialize several do_image_* tasks and result in slower builds, especially for large images. Another drawback of this solution is that it is hard to selectively optimise - adding lockfiles to do_image_* tasks would result in these tasks always being serialized even if no fstab modification will take place. An alternative solution is to copy the rootfs directory when fstab needs to be modified. The code to do this in wic already exists as it is needed when including or excluding content in the rootfs. This still results in an impact on build times but the copy uses hardlinks if possible (so little data is actually copied) and we can make selective optimisations to improve things. The rootfs copy will only take place if fstab modification is required (or if it was already needed to include or exclude rootfs content). We can also follow up with further optimisations after this commit. So this second solution is chosen. Fixes [Yocto #13994] (From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Update pseudo db when excluding content from rootfsPaul Barker2021-01-201-5/+8
| | | | | | | | | | | | | | | | | | | | | | | | To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. (From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Ensure internal workdir is not reusedPaul Barker2021-01-201-1/+11
| | | | | | | | | | | | If a path is specified for the internal wic working directory using the -w/--workdir argument then it must not already exist. Re-using a previous workdir could easily result in rootfs and intermediate files from a previous build being added to the current image. (From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Allow exec_native_cmd to run HOSTTOOLSPaul Barker2021-01-131-2/+5
| | | | | | | | | | | This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to be more easily executed by wic. Without this change only programs from an actual *-native recipe built by bitbake can be executed by wic. (From OE-Core rev: 8eb186acdecfbb3151c9a0ab148358e3fe5cce39) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Add workdir argumentPaul Barker2021-01-131-1/+1
| | | | | | | | | | This allows the path for the temporary workdir used by wic to be set when running wic from bitbake or directly from the command line. (From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/direct/kparser: ensure fsuuid for vfat and msdos align with formatLee Chee Yang2021-01-012-0/+24
| | | | | | | | | | | | | | | | | | | | | | vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x" in front follow with 8 hexadecimal number in uppercase. In wic, when using custom fsuuid for vfat/msdos partition in wks, it is able to set the value in any length, with or without leading "0x". This can cause fsuuid missaligned when fstab updates, fstab expect exactly 10 character fsuuid for vfat/msdos partition and all in uppercase. if custom fsuuid for vfat/msdos is set, check the length and format, error if it exceed the format size. Amend it so it is align with format 0xYYYYYYYY. This is done before image create and fstab update to ensure the fsuuid are same in all followup process. if custom fsuuid length less than expected, fill in "0". [YOCTO #14161] (From OE-Core rev: d9686ae511ef10a504becfd81bfe296b788e1456) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHSPeter Kjellerstedt2020-12-201-1/+11
| | | | | | | (From OE-Core rev: 55618cbd58d6784a82e773f323723be6f722232f) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Introduce empty plugin to create unformatted empty partitionsDiego Sueiro2020-12-151-0/+32
| | | | | | | | | | | | | | | | | | The empty wic plugin is used to create unformatted empty partitions for wic images. To use it you must pass "empty" as argument for the "--source" parameter in the wks file. For example: part foo --source empty --ondisk sda --size="1024" --align 1024 Also adds a selftest for this plugin where the 'Fstype' column from 'wic ls' should be empty for the second partition as listed in test_empty_plugin.wks. (From OE-Core rev: 77d174fc80663403ef76c5b808aafc1117d3545c) Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* isoimage-isohybrid.py: Support adding files/dirsChaitanya Vadrevu2020-11-081-0/+14
| | | | | | | | | | Add "payload_dir" parameter so user can add other files/dirs to the ISO. (From OE-Core rev: bbe84a33c81066fee96dd28a4009d8404b51e642) Signed-off-by: Chaitanya Vadrevu <chaitanya.vadrevu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Handle new PSEUDO_IGNORE_PATHS variableRichard Purdie2020-10-071-5/+7
| | | | | | | | | Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid inode corruption issues. (From OE-Core rev: 13500f5234361385c365c7c35e83f99435500481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi ↵Khairul Rohaizzat Jamaluddin2020-09-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | and bootimg-partition Due to recent changes in bootimg-efi to include IMAGE_BOOT_FILES, when both bootimg-partition and bootimg-efi occur in a single .wks and IMAGE_BOOT_FILES are defined, files listed in IMAGE_BOOT_FILES will be duplicated in both partition. Since IMAGE_BOOT_FILES are crucial for bootimg-partition, but optional for bootimg-efi, hence allowing bootimg-efi to have the option to ignore it. The new variable, IMAGE_EFI_BOOT_FILES, was added to help handle this issue. Its basic usage is the same as IMAGE_BOOT_FILES. Usage example: ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \ This commit is also squashed with the updated testcase to cover for this change. [YOCTO #14011] (From OE-Core rev: 945339e06b273df1935cfd784f548ef57e0b7f4c) Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-06-30 22:59:17 +0000