summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/image.py
Commit message (Collapse)AuthorAgeFilesLines
* image: Create separate tasks for rootfs constructionRichard Purdie2016-01-111-412/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch splits the code in lib/oe/image into separate tasks, one per image type. This removes the need for the simple task graph code and defers to the bitbake task management code to handle this instead. This is a good step forward in splitting up the monolithic code and starting to make it more accessible to people. It should also make it easier for people to hook in other tasks and processes into the rootfs code. Incidentally, the reason this code was all combined originally was due to limitations of fakeroot where if you exited the session, you lost permissions data. With pseudo this constraint was removed. We did start to rework the rootfs/image code previously and got so far with untangling it however we did prioritise some performance tweaks over splitting into separate tasks and in hindsight, this was a mistake and should have been done the other way around. That work was suspended due to changes in the people working on the project but this split has always been intended, now is the time to finish it IMO. There were some side effects of doing this: * The symlink for the manifest moves to the rootfs-postcommands class and into the manifest function. * There is no seperate "symlink removal" and "symlink creation", they are merged * The date/time stamps of the manifest and the built images can now be different since the tasks can be run separately and the datetime stamp will then be different between do_rootfs and the do_image_* tasks. (From OE-Core rev: c2dab181c1cdabac3be6197f4b9ea4235cbbc140) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image: Move pre/post process commands to bbclassRichard Purdie2016-01-111-9/+0
| | | | | | | | | | As the next step in splitting up do_image, move the pre and post processing commands to separate tasks. This also creates the do_image_complete task which acts as the end marker task for image generation. (From OE-Core rev: 800528eaa421d451b596545125cb218e08989151) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: avoid mkdir race when building multiple imagesMike Crowe2015-12-011-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If multiple images are being built simultaneously against the same sysroot then the call to os.makedirs in Image._write_wic_env can fail with: File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env 0337: """ 0338: stdir = self.d.getVar('STAGING_DIR_TARGET', True) 0339: outdir = os.path.join(stdir, 'imgdata') 0340: if not os.path.exists(outdir): *** 0341: os.makedirs(outdir) 0342: basename = self.d.getVar('IMAGE_BASENAME', True) 0343: with open(os.path.join(outdir, basename) + '.env', 'w') as envf: 0344: for var in self.d.getVar('WICVARS', True).split(): 0345: value = self.d.getVar(var, True) File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs 0153: if e.errno != errno.EEXIST: 0154: raise 0155: if tail == curdir: # xxx/newdir/. exists if xxx/newdir exists 0156: return *** 0157: mkdir(name, mode) 0158: 0159:def removedirs(name): 0160: """removedirs(path) 0161: Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata' Using bb.utils.mkdirhier() protects against this. There's also little point in checking to see if the directory already exists - we might as well just try and create it regardless. Once the directory has been created, there's no race on the actual file since the filename contains IMAGE_BASENAME. (From OE-Core rev: e3f216eef1b7a8ecfd00317e60704cff472238ba) Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: Avoid creating empty .env file in _write_wic_envMike Crowe2015-12-011-1/+5
| | | | | | | | | | | | Creating a file for every image containing a few variables isn't necessary if wic is not being used, so don't write the file if WICVARS is empty. (From OE-Core rev: d40779a7d3ce4f326c29ec7971731cab1f505a37) Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes: Ensure pass setVar/setVarFlag strings, not integersRichard Purdie2015-11-241-2/+2
| | | | | | | | | | | This doesn't cause any issues right now but it make sense to standardise on consistently using strings in the data store. (From OE-Core rev: 99203fbe5ad470ef65cff93cec9d7f332883b5ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: Fix dependency handling for compressed typesOtavio Salvador2015-10-141-8/+10
| | | | | | | | | | | | | | | | | | | | | The dependency code needs to also include the dependency of base types. For example: - sdcard.gz image with ext4 The dependency chain needs to include: - sdcard - ext4 - gz Until this change, the ext4 dependency were not being taken into account when using the compressed one. (From OE-Core rev: 10e5df3503632a6e1c54612055b19f7258c3ae2f) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: Add image generation for companion debug filesystemMark Hatle2015-10-071-9/+37
| | | | | | | | | | | | The companion debug filesystem, enabled with IMAGE_GEN_DEBUGFS, was creating the companion filesystem but was missing the code to actually package it into a usable filesystem. The code (and associated documentation) will allow the debugfs to generate a companion tarball or other image. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: Ensure base image size is an integerPatrick Williams2015-09-181-0/+2
| | | | | | | | | | | | | | | There is a floating point multiplication done of a base image size and an "overhead factor", which is currently rounded up to the next integer. If the multiplication results in a whole number, the value will still be a float. When this float is used to generate a shell script, a buggy script is generated. Fix this by always forcing to an integer. (From OE-Core rev: bf74a002b8fa18d94cec93f0341cbe74cc010ca7) Signed-off-by: Patrick Williams <patrick@stwcx.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: Allow IMAGE_LINK_NAME to be emptyMike Looijmans2015-09-141-1/+1
| | | | | | | | | | | | When IMAGE_LINK_NAME is empty, OE will try to create a "blank" link instead of just skipping it. The code checks for "link_name is not None" which will never evaluate to true. Change the test to a simple "if link_name:" so it no longer attempt to create links when the variable is an empty string. (From OE-Core rev: 7f03dd50de76f0b5c3a10c514e920615ffaa846c) Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: rename _write_env -> _write_wic_envEd Bartosh2015-09-031-2/+2
| | | | | | | | | | | | | Renamed this function as it's too generic name for it. It writes variables, which are used by wic to .env file, so _write_wic_env is better name for it. Thanks Christopher Larson for poining out to this. (From OE-Core rev: f8c7542164ebbe29613532c93ddc34c94238453c) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: write bitbake variables to .env fileEd Bartosh2015-08-301-0/+18
| | | | | | | | | | | | | | | | Write set of bitbake variables used by wic into build/tmp/sysroots/<machine>/imagedata/<image>.env List of variables is defined in WICVARS variable in meta/classes/image_types.bbclass. This is needed for wic to be able to get bitbake variables without running 'bitbake -e'. (From OE-Core rev: 861ce6c5d4836df1a783be3b01d2de56117c9863) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: add script output to the rootfs logEd Bartosh2015-08-301-1/+4
| | | | | | | | | | | | | | | Let's add output of image creation script to the bitbake log as it can contain useful information. One good example of such an information is wic report about artifacts and .wks file used for image creation. (From OE-Core rev: fd6eba587e39142134aeb9044393c08a3f79d28c) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: set bitbake variable ROOTFS_SIZEEd Bartosh2015-08-301-1/+3
| | | | | | | | | | | | | This variable is going to be used by wic to set partition size. Setting it in image.py makes it possible for wic to use it without calculating it again. (From OE-Core rev: af37bb8ae71c4f932e2126bb620b3fb2b11cc466) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* IMAGES_FSTYPES: default to EXT4Juro Bystricky2015-08-161-2/+2
| | | | | | | | | | | | | | | | | The following IMAGES_FSTYPES defaulted to ext3: "vmdk", "vdi", "qcow2", "live", "iso", "hddimg" This patch changes the default for those IMAGES_FSTYPES to ext4 in order to bring the images more in line with other BSPs. Besides improvements in performance and reliability ext4 provides additional functionality as well (option to turn off the journaling, dynamic resizing of VDI volumes etc.). (From OE-Core rev: 2b56d671d2f0ef22786c97e29e1215eb80c94490) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image: Support for VDIJuro Bystricky2015-06-111-2/+2
| | | | | | | | | | | | | Added support for VirtualBox VDI format. The support was implemented by merging with the already existing VMDK support for VM player by creating a new class image-vm.bbclass. This class replaces the previous VMDK only image-vmdk.class. (From OE-Core rev: 0a3e8eb9f592c3f1edd2c7521855f7406541651a) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: add error checking for missing IMAGE_CMDPaul Eggleton2015-02-031-1/+5
| | | | | | | | | | An invalid value in IMAGE_FSTYPES was triggering a traceback. Add a check and a reasonable error message instead. (From OE-Core rev: f38c35b925d6eab0d5a41b8592b0eb8b0a3b7b72) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: Handle compressed IMAGE_TYPEDEP valuesOtavio Salvador2014-12-251-2/+16
| | | | | | | | | | | | When computing the dependency graph for the image generation, we need to take into account the compression type and identify the base type it relates to. This allow for a more robust graph generation even when using composed image types. (From OE-Core rev: 6e7d1de6cc99ed2def346dc40310573f5f0ce5ca) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: Fix error in graph sortingPascal Bach2014-10-301-1/+1
| | | | | | | | | | | | | The graph sorting algorithm for image dependencies does a look for an occurrence of a searched string instead of comparing the chunk to the searched string. This leads to the problem that ubifs is recognized as ubi aswell. This fixes this by splitting up the string into chunks. (From OE-Core rev: cec9725c540c2d54c27092e40d159694cea75b5f) Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: check the rootfs size against IMAGE_ROOTFS_MAXSIZERobert Yang2014-07-101-0/+8
| | | | | | | | | | | | | * Check the rootfs size against IMAGE_ROOTFS_MAXSIZE (if set) * Add comments for IMAGE_ROOTFS_SIZE to not confuse with IMAGE_ROOTFS_MAXSIZE [YOCTO #2610] (From OE-Core rev: 6acd4fc8d5e642b5c6c75fcc40dd8f37caf7ddcf) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py: check file exists before deletingLaurentiu Palcu2014-03-301-1/+2
| | | | | | | | | | | | When RM_OLD_IMAGE = "1", we delete old images but we didn't check they actually exist... [YOCTO #6029] (From OE-Core rev: 8910d3cc94899ab4d509e681b438ae96218fa777) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py, rootfs.py, package_manager.py: redirect stderr to stdout when ↵Laurentiu Palcu2014-03-071-1/+1
| | | | | | | | | | | | | | | | | | | calling check_output() If a command executed with subprocess.check_output() fails, the subprocess.CalledProcessError.output contains only STDOUT and the user needs to check the log.do_rootfs to see any other details. This commit forwards stderr to stdout so that, in case of failure, the entire error output will be displayed in terminal. [YOCTO #5902] (From OE-Core rev: 4661f1441429952f30e19cebd93dc42ce31fd868) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: add image dependency mechanismLaurentiu Palcu2014-02-201-91/+191
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit adds a dependency mechanism to image creation, so that we can split the images creation execution in groups, that can be executed in parallel, having the dependencies satisfied in the same time. The old code didn't need this since everything was serialized. Technically, it adds a dependency graph topological sort class that the main Image class can use to sort out the dependencies. Images that have dependencies have to declare them using the NEW IMAGE_TYPEDEP variable, like in the example below: For: IMAGE_FSTYPES = "i1 i2 i3 i4 i5" IMAGE_TYPEDEP_i4 = "i2" IMAGE_TYPEDEP_i5 = "i6 i4" IMAGE_TYPEDEP_i6 = "i7" IMAGE_TYPEDEP_i7 = "i2" We'll get the following image groups, sorted out by their dependencies: [['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']] The algorithm can probably be optimized but, given the small size of the graphs, it'll do. [YOCTO #5830] (From OE-Core rev: db9dd4b4ef9120baccbccae77d9c31f54a6eb9a1) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "lib/oe/image.py: fix working directory"Richard Purdie2014-02-181-1/+1
| | | | | | | | | | | This reverts commit 3f49597225a58965124503ca5f3cc4011b04b3c0. This change appears to cause more problems than it fixes since the compression commands usually work in the deploy dir but the archive ones have always worked in the rootfs dir (which is clear from the tar command we use). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: fix working directoryJonathan Liu2014-02-171-1/+1
| | | | | | | | | | The working directory needs to be changed before the image creation commands instead of afterwards. (From OE-Core rev: 9d9bca8785911e8ae06d507bbfb99d6a811f072e) Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image_types: sum.jffs2 is replaced by jffs2.sumDmitry Eremin-Solenikov2014-02-141-3/+0
| | | | | | | | | | Previous commit added support for sum 'compression' (rather postprocessing) of jffs2 images. Drop support for sum.jffs2 image type. (From OE-Core rev: 295c6c1452c8789a8b1d0090ff09a5cd7cf16507) Signed-off-by: Dmitry Eremin-Solenikov <dmitry_eremin@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image.py, package_manager.py, rootfs.py: dump command output on errorLaurentiu Palcu2014-02-111-2/+2
| | | | | | | | | Print the entire command output in case of errors. (From OE-Core rev: 2253c9ac2caa61dee0bd4fea04d4d77b79be7b36) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: fix image size calculation routineLaurentiu Palcu2014-02-111-2/+2
| | | | | | | | | | The IMAGE_ROOTFS_EXTRA_SPACE value was not correctly added to the base size. (From OE-Core rev: f3ac2a68ae0ae9bdf2a03c1b6d0f83c9918d58bd) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: fix get rootfs_extra_space failedHongxu Jia2014-02-111-1/+1
| | | | | | | | | | | The value of IMAGE_ROOTFS_EXTRA_SPACE is "0 + 51200", we should use eval rather than int in python. (From OE-Core rev: dd3418e2db732ca1bc78fd93efa3f08da88a1183) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/image.py: add new image creation libraryLaurentiu Palcu2014-02-111-0/+239
This will replace the old bash image creation code. This needs the rootfs to be already generated in order to work. Usage: Image(d).create() or using the provided wrapper function: create_image(d) (From OE-Core rev: b75b78ce534fbf0d4de2f7f66af5b721d68b7471) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-27 09:29:26 +0000