linux/poky.git/bitbake/lib/bb/utils.py, branch thud-next

bitbake: utils: Fix movefile() exception handling with python3

2019-09-06T10:58:56+00:00

* with python3 this fails with: File: 'bitbake/lib/bb/utils.py', lineno: 799, function: movefile 0795: try: 0796: os.rename(src, destpath) 0797: renamefailed = 0 0798: except Exception as e: *** 0799: if e[0] != errno.EXDEV: 0800: # Some random error. 0801: print("movefile: Failed to move", src, "to", dest, e) 0802: return None 0803: # Invalid cross-device-link 'bind' mounted or actually Cross-Device Exception: TypeError: 'OSError' object is not subscriptable (Bitbake rev: 9f92322fa8d6f1a68c0c3f4984afdf65126b51dc) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

bitbake: bitbake: fix version comparison when one of the versions ends in .

2019-06-15T10:06:38+00:00

Previously, this would happen: ====================================================================== ERROR: test_vercmpstring (bb.tests.utils.VerCmpString) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/tests/utils.py", line 45, in test_vercmpstring result = bb.utils.vercmp_string('1.', '1.1') File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 143, in vercmp_string return vercmp(ta, tb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 135, in vercmp r = vercmp_part(va, vb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 124, in vercmp_part elif ca < cb: TypeError: '<' not supported between instances of 'NoneType' and 'int' ---------------------------------------------------------------------- (Bitbake rev: bd953d56d007a8bfa5ecb6e753da4abfb035f9f2) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: utils: lockfile: Fix infinite loop

2018-09-25T22:16:21+00:00

A nasty corner case leads to a hang when utils.lockfile is called from oe-core's package-manager:deploy_dir_lock (in turn called from rootfs:_create further up the call stack) with "name" owned by root and the user running bitbake has no write access. Because this code runs under pseudo, the UID and EUID of the bitbake worker process are 0, so the os.access(dirname, os.W_OK) returns True i.e. it thinks the path is writable when in fact it's not writable. Only later when trying to open the file an Exception it thrown because the OS prohibits writing, but the Exception is ignored and the open is retried leading to an infinite loop. So this fix is to not ignore the "Permission Denied" exception. An alternative fix would be to replace the os.access() call with an try: open() except() at the beginning of the function. (Bitbake rev: 0cb64d0f85b41b2fa764baf6ff7ea1b13f95004e) Signed-off-by: Ioan-Adrian Ratiu Signed-off-by: Richard Purdie

bitbake: bb/utils.py: add break_hardlinks helper

2018-08-16T08:50:42+00:00

bb.utils.copyfile is called in a few places with identical src and dst in order to create an st_nlinks==1 version of the file. That that even works relies on an implementation detail of copyfile (namely, that it creates a temporary file and then does a rename). Moreover, it's a waste of time if the file already has st_nlinks==1. So create a helper that optimizes away the copy in the st_nlinks==1 case. Of course, this helper relies on the same implementation detail, but that's now contained within bb.utils itself. To test that we do at least sometimes hit the no-copy path, I tested locally with if sstat[stat.ST_NLINK] == 1: + bb.note("Woohoo, 2*%d bytes I/O avoided" % sstat[stat.ST_SIZE]) return True (and the obvious places in oe-core patched), and the do_package log files are indeed filled with woohoo notes. (Bitbake rev: 7ae93cf40ab91965147055100432961436bce46c) Signed-off-by: Rasmus Villemoes Signed-off-by: Richard Purdie

bitbake: utils/md5_file: don't iterate line-by-line

2018-08-15T08:45:02+00:00

Opening a file in binary mode and iterating it seems like the simple solution but will still break on newlines, which for binary files isn't really useful as the size of the chunks could be huge or tiny. Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't want to fill up memory: use mmap() to open the file and read the file in 8k blocks. (Bitbake rev: a0ac8d67f1471a0c611d691b856fede67efb53f6) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

bitbake: utils: add optional callback to edit_bblayers_conf()

2018-07-30T11:45:15+00:00

Add a callback that lets you modify or remove items in addition to the current scheme where you can only add or remove. This enables you to for example replace a layer with a temporary copy (which is what we will use this for first in OE's oe-selftest). (Bitbake rev: bfedb4e85a84e817dbe5d8694b8f8fcdd6f2f22a) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

bitbake: lib/bb/utils: prevent movefile from changing dest dir permissions

2018-04-23T16:26:05+00:00

Prevent movefile from falsely setting the source file's owner and permissions on the destination directory instead of the destination file when moving between devices. This bug caused the last file moved into a directory to dictate the directory's owner and permissions. (Bitbake rev: 82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40) Signed-off-by: Mattias Hansson Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

bitbake: utils.py: Add option for explode_dep_versions2 to return unsorted

2018-03-15T10:39:42+00:00

Before, explode_dep_versions2 would sort the OrderedDict before returning. This function will still sort the OrderedDict by default, but will now have the option to return the OrderedDict unsorted. This option will allow us to check if the order of the package list has changed. (Bitbake rev: 39d6a30a28f66c599e18beddbd847f40dcff623c) Signed-off-by: Amanda Brindle Signed-off-by: Richard Purdie

bitbake: bitbake: lib/bb/utils: fix movefile() copy to dir fallback

2017-09-25T13:14:17+00:00

When the destination is a directory, building the the destination file path is always needed. That's because even if the copy fallback is taken, it's always followed by a rename. (Bitbake rev: 14c17480827ced2e03c1b62dc839696421fc4de8) Signed-off-by: Alberto Mardegan Signed-off-by: Richard Purdie

bitbake: lib/bb/utils.py: Add missing debug level

2017-07-27T14:14:20+00:00

(Bitbake rev: a0cd748d2f830a305da086eff3462875f64f2a70) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie