linux/poky.git/meta/classes/sstate.bbclass, branch yocto-3.1.14

sstate: Avoid deploy_source_date_epoch sstate when unneeded

2021-11-15T11:53:55+00:00

This sstate task is only needed when depended upon, it can be skipped if there are no tasks running that directly depend upon it. This reduced the number of sstate tasks in something like an image build. (From OE-Core rev: 884e44701ada57abe4d8ad9ece424435be25c6a5) Signed-off-by: Richard Purdie (cherry picked from commit 601cee016da5c7505915e26641a085714de175ce) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: Ensure SDE is accounted for in package task timestamps

2021-11-15T11:53:55+00:00

When creating packages we build them with --clamp-mtime and use SOURCE_DATE_EPOCH as the maximum mtime. This makes the end packages reproducible. The data stored in sstate for do_package and the package task doesn't benefit from this though and have varying timestamps. This means their outhash varies and means hash equivalance isn't effective at all and doesn't work as intended/desired. We could create the sstate archives with the same clamping however that would lead to different results depending on whether a task was installed from sstate or not. Making that differ is a path to madness. It also wouldn't fix the outhash of the task to be determninistic without clamping of the date in the hash calculation code. Instead, iterate over the files in sstate output and clamp them at the code level. This isn't ideal but does make the file timestamps determnistic everywhere and means we don't have to change the hash calculation code. This issue can be clearly seen looking at the do_package outhash for a recipe which you then re-run the package task for after adding something like whitespace to the install task. The outhash shouldn't change but currently does. (From OE-Core rev: 06b8f2a5a24be1a87f0eaf29fdba719ebe3bb06e) Signed-off-by: Richard Purdie (cherry picked from commit c3b3cc4745811b48b9193f83889946b2e1788932) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: another fix for touching files inside pseudo

2021-11-15T11:53:55+00:00

This patch is a fixup for 676757f "sstate: fix touching files inside pseudo" running the 'id' command inside the sstate_unpack_package function shows that this funcion run inside the pseudo: uid=0(root) gid=0(root) groups=0(root) The check for [ -w ${SSTATE_PKG} ] and [ -O ${SSTATE_PKG}.siginfo ] will always return true and the touch can fail when the real user don't have permission or in readonly filesystem. As the documentation refers: - the file test operator "-w" check if the file has write permission (for the user running the test). - the file test operator "-O" check if you are owner of file We can avoid this test running the touch and mask any return errors that we have. (From OE-Core rev: 29fc85997ade490ae46ffca37ef8e1a56957c876) (From OE-Core rev: e7d94a9cc5ab1b2c5d160fd06d643a4bc3409d26) Signed-off-by: Jose Quaresma Signed-off-by: Richard Purdie (cherry picked from commit 5b9210d66c78bb3f79056e5586cea7b0edd714a9) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: don't silently handle all exceptions in sstate_checkhashes

2021-11-03T11:18:56+00:00

If checkstatus returns an exception we should silently handle FetchError, as this means the fetch failed for 'normal' reasons such as file not found. However, other exceptions may be raised, and these should be made visible. (From OE-Core rev: f16803718a19ace7b582c5b73924fe3112502927) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie (cherry picked from commit 50d99faf88a1d82cbd939b9bd6e33ebed2b1ffd8) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: fix touching files inside pseudo

2021-11-03T11:18:56+00:00

running the 'id' command inside the sstate_create_package function shows that this funcion run inside the pseudo: uid=0(root) gid=0(root) groups=0(root) The check for touch files [ ! -w ${SSTATE_PKG} ] will always return true and the touch can fail when the real user don't have permission or in readonly filesystem. As the documentation refers, the file test operator "-w" check if the file has write permission (for the user running the test). We can avoid this test running the touch and mask any return errors that we have. (From OE-Core rev: 590de1dd89cfd5f0ca7395880ba88b27ee35470d) Signed-off-by: Jose Quaresma Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit f6e7445c94443544e92fda97a017ce93393c5f84) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate.bbclass: fix error handling when sstate mirrors is ro

2021-08-18T17:00:19+00:00

The commit dd555537fc35c5f934af09d601d70772eb5955ae 'sstate.bbclass: fix errors about read-only sstate mirrors' adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The exception handler checks the error type with the python module errno but this module needs to be imported as it don't exist. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 778, function: sstate_task_postfunc 0774: 0775: omask = os.umask(0o002) 0776: if omask != 0o002: 0777: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0778: sstate_package(shared_state, d) 0779: os.umask(omask) 0780: 0781: sstateinst = d.getVar("SSTATE_INSTDIR") 0782: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 708, function: sstate_package 0704: except PermissionError: 0705: pass 0706: except OSError as e: 0707: # Handle read-only file systems gracefully *** 0708: if e.errno != errno.EROFS: 0709: raise e 0710: 0711: return 0712: Exception: NameError: name 'errno' is not defined (From OE-Core rev: ce78c16409363741d59a2f787aca66077bec93cd) Signed-off-by: Jose Quaresma Signed-off-by: Richard Purdie (cherry picked from commit 15f30ad144fbe25e9a5e71bc7e42e746d2039992) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: Fix rebuilds when changing layer config

2021-08-10T10:14:11+00:00

When adding a layer which changed SSTATE_EXCLUDEDEPS_SYSROOT, the state hashes were changing when they should not. This was caused by wider use of setscene_depvalid which means the dependency on the variable was seen when it was previously not. Exclude the variable since this should be be included in the hashes. (From OE-Core rev: bae9c6482271d53dc28d3c801fba467e268003bd) Signed-off-by: Richard Purdie (cherry picked from commit 09725a29365c69ccbd603fe3a1de72189f26d5ac) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate: Drop pseudo exclusion

2021-07-20T18:05:39+00:00

Now that pseudo-native always includes statx support and we have sanity checks on pseudo-native to ensure it always contains a minimum feature set, we no longer need to mark pseudo-native as distro specific. This fixes eSDK build problems. [YOCTO #14428] (From OE-Core rev: d7e87a5851d717da047f552be394d5712efa0402) Signed-off-by: Richard Purdie (cherry picked from commit 3279e43fcb469edb63c7c4eb60fdc565d5751f9d) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate/staging: Handle directory creation race issue

2021-07-10T21:46:19+00:00

The sstate code tries to be careful about racing around directory creation. In particular, the copyhardlinktree code creates the directory tree first allowing for "already exists" errors and ignoring them, then hardlinks the files in. Unfortunately the sstate removal code can race against this since it will try and remove empty directories. If there is some bad timing, a newly created directory can be removed before it was populated, leading to build failures. We could try and add locking but this would damage performance, we've been there before. It is also unclear where to actually place locks just based on the contents of a manifest file which may cover multiple sstate install locations for a given task. Instead, lets disable directory removal in the problematic "shared" core path. This could result in a few more empty directories being left on disk but those should be harmless and better than locking hurting performance or rare build races. [YOCTO #13999] [YOCTO #14379] (From OE-Core rev: fa49622521b6386d8031b1e7519f087aa9d99b19) Signed-off-by: Richard Purdie (cherry picked from commit 4f94d9296394bc7ce241439f00df86eb5912875f) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

sstate.bbclass: fix errors about read-only sstate mirrors

2021-07-10T21:46:19+00:00

If a read-only sstate mirror is used in conjunction with hash equiv, then OSError will be raised when an sstate-cache hit is achieved. This is because sstate_task_postfunc will try to "touch" the symlinks that point to the read-only sstate mirror when sstate_report_unihash has changed SSTATE_PKG. This commit adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The fix is also duplicated to sstate_eventhandler as the code is very similar but it may not be needed there. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function: sstate_task_postfunc 0770: 0771: omask = os.umask(0o002) 0772: if omask != 0o002: 0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0774: sstate_package(shared_state, d) 0775: os.umask(omask) 0776: 0777: sstateinst = d.getVar("SSTATE_INSTDIR") 0778: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package 0699: if not os.path.exists(siginfo): 0700: bb.siggen.dump_this_task(siginfo, d) 0701: else: 0702: try: *** 0703: os.utime(siginfo, None) 0704: except PermissionError: 0705: pass 0706: 0707: return Exception: OSError: [Errno 30] Read-only file system (From OE-Core rev: f2360f796016cce93610a080ff07c8047bda6ee8) Signed-off-by: Michael Ho Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit 244b3be0358a66e0cca4016fe26144e3d7323390) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie