linux/poky.git/meta/classes/sstate.bbclass, branch hardknott-3.3.4

sstate: fix touching files inside pseudo

2021-11-03T11:19:04+00:00

running the 'id' command inside the sstate_create_package function shows that this funcion run inside the pseudo: uid=0(root) gid=0(root) groups=0(root) The check for touch files [ ! -w ${SSTATE_PKG} ] will always return true and the touch can fail when the real user don't have permission or in readonly filesystem. As the documentation refers, the file test operator "-w" check if the file has write permission (for the user running the test). We can avoid this test running the touch and mask any return errors that we have. (From OE-Core rev: 8acd5626b5b89fb48d5d7f0e4eee263793ba383f) Signed-off-by: Jose Quaresma Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit f6e7445c94443544e92fda97a017ce93393c5f84) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate: Avoid problems with recipes using SRCPV when fetching sstate

2021-09-24T09:17:32+00:00

When fetching sstate, SRCPV can be expanded withuin PV which breaks the sstate fetcher as other variables like SRC_URI are changed by the fetcher code. Expand the variable before doing this to avoid those problems and hence avoid setscene task failures with remote sstate storage in git recipes. (From OE-Core rev: 5232aa1c46380e7197098bed39fc6d5b8456caa7) Signed-off-by: Richard Purdie (cherry picked from commit 0d9d7acebe0be42d4cf8a30ecbbbbcb2dfd16a4b) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate.bbclass: fix error handling when sstate mirrors is ro

2021-08-26T07:32:13+00:00

The commit dd555537fc35c5f934af09d601d70772eb5955ae 'sstate.bbclass: fix errors about read-only sstate mirrors' adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The exception handler checks the error type with the python module errno but this module needs to be imported as it don't exist. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 778, function: sstate_task_postfunc 0774: 0775: omask = os.umask(0o002) 0776: if omask != 0o002: 0777: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0778: sstate_package(shared_state, d) 0779: os.umask(omask) 0780: 0781: sstateinst = d.getVar("SSTATE_INSTDIR") 0782: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 708, function: sstate_package 0704: except PermissionError: 0705: pass 0706: except OSError as e: 0707: # Handle read-only file systems gracefully *** 0708: if e.errno != errno.EROFS: 0709: raise e 0710: 0711: return 0712: Exception: NameError: name 'errno' is not defined (From OE-Core rev: 2e5cf13aa46f25d8976ece55fcb4129bfd75659f) Signed-off-by: Jose Quaresma Signed-off-by: Richard Purdie (cherry picked from commit 15f30ad144fbe25e9a5e71bc7e42e746d2039992) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate: Fix rebuilds when changing layer config

2021-08-14T11:01:44+00:00

When adding a layer which changed SSTATE_EXCLUDEDEPS_SYSROOT, the state hashes were changing when they should not. This was caused by wider use of setscene_depvalid which means the dependency on the variable was seen when it was previously not. Exclude the variable since this should be be included in the hashes. (From OE-Core rev: 084d81058682e0c9074203c907f80d6eabe63534) Signed-off-by: Richard Purdie (cherry picked from commit 09725a29365c69ccbd603fe3a1de72189f26d5ac) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate: Drop pseudo exclusion

2021-08-06T08:59:14+00:00

Now that pseudo-native always includes statx support and we have sanity checks on pseudo-native to ensure it always contains a minimum feature set, we no longer need to mark pseudo-native as distro specific. This fixes eSDK build problems. [YOCTO #14428] (From OE-Core rev: 4c7c9326da92ea93fdef1ad804685c21a197f1d4) Signed-off-by: Richard Purdie (cherry picked from commit 3279e43fcb469edb63c7c4eb60fdc565d5751f9d) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate/staging: Handle directory creation race issue

2021-07-20T18:05:45+00:00

The sstate code tries to be careful about racing around directory creation. In particular, the copyhardlinktree code creates the directory tree first allowing for "already exists" errors and ignoring them, then hardlinks the files in. Unfortunately the sstate removal code can race against this since it will try and remove empty directories. If there is some bad timing, a newly created directory can be removed before it was populated, leading to build failures. We could try and add locking but this would damage performance, we've been there before. It is also unclear where to actually place locks just based on the contents of a manifest file which may cover multiple sstate install locations for a given task. Instead, lets disable directory removal in the problematic "shared" core path. This could result in a few more empty directories being left on disk but those should be harmless and better than locking hurting performance or rare build races. [YOCTO #13999] [YOCTO #14379] (From OE-Core rev: 3bea0da687d28c80f0b63e2ad81573a3601bb482) Signed-off-by: Richard Purdie (cherry picked from commit 4f94d9296394bc7ce241439f00df86eb5912875f) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate.bbclass: fix errors about read-only sstate mirrors

2021-07-06T12:02:30+00:00

If a read-only sstate mirror is used in conjunction with hash equiv, then OSError will be raised when an sstate-cache hit is achieved. This is because sstate_task_postfunc will try to "touch" the symlinks that point to the read-only sstate mirror when sstate_report_unihash has changed SSTATE_PKG. This commit adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The fix is also duplicated to sstate_eventhandler as the code is very similar but it may not be needed there. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function: sstate_task_postfunc 0770: 0771: omask = os.umask(0o002) 0772: if omask != 0o002: 0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0774: sstate_package(shared_state, d) 0775: os.umask(omask) 0776: 0777: sstateinst = d.getVar("SSTATE_INSTDIR") 0778: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package 0699: if not os.path.exists(siginfo): 0700: bb.siggen.dump_this_task(siginfo, d) 0701: else: 0702: try: *** 0703: os.utime(siginfo, None) 0704: except PermissionError: 0705: pass 0706: 0707: return Exception: OSError: [Errno 30] Read-only file system (From OE-Core rev: 1d86f3c8f5d47e1b3e3ca759076e24710ee7dc02) Signed-off-by: Michael Ho Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit 244b3be0358a66e0cca4016fe26144e3d7323390) Signed-off-by: Anuj Mittal Signed-off-by: Richard Purdie

sstate: Ignore sstate signing key

2021-05-30T07:19:35+00:00

What key is used to sign sstate artefacts should not affect the hash of the object, otherwise everyone would need to use the same signing key. (From OE-Core rev: 01a9358abe821c1da06c3243ccbcc93348042937) Signed-off-by: Daniel McGregor Signed-off-by: Richard Purdie

sstate: Handle manifest 'corruption' issue

2021-05-22T09:03:10+00:00

Under certain build patterns, warnings about missing manifests can appear. These are real issues where the manifest was removed and shouldn't have been. Martin Jansa was able to find a reproducer of: MACHINE=qemux86 bitbake zlib-native echo 'PR = "r1"' >> meta/recipes-core/zlib/zlib_1.2.11.bb MACHINE=qemux86-64 bitbake zlib-native MACHINE=qemux86 bitbake zlib-native The code maintains a per machine list of stamps but a per PACAGE_ARCH list of stamp/manifest/workdir mappings. The latter is only appended to for speed with the assumption that once stamps are gone, the code wouldn't trigger. The code only ever appends to the mapping list (for speed/efficency under lock) meaning that multiple entries can result where the stamp/workdir differs due to version changes but the manifest remains the same. By switching MACHINE part way through the build, the older stamp is referenced and the manifest is incorrectly removed as it matches an now obsolete entry in the mapping file. There are two possible fixes, one is to rewrite the mapping file every time which means adding regexs, iterating and generally complicating that code. The second option is to only use the last mapping entry in the file for a given manifest and ignore any earlier ones. This patch implments the latter. Also drop the stale entries if we are rewriting it. (From OE-Core rev: fe468802f697d0be41cf3407df2460e1473e35f8) Signed-off-by: Richard Purdie

sstate: Add documentation for eventhandlers and tweak naming

2021-03-23T22:51:25+00:00

It isn't immediately obvious what these do, add some better examples to document them. Also improve the naming for one of the handlers. This will help make it clear in any tracebacks which code is being used. (From OE-Core rev: 2b5c553744804f44a17b61c345e482027e45b71e) Signed-off-by: Richard Purdie