linux/poky.git/meta/classes/sstate.bbclass, branch uninative-2.10

uninative: Don't use single sstate for pseudo-native

2020-11-24T15:53:07+00:00

pseudo-native is a bit special. It conditionally compiles in support for xattr, statx and statvfs amongst other options. If a pseudo-native binary is used on a system where these functions are present but it wasn't compiled in we see hard to debug permissions problems. An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target oe-selftest which shows a cryptic error: File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target self.assertEqual(filelist1, filelist2) File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual self.assertSequenceEqual(list1, list2, msg, seq_type=list) File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual self.fail(msg) File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail raise self.failureException(msg) AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] First differing element 0: '-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor' '-rwxr-xr-x root root /etc/init.d/mdmonitor' This is due to a version of pseudo without statx being used on a system where ls uses statx, hence the files are displayed as 6000.6000 instead of root.root. Avoid this by always building pseudo-native for the specific distro in question rather than using a universal sstate feed. This hopefully fixes one of the mysterious AB-INT issues. (From OE-Core rev: 6e3785a3f1f3cf68f5fe101cd6bebe91db165973) Signed-off-by: Richard Purdie

sstate: set mode explicitly when creating directories in sstate-cache

2020-09-30T14:01:51+00:00

When creating directories in the sstate-cache, explicitly set the mode passed to mkdir to 0775 so that the directories are group writable, as otherwise they cannot be shared with other users. (From OE-Core rev: 1b0624dffe1b8496533c86dfed873112c8c0a01b) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

sstate.bbclass: Check file ownership before doing 'touch -a'

2020-09-28T13:15:48+00:00

In contrast to 'touching' a file without parameters 'touch -a' can only be performed by the file owner. In case of a shared sstate-cache served locally (e.g. over NFS), where the user running bitbake is not the owner of sstate-cache files, even if he has write access on group level, the sstate_unpack_package task will fail. Checking if the file is owned by the user before attempting to run 'touch -a' on it solves this. (From OE-Core rev: f1554e8bd40c640fd58daac18ecf7b458e754109) Signed-off-by: Norman Stetter Signed-off-by: Richard Purdie

sstate.bbclass: When siginfo or sig files are missing, stop fetcher errors

2020-05-30T11:32:48+00:00

Prior to fetching, the system checks if the sstate file is present either locally or on the mirror. If it is, then it goes to the fetch stage. Up to three files can be fetched, sstate, sstate.siginfo and sstate.sig (if signature validation is enabled). The previous pstaging_fetch function would iterate over these, and if a download error occurred would spew forth a great amount of fetcher failure messages as well as stop fetching the next item in the set. This was resolved by adding a fetcher.checkstatus() call prior to the download. If the file isn't present, then the exception will be triggered, and no fetcher failure messages will reach the user. The exception handler is then modified to be a pass so that it will loop and pull the rest of the files that that are requested. Additionally, a check for the existance of the .sig file was added to the sstate_installpkg to avoid an error trying to load the .sig if it wasn't downloaded. (From OE-Core rev: a9085140434e2d26c0bb75bb53fcb7f7c19ef86d) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

sstate.bbclass: Do not fail if files cannot be touched

2020-05-09T17:57:21+00:00

It may be that a file is not allowed to be touched, e.g., if it is a symbolic link into a global sstate cache served over NFS. (From OE-Core rev: 462e3f505217c9b9e0d2c1cbdafb0c8910bb6a59) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie

sstate: Drop obsolete check in hash validation

2020-03-06T17:14:04+00:00

Now this function has a summary parameter we can drop this check. It could well be why the mysterious "locked sigs" selftest fails intermittently if this function were called with a single hash to check. [YOCTO #13605] (with luck) (From OE-Core rev: 02aabe0e59f73bf206d9bada1e7089832ceed254) Signed-off-by: Richard Purdie

sstate.bbclass: fix issue while handling long sstate filenames

2020-02-08T13:20:02+00:00

When moving to python3, divison using '/' now returns float instead of an integer. In upstream commit b8025e972081b70960ffcbcbe43a7118041556a1 sstate filenames longer than the limit are changed to just include necessary info + 3 fields just for information. The space left over after the necessary info is divided into 3 for each of the fields. Using '//' instead to do the division to solve the following error message: avail = (254 - len(hash + "_" + taskname + extension) - len(components[0]) - len(components[1]) - len(components[5]) - len(components[6]) - 7) / 3 > components[2] = components[2][:avail] components[3] = components[3][:avail] TypeError: slice indices must be integers or None or have an __index__ method (From OE-Core rev: 2acfee61a062c6520a413b2a797544d968bb0c76) Signed-off-by: Jaewon Lee Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

sstate.bbclass: Only show sstate mirror progress bar for >= 100 objects

2020-01-27T16:48:08+00:00

With the introduction of the hash equivalence server, the progress bar for "Checking sstate mirror object availability" is shown repeatedly. Most of the times the number of objects scanned is very low and the progress bar completes almost immediately. To avoid all these unnecessary progress bars, set the minimum number of objects to 100 before the progress bar is shown. (From OE-Core rev: 114340e8d393cf60d0a596cb0800cf1b7fd198df) Signed-off-by: Peter Kjellerstedt Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

sstate: Improve move into place op and touch file

2020-01-13T13:07:01+00:00

Depending on the kernel and coreutils version, mv operations test for existence of files and can potentially race. It also leads to the file always changing which leads to potential problems if using and NFS share and there are other readers. Using ln instead means we don't overwrite the file if it already exists meaning other readers aren't disrupted and should work more reliably on NFS which is used for sstate on the autobuilder. Since we're not overwriting files, touch the file to show activity as would have been done it it were reused from sstate. (From OE-Core rev: d8e9a22a4e23616ad01627c1e472296b1e26f13c) Signed-off-by: Richard Purdie

sstate: Avoid races over rewritten siginfo files

2020-01-06T23:54:39+00:00

Try and avoid errors like: Exception: bb.process.ExecutionError: Execution of 'build/tmp/work/x86_64-linux/libxslt-native/1.1.34-r0/temp/run.sstate_unpack_package.12343' failed with exit code 1: touch: setting times of 'build/sstate_devtool/80/4d/sstate:libxslt-native::1.1.34:r0::3:804d33f3f8d032b01ae92207669dd0a8f95a84917d563d122a77df19e786d73c_populate_lic.tgz.siginfo': Stale file handle WARNING: exit code 1 from a shell command. by not overwriting existing files. (From OE-Core rev: b2a389ad5111d587db3f95c6ce13fc2eafc22f27) Signed-off-by: Richard Purdie