linux/poky.git/bitbake/lib, branch kirkstone-4.0.22

bitbake: hashserv: tests: Omit client in slow server start test

2024-10-10T19:01:06+00:00

On Fedora 39 and Fedora 40 hosts, this version of the hash server exhibits different behavior on exit when using Unix Domain sockets. Instead of closing the client connections and exiting immediately, the server will wait until all clients have disconnected before exiting. It is unknown why this changed, or why it only affects Unix Domain sockets and not TCP sockets. Because of this behavior change, the Slow Server Start test is failing on these hosts. This test is primarily concerned with ensuring that the server will actually exit, even if it gets a termination signal before it enters its main loop, and doesn't really care about clients. As such, modify the test so that a client is not pre-connected to the server. This allows the server to actually exit so that the test can verify the signal behavior. The latest version of the hash equivalence server (on master) does not exhibit this behavior. Speculation is that the more standardized usage of asyncio allows the server to properly exit, even if clients are still connected. Regardless, this patch is only intended for the older versions, and should not be applied to master (Bitbake rev: eb5c1ce6b1b8f33535ff7b9263ec7648044163ea) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: fetch2/wget: Canonicalize DL_DIR paths for wget2 compatibility

2024-10-10T19:01:06+00:00

Some distributions (namely Fedora Core 40) have started replacing wget with wget2. There are some changes to wget2 that make it incompatible with wget: 1. ftp/ftps is not supported anymore 2. progress 'dot' is not yet supported 3. Relative paths in -P and -O are not correctly dealt with Item 1: Is already dealt with since Scarthgap by only adding the option --passive-ftp when the URL specifies ftp/sftp. While that won't help if ftp/sftp is actually required it at least does not break http/https downloads. Item 2: While not supported it at least does not break the operation. Item 3: If there are relative path components in -P or -O then wget2 only deals with them correctly if there is one, and only one, relative path component at the beginning of the path: -P ./downloads works -P ../downloads works -P ../../downloads does not work -P ./../downloads does not work -P /home/user/downloads/../downloads does not work In cases where there are more than one relative path component at the beginning of the path and/or one or more reltaive path component somewhere in the middle or end of the path, wget2 aborts with the message Internal error: Unexpected relative path: '') Such can happen if DL_DIR includes relative path components e.g. DL_DIR = "${TOPDIR}/../../downloads". This patch canonicalizes DL_DIR before it is passed to wget. (Bitbake rev: 07081a94997142746f7d345c27bc6805231d025d) Signed-off-by: Rudolf J Streif Signed-off-by: Richard Purdie (cherry picked from commit 3e4208952b086adc510e78c1c5f9cf4550d79dc9) Signed-off-by: Steve Sakoman (cherry picked from commit 47678142e26bb76d1351886060deff5e75039bc9) Signed-off-by: Steve Sakoman

bitbake: fetch/wget: Move files into place atomically

2024-10-10T19:01:06+00:00

(Bitbake rev: aaf9c1b8673f5b181255a08eee4e9a342863e896) Signed-off-by: Richard Purdie (cherry picked from commit cd7cce4cf4be5c742d29671169354fe84220b47a) Signed-off-by: Steve Sakoman

bitbake: tests/fetch: Tweak to work on Fedora40

2024-10-10T19:01:06+00:00

On Fedora40, "localhost" sometimes resolves to ::1 and sometimes to 127.0.0.1 and python only binds to one of the addresses, leading to test failures. Use 127.0.0.1 explicitly to avoid problems of the name resolution, we're trying to test things other than the host networking. (Bitbake rev: 3147ebb0edcca8f7f850aabd86ef010ab593496f) Signed-off-by: Richard Purdie (cherry picked from commit 9adc6da42618f41bf0d6b558d62b2f3c13bedd61) Signed-off-by: Steve Sakoman

bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftps

2024-10-10T19:01:06+00:00

Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: 477f90f9515cb7cd093920cee5518f662cef2e05) Signed-off-by: Rob Woolley Signed-off-by: Richard Purdie (cherry picked from commit f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Steve Sakoman

bitbake: data_smart: Improve performance for VariableHistory

2024-08-23T21:51:51+00:00

Fixed: - BBMULTICONFIG = "qemux86-64 qemuarm64" and more than 70 layers in BBLAYERS $ bitbake -p -P Check profile.log.processed, the record() cost more than 20 seconds, it is less than 1 second when multiconfig is not enabled, and there would be the following error when more muticonfigs are enabled: Timeout while waiting for a reply from the bitbake server Don't change the type of loginfo['detail'] or re-assign it can make record() back to less than 1 second, this won't affect COW since loginfo is a mutable type. The time mainly affected by two factors: 1) The number of enabled layers, nearly 1 second added per layer when the number is larger than 50. 2) The global var such as USER_CLASSES, about 1 ~ 2 seconds added per layer when the layers number is larger than 50. (Bitbake rev: ec2a99a077da9aa0e99e8b05e0c65dcbd45864b1) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit 0596aa0d5b0e4ed3db11b5bd560f1d3439963a41) Signed-off-by: Steve Sakoman

bitbake: tests/fetch: Tweak test to match upstream repo url change Upstream changed their urls, update our test to match.

2024-06-24T13:53:45+00:00

(Bitbake rev: 734b0ea3dfe45eb16ee60f0c2c388e22af4040e0) Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: parse: Improve/fix cache invalidation via mtime

2024-05-29T12:24:08+00:00

We have been seeing obscure failures in devtool, particularly on newer autobuilder workers where it appears the cache is assumed to be valid when it shouldn't be. We're using the 'seconds' granulation mtime field which is not really a good way of telling if a file has changed. We can switch to the "ns" version which is better however also add in inode number and size as precautions. We already have all this data and tuples are fast so there isn't really any cost to do so. This hopefully fixes [YOCTO #15318]. (Bitbake rev: 5a90927f31c4f9fccbe5d9d07d08e6e69485baa8) Signed-off-by: Richard Purdie (cherry picked from commit d9e5d313c79500e3c70ab9c3239b6b2180194f67) Signed-off-by: Steve Sakoman

bitbake: fetch2: Ensure that git LFS objects are available

2024-03-01T18:00:58+00:00

The current implementation only performs a git lfs fetch alongside of a regular git fetch. This causes issues when the downloaded revision is already part of the fetched repository (e.g. because of moving back in history or the updated revision already being part of the repository at the time of the initial clone). Fix this by explicitly checking whether the required LFS objects are available in the downloade directory before confirming that a downloaded repository is up-to-date. This issue previously went unnoticed as git lfs would silently fetch the missing objects during the `unpack` task. With network isolation turned on, this no longer works, and unpacking fails. (cherry picked from commit cfae1556bf671acec119a6c8bbc4b667a856b9ae) (Bitbake rev: 40fd5f4eef7460ca67f32cfce8e229e67e1ff607) Signed-off-by: Philip Lorenz Signed-off-by: Richard Purdie Signed-off-by: Philip Lorenz Signed-off-by: Steve Sakoman

bitbake: tests/fetch: Add real git lfs tests and decorator

2024-03-01T18:00:58+00:00

Added tests that verify that git-lfs works with an actual real git-lfs server. This was not previously the case because the repo in the test was a simulation of git-lfs but not a real git lfs repo. The 2 added tests are almost the same but test that the git lfs file checkout is successfult with or without the lfs=1 flag. The lfs=1 URI parameter is a quirk that triggers 2 different code paths for git lfs. lfs=1, when used on git lfs repositories triggers the git lfs downloading at the fetch bare stage. lfs query parameter unset triggers the git lfs downloading only on checkout as an implicit behavior of git. This leads to possible network access on the unpack stage and outside the DL_DIR. lfs=0 actually disables git-lfs functionality even if supported. (cherry picked from commit d2be7f7f652360f13cd66d0850f3e19ffe2afb0a) (Bitbake rev: 2a6fd774405a58f3ef2953a0dc3eca063e0cf890) Signed-off-by: Paulo Neves Signed-off-by: Richard Purdie Signed-off-by: Philip Lorenz Signed-off-by: Steve Sakoman