summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: utils: Update to use exec_module() instead of load_module()Richard Purdie2022-01-111-2/+5
| | | | | | | | | | | | This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. (Bitbake rev: c039182c79e2ccc54fff5d7f4f266340014ca6e0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 68a18fbcb5959e334cf307d7fa8dc63832edb942) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Drop gnu urls from wget connectivity testRichard Purdie2022-01-051-3/+0
| | | | | | | | | | | | These urls are no longer adding much to the test coverage but the intermittent network issues connecting to them are painful. Drop the urls. (Bitbake rev: 92a16b425fee5f0c46122eb126ecd8b381cf1ec5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bdf5739c5d831dc97a7d81568f94a0953c71017f) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: npm: Use temporary file for empty user configStefan Herbrechtsmeier2022-01-051-9/+5
| | | | | | | | | | | | | | | Always use a temporary file for the user config 'NPM_CONFIG_USERCONFIG' because npm otherwise failed if configs and npmrc aren't set: double-loading config "/dev/null" as "global", previously loaded as "user" (Bitbake rev: 56f6e7b5f86f1dc630c50a67e9027c1798a56a34) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9f272ad7f76c1559e745e9af686d0a529f917659) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: npm: Quote destdir in run chmod commandStefan Herbrechtsmeier2022-01-051-1/+1
| | | | | | | | | | | | | | Quote destdir in run chmod command to support special characters in package name and to avoid syntax error for packages like '@(._.)/execute'. (Bitbake rev: e5b8983bec98b33dab2706575b9e7763c102f720) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a701dfce3f0e74b4d7c687eeda83fe9c8e7240b1) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Do not mix stderr with stdoutAnton Mikanovich2022-01-051-1/+1
| | | | | | | | | | | | | We should not redirect stderr to stdout if we need to get separated stdout and stderr contents from Popen.communicate() later. (Bitbake rev: eef2355331e7e03b2c7615695694c5ba9877fb36) Signed-off-by: Anton Mikanovich <amikan@ilbers.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1ecc1d9424877df89fcda2f23c306998998a65ff) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/pyinotify.py: Remove deprecated module asyncoreRobert Yang2021-12-141-30/+0
| | | | | | | | | | | | | | | | When build with nativesdk-python3 (3.10) from buildtools: /path/to/bitbake/lib/pyinotify.py:55: DeprecationWarning: The asyncore module is deprecated. The recommended replacement is asyncio The pyinotify.py's upstream didn't have any update in recent 7 years: https://github.com/seb-m/pyinotify And bitbake doesn't use the asyncore module, so remove the related code. (Bitbake rev: 6bfb02f845001bf4f05d9b68695bb616b1642b2a) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: Skip old override syntax checking for anonymous ↵Robert Yang2021-12-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | functions Fixed when oe-core's path contians append/prepend/remove, e.g.: /path/to/append_test/oe-core/ Initial a build in any build dirs: $ bitbake -p ERROR: Variable __anon_32__buildarea2_xhou_builds_append_test_layers_oe_core_meta_classes_patch_bbclass contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake. The anonymous fuctions has no names, so skip checking for it to fix the issue. (Bitbake rev: 5c46ce77aa55cb46abe224b6a31dad87b9249bcf) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ebd00330c41c75797529ff38d6a0955b93f05d1b) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix url remap issue and add testcaseRichard Purdie2021-11-242-1/+2
| | | | | | | | | | | | | Using "" as a target for .replace() is a really bad idea as it duplicates the replacement for every character in the string. Add a testcase which triggered this and correct the code to return the correct result. (Bitbake rev: 44a83b373e1fc34c93cd4a6c6cf8b73b230c1520) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3af1ecf049d2eed56f6d319dc7df6eb4a3d4eebc) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Handle lockfile filenames that are too long for filesystemsRichard Purdie2021-11-241-1/+5
| | | | | | | | | | | | | | | | The fetcher mirror code can go crazy creating lock filenames which exceed the filesystem limits. When this happens, the code will loop/hang. Handle the filename too long exception correctly but also truncate lockfile lengths to under 256 since the worst case situation is lockfile overlap and lack of parallelism. (Bitbake rev: 64498ecb094b7911d10b07c098d5a966e79f95b3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 63baf3440b16e41ac6601de21ced94a94bdf1509) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Fix task-depends.dot for multiconfig targetsRichard Purdie2021-11-211-1/+3
| | | | | | | | | | | | | | | The right hand side of dependencies in the task dependency file generated by bitbake -g was missing multiconfig prefixes, corrupting the data. Fix this. [YOCTO #14621] (Bitbake rev: c1938abf51b57938a21948bb414ad0467e4368d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1d5ca721040c5e39aefa11219f62710de6587701) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle parsing results queue raceRichard Purdie2021-11-211-5/+9
| | | | | | | | | | | | | The previous fix introduced a race where the queue might not be empty but all the parser processes have exited. Handle this correctly to avoid occasional errors. (Bitbake rev: 8eaddb92a5fd14de6b5995aa92a6eed03b90a252) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8e7f2b6500e26610f52d128b48ca0a09bf6fb2cb) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Remove debug code, oops :(Richard Purdie2021-11-211-1/+0
| | | | | | | | | (Bitbake rev: ae1bfbf9523e8f6155bb43ee3adba17af3ec9630) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 19291665fa8b6cc331290f2542af3e8e653203f1) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle parse threads disappearing to avoid hangsRichard Purdie2021-11-211-1/+10
| | | | | | | | | | | | | If one of the parse threads disappears during parsing for some reason, bitbake currently hangs. Avoid this (and zombie threads hanging around) by joining() threads which have exited. (Bitbake rev: 920111a330be59e5be2068a8f1a9edcbc6c14402) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dc86a533d951d13643ce446533370da804782afc) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix runall option handlingRichard Purdie2021-11-121-0/+1
| | | | | | | | | | | | | The previous fix for runall option handling had a small bug in it, it didn't clear the originally processed task list which meant it was running too many tasks. Fix this so the list is reset and rebuild correctly. (Bitbake rev: 693eec8edf8d3b2b01c53be6776213cccd797485) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 87c9e120897ed04dfc64d4752fc602f9bfcb8645) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix runall option task deletion ordering issueRichard Purdie2021-11-121-19/+16
| | | | | | | | | | | | | | | | | | | The runbuild option handling in runqueue was flawed as items deleted from the main task list may be dependencies and hence cause index errors. Rather than modify runtaskentries straight away, compute a new shorted list and use that as an input to the second phase. This avoids the need to add tasks back to the list meaning delcount can be simplifed to a simple counter. The second use case in runonly doen't re-add items so doesn't have this issue. (Bitbake rev: cc2e9c4800a8dfde24b3b5fa7184d0bb6398d4fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3428e3c54eb5cc03ff96f9cee6dc839afee7a419) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update pcre.org address after github changesRichard Purdie2021-11-121-1/+1
| | | | | | | | | | | | | vcs.pcre.org was a redirect to github which we use for subversion testing. With the protocol changes at github and the removal of the redirect, use a direct address for github. (Bitbake rev: 85eb90edb4b912b4befb10128d60d342d0525eb3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6230ca71eb7eb2a6db162e28a01727d00af5299b) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: check if upstream hash equivalence server is availableJose Quaresma2021-11-121-1/+11
| | | | | | | | | | | | | When the user specify an invalid upstream hash equivalence server in BB_HASHSERVE_UPSTREAM notify the user that we can't connect the server. (Bitbake rev: 7561fdc23f1aff370ead2abc5747c3a1c8b4ae4d) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit be45aeb9a84f30c28711e87e2d2a4a86320a8d94) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: Handle mirror user/password replacements correctlyRichard Purdie2021-11-122-0/+8
| | | | | | | | | | | | | | Username or password replacements in URIs were being appended rather than replaced in mirror url remapping. Fix this and add a test case. [YOCTO #13823] (Bitbake rev: 85e7af227a48faec65838dcb7e73b17344bb2a0d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 66ad58bb87e5158aced572be4f1d5726bc97fcce) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update github urlsRichard Purdie2021-11-121-13/+13
| | | | | | | | | (Bitbake rev: 5e9bb32f229d4beebf11b880841edd5a7417bb70) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 07fca7e3ab696ba985b3ef86ab9031d688bf2df2) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: Add timeout for checkstatus calls (30s)Richard Purdie2021-11-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | We had an issue where a webserver serving sstate had filesystem issues so would accept connections but effectively not do anything with them. This causes bitbake to hang whilst processing things like sstate objects inside the checkstatus() calls. It can be replicated by setting up a server like: socat -u TCP4-LISTEN:NNN,fork OPEN:/dev/null and pointing SSTATE_MIRRORS in OE at that address. Adding a timeout to the checkstatus calls of 30s means that whilst the system will pause, it will then continue and not hang entirely. Since there isn't a large transfer here, 30s should be a reasonable response time after which we should fall back to building things ourselves. [YOCTO #13716] (Bitbake rev: ba97caa58efe25bb62d2378fa52d21b6a6aa446c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "parse/ast: Show errors for append/prepend/remove operators ↵Richard Purdie2021-11-081-4/+0
| | | | | | | | | | | | combined with +=/.=" This reverts commit ae2b34285f8b3a1a3067c5e9b5d29e32e68c75f1. Accidentally applied to the wrong branch. (Bitbake rev: 1ac73638c1504cf2aa7f13257396aad617f25e8f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: Show errors for append/prepend/remove operators combined ↵Richard Purdie2021-11-081-0/+4
| | | | | | | | | | | | with +=/.= Operations like XXX:append += "YYY" are almost always wrong and this is a common mistake made in the metadata. Show warnings for these usages with a view to making it a fatal error eventually. (Bitbake rev: ae2b34285f8b3a1a3067c5e9b5d29e32e68c75f1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix downloadfilename issue with premirrorChen Qi2021-11-031-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following commit to fix [Yocto #13039] causes regression of the behavior of PREMIRRORS. "bitbake: fetch2: fix premirror URI when downloadfilename defined" Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb as an example. SRC_URI = "\ http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \ ... " With the above commit, it now tries to fetch 1.4.0.tar.gz instead of freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz. The commit above tries to avoid fetching from invalid url such as: https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz. And its solution is to make basename to be 1.4.0.tar.gz, thus causing the regression. This patch fixes the above regression. For Yocto #13039, it now tries to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz. (Bitbake rev: 78949cf3fd31d8a408e93af7e27bcf26ae7942f4) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 96c30007dc0b32eee2b15771daec7948bc9bfd97) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: add test case to ensure downloadfilename is used ↵Chen Qi2021-11-031-0/+8
| | | | | | | | | | | | | | | | | | | | for premirror Add a test case test_fetch_premirror_use_downloadfilename_to_fetch to ensure that 'downloadfilename' is used when fetching from premirror. Although the other two previous test cases, test_fetch_premirror_specify_downloadfilename_regex_uri and test_fetch_premirror_specify_downloadfilename_specific_uri already implicitly contain such verification, we still need to add a very clear case to ensure no regression. (Bitbake rev: 057cbba6b7ade134e4fa3584b9e896be025a6f46) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 20aabc3d53f69949810ecf02295725db947ffef8) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch.py: fix premirror test casesChen Qi2021-11-031-4/+4
| | | | | | | | | | | | | When downloadfilename is specified, it is used to fetch from premirror. So fix the test cases accordingly. (Bitbake rev: af573273e4a5b73550af9639da18906f13bfa1a9) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3b4d2e3b5024324058360a2a28f33c34114218d0) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Handle github dropping git:// supportRichard Purdie2021-11-031-0/+4
| | | | | | | | | github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: f19eefdaa5b43460f00d79d002f96112a6aa3c9a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Add debug when unpickle failsRichard Purdie2021-10-261-1/+5
| | | | | | | | | | | | | | We occasionally see bitbake-worker failing and from the logs, an unpickle error occurs. Add more debug so we can further debug this next time it fails. [YOCTO #14595] (Bitbake rev: 692fa35f4c23722f3179502cb965960cc230e709) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fe8105cc06beca8240b76ea366a1eff5aa9c5412) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Ensure hashserv exits before deleting filesRichard Purdie2021-10-261-1/+1
| | | | | | | | | | | | | | We've seen races where the socket may be gone but the server is still writing out it's database. Handle that case too to avoid cleanup tracebacks. [YOCTO #14440] (Bitbake rev: 36b1b4c4fcee9dde628c7113203939730ab12ae5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b9e4fb843cb9d3a4d4404af093a781fab5520465) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/perforce: Fix typoRichard Purdie2021-10-261-1/+1
| | | | | | (Bitbake rev: 20eae05fdd6cb7ace87ad005f72c256e2fddb3d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: Update urls to match upstream branch name changesRichard Purdie2021-10-201-3/+3
| | | | | | (Bitbake rev: 036ad517921a68525a9b2564363b01332d668e4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 1.52.0Richard Purdie2021-10-112-2/+2
| | | | | | (Bitbake rev: c78ebac71ec976fdf27ea24767057882870f5c60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Improve behaviour for better determinism/sstate reuseRichard Purdie2021-10-112-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | We have a choice of policy with hashequivalence - whether to reduce sstate duplication in the sstate feed to a minimum or have maximal sstate reuse from the user's perspective. The challenge is that non-matching outhashes are generated due to determinism issues, or due to differences in host gcc version, architecture and so on and the question is how to reconcile then. The approach before this patch is that any new match is added and matches can update. This has the side effect that a queried value from the server can change due to the replacement and you may not always get the same value from the server. With the client side caching bitbake has, this can be suboptimal and when using the autobuilder sstate feed, it results in poor artefact reuse. This patch switches to the other possible behaviour, once a hash is assigned, it doesn't change. This means some sstate artefacts may be duplicated but dependency chains aren't invalidated which I suspect may give better overall performance. Update the tests to match the new behaviour. (Bitbake rev: 20d6ac753efa364349100cdc863e5eabec8e5b78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Fix diverging report race conditionJoshua Watt2021-10-114-153/+314
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes the hashequivalence server to resolve the diverging report race error. This error occurs when the same task(hash) is run simultaneous on two different builders, and then the results are reported back but the hashes diverge (e.g. have different outhashes), and one outhash is equivalent to a hash and another is not. If taskhash was not originally in the database, the client will fallback to using the taskhash as the suggested unihash and the server will see reports come in like: taskhash: A unihash: A outhash: B taskhash: C unihash: C outhash: B taskhash: C unihash: C outhash: D Note that the second and third reports are the same taskhash, with diverging outhashes. Taskhash C should be equivalent to taskhash (and unihash) A because they share an outhash B, but the server would not do this when tasks were reported in the order shown. It became clear while trying to fix this that single large table to store all reported hashes was going to make these updates difficult since updating the unihash of all entries would be complex and time consuming. Instead, it makes more sense to split apart the database into two tables: One that maps taskhashes to unihashes and one that maps outhashes to taskhashes. This should hopefully improve the parsing query times as well since they only care about the taskhashes to unihashes table, at the cost of more complex INNER JOIN queries on the lesser used API. Note this change does delete existing hash equivlance data and starts a new database table rather than converting existing data. (Bitbake rev: dff5a17558e2476064e85f35bad1fd65fec23600) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add tests for diverging reportsJoshua Watt2021-10-111-0/+53
| | | | | | | (Bitbake rev: 953c8d622c9d1bc1eb06bcaf1eaa3aa9f85d0bc2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: async: Close sync client event loopJoshua Watt2021-10-112-2/+8
| | | | | | | | | | Prevents `ResourceWarning: unclosed event loop` warnings when using the synchronous client and python exits (Bitbake rev: 8b95972bc04ce52a98c7780184af15a5e95f987b) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: replace http with https for URLsJon Mason2021-10-0815-54/+54
| | | | | | | | | | | https has been the preferred way to access websites for many years now. Change all of the URLs with a _working_ https server/certificate to use that URL. (Bitbake rev: da543cdaf88a387675e25d3555765f1146e4105e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake:toaster:test: Update SSTATE URLJon Mason2021-10-081-1/+1
| | | | | | | (Bitbake rev: b3c0dbddd7eb3c87e3989977d7640f09b49a460b) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Handle pseudo shutdown in Ctrl+C caseRichard Purdie2021-10-081-4/+6
| | | | | | | | | | | If the build is interrupted, handle the shutdown of pseudo even in this case to avoid data corruption inside docker containers. [YOCTO #14555] (Bitbake rev: a2a04c6fe94bc56efcff299c669a151746e35916) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty/uihelper: Show setscene task progress in summary outputRichard Purdie2021-10-082-4/+6
| | | | | | | | | | | | | | | With the changes to task accounting, bitbake doesn't show progress when executing setscene tasks on the summary console output. Change to show a progress within the setscene tasks and a progress within the main tasks. I can't see any way to display this more easily without confusing users. [YOCTO #14586] (Bitbake rev: 0244acb968eb593d2ad7bc6e52f222c2b1d39aa9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npmsw: Add support for local tarball and link sourcesStefan Herbrechtsmeier2021-10-081-3/+20
| | | | | | | (Bitbake rev: 4f983dc419a1a6f635a5d333f253d49244cec374) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npm: Create config npmrc in environment instantiationStefan Herbrechtsmeier2021-10-081-15/+27
| | | | | | | | | | | | | | | Create a configuration npmrc per npm environment to avoid repeated creation of the same configuration file. Create the file via python to avoid multiple npm config calls and add the ability to pass a file path instead of a temporary file. Deprecate the npm configs argument of the run function. The configs should be passed to npm environment or as command specific arguments. (Bitbake rev: 2c2df49b06a2bad7a5b8872a9998338a4660498f) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npm: Support npm archives with missing search directory modeStefan Herbrechtsmeier2021-10-081-0/+2
| | | | | | | | | | Delay directory restore and set execute/search directory mode bits in unpack to support npm archives with a missing search directory mode. (Bitbake rev: 60cbd34d3da8f0f523281aad7eec93eec9cd4db8) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gitsm: remove the 'nugget' SRCREV cachingRoss Burton2021-10-081-18/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cached revisions which are used to decide if a repository doesn't need to be updated are misleading when used in conjunction with mirror tarballs and can cause partial fetches to happen, resulting in unpack errors as repositories were not fetched. A concrete example: edk2-firmware in meta-arm is at version 202102 (ef91b0). This is built on the autobuilder so the source mirror contains the repository as a mirror tarball. If I build edk2-firmware 202102 the gitsm fetcher will initially download the top-level repository and then iterate into the submodules to also fetch those repositories, including cmocka from cryptomilk.org. edk2-firmware will then unpack and build successfully. I then update edk2-firmware to 202105 (e1999b) and build it. Gitsm.needs_update() starts by calling Git.needs_update() which returns False, as the mirror tarball contains this revision. It then looks at the "nuggets" which are SRCREVs it has fetched before. The mirror tarball itself contains the nugget for e1999b as this has been built on the autobuilder, so needs_update return False, no more fetching is done, and the build proceeds to unpack. However, as part of the 202105 upgrade the URL of the cmocka submodule changed, and this new repository was never fetched. This means that unpack fails as one of the required git repositories isn't available. The nugget codepaths appear to be an attempt at optimising the fetch process, but have demonstratable failure cases. Just removing them entirely solves the edk2-firmware example, and all of the fetcher test cases still pass. (Bitbake rev: 51212507ce3f670ace9efb691c92887d66f7aaf8) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: clarify the command-no-found error messageRoss Burton2021-10-081-1/+1
| | | | | | | | | | | | | If runfetchcmd() fails with bb.process.NotFoundError, the message output is simply "Fetch command" which doesn't really explain what the problem is. Add "not found" to clarify what happened. (Bitbake rev: 8de9dc02ed6a73b47f2ab10be30d1aed7954bc72) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: prefix the FetcherTest temporary directoryRoss Burton2021-10-081-1/+1
| | | | | | | | | Set a prefix so the temporary directories are identifable. (Bitbake rev: c3440b82cbe9c317f9961d61e12ea37fc9541ce0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/utils: mark a regex as a raw stringRoss Burton2021-10-081-1/+1
| | | | | | | | | Avoids a warning from Python as \s isn't a valid escape. (Bitbake rev: be39c2e926beebe78030fbe26c6737f08f960fcb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Ensure functions are defined in a deterministic orderRichard Purdie2021-10-031-1/+1
| | | | | | | | | | | When writing functions into shell scripts, write then in a deterministic order. This is unlikely to affect anything at runtime but it does change the signatures of the generated useradd postinst scripts in OE-Core and is a good thing to be consistent about in general. (Bitbake rev: 8a181dc8f3c8c9c9885ea3011cb234321a296d92) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/command: Add a dummy event for tinfoil testingRichard Purdie2021-09-262-0/+15
| | | | | | | | | | We need a command genetating an event to test through the tinfoil API. The current test has IO load issues so add a dummy version which won't have the IO constraints. (Bitbake rev: a144178584394ea0700ffc2f2bfac94e8690effc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix sorting in diff outputRichard Purdie2021-09-261-7/+7
| | | | | | | | | The diff output isn't deterministic at the moment as the sets can have differing ordering. Sort the output so it is consistent. (Bitbake rev: 117830c1d7ef3e53052fa326e1ca62c5c3946c45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Allow shutdown/database flush of pseudo server at ↵Richard Purdie2021-09-261-1/+9
| | | | | | | | | | | | | | | | | | task exit We have a problem where pseudo server processes exist after bitbake exits and hold the pseudo database in memory. In a docker container, the processes will be killed as the container is destroyed with no warning and no opportunity to write the data to disk. This leads to permissions/inode corruptions and data loss. Send a shutdown message to pseudo which in new versions of pseudo will flush the database, thereby fixing some of the issues people using docker containers see. (Bitbake rev: a07a971b40acd3eee12e203d2cfa3e49f56109f6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-26 19:35:56 +0000