linux/poky.git/bitbake/lib, branch yocto-2.6.4

bitbake: cookerdata: Add mc conffiles hashes to cache hash

2019-10-10T15:53:26+00:00

The variable values that result from parsing multiconfig should be included in the cooker data hash, otherwise changes to these files won't be detected, which will allow the parsing cache to be loaded with the old values for the multiconfigs. This can either manifest as the variable values simply not updating, or getting basehash changed errors when building. This bug was previously undetected because all of the multiconfig base files were a direct file dependency in all parsed recipes. This was fixed in 34137a00f60 ("bitbake: bitbake: cooker: Rename __depends in all multiconfigs"), exposing this bug. [YOCTO #13541] (Bitbake rev: 6b045e074c6fea97d4e305a5a3c8bf82135d95eb) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: tests/fetch: Resolve fetch error in bitbake-selftest

2019-09-09T08:58:09+00:00

FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 8a58c3c64240c6ab14858d18e6b89febdb315311) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: fetch2: show warning when renaming the archive with bad checksum failed

2019-09-06T10:58:56+00:00

* noticed on read-only sshfs premirror * it was showing the warning about renaming the file: WARNING: laser-geometry-1.6.4-r0 do_fetch: Renaming /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 and then failed because of movefile() issue with python3 (fixed in previous commit): ERROR: laser-geometry-1.6.4-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated: with movefile() fixed, it let do_fetch continue and re-fetch locally with the right checksum, but still the renamed file didn't exist, because of movefile failure - add another warning when the movefile fails - for whatever reason - unfortunately movefile prints error messages with just print() so the real error is hidden only in log.do_fetch in this case: movefile: Failed to move /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 [Errno 30] Read-only file system: '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz' -> '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441' (Bitbake rev: d36438759344caa447d9a0bf30749a0aa31d1fba) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

bitbake: utils: Fix movefile() exception handling with python3

2019-09-06T10:58:56+00:00

* with python3 this fails with: File: 'bitbake/lib/bb/utils.py', lineno: 799, function: movefile 0795: try: 0796: os.rename(src, destpath) 0797: renamefailed = 0 0798: except Exception as e: *** 0799: if e[0] != errno.EXDEV: 0800: # Some random error. 0801: print("movefile: Failed to move", src, "to", dest, e) 0802: return None 0803: # Invalid cross-device-link 'bind' mounted or actually Cross-Device Exception: TypeError: 'OSError' object is not subscriptable (Bitbake rev: 9f92322fa8d6f1a68c0c3f4984afdf65126b51dc) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

bitbake: gitsm: Add need_update method to determine when we are going to a new SRCREV

2019-06-18T10:31:10+00:00

If the system had previously fetched a source repository for use by gitsm, and then the SRCREV was updated and the new commit already existed, the system would not re-evaluate the submodules and update them accordingly. The cause of this issue was that need_update was being used, unmodified, from the base git fetcher. It did not have any knowledge, nor did it care if we were moving commits and needed to re-evaluate what was happening due to this switch. To fix the issue, during the download process we add all processed (by gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to use a new need_update function that not only checks if the git commit is present, but if we have previously processed this commit to ensure all of the submodule components are also present. This approach is used, instead of iterating over the submodules in need_update to avoid a potential race condition that has affected us in the past. The need_update is called only with the parent locking. Any time we need to dive into the submodules, we need to lock, and unlock them, at each stage. This opens the possibility of errors in either the code, or unintended race conditions with rm_work. This issue was discovered by William A. Kennington III . The included test case was also written by him, and included unmodified. (Bitbake rev: 4ce92f43eeac6a4bfd06e8567fa6891614b5b3b0) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

bitbake: gitsm: Fix a bug where the wrong path was used for the submodule init

2019-06-18T10:31:10+00:00

Because we are trying to avoid network activity and use our own fetcher, the system emulates the behavior of 'git submodule init'. git submodule init uses the .gitmodules file, where typically the module name and path are the same. However, in this case the module name and path (in the tree) were different. i.e.: [submodule "edgelet/hsm-sys/azure-iot-hsm-c/deps/azure-c-shared-utility"] path = edgelet/hsm-sys/azure-iot-hsm-c/deps/c-shared url = https://github.com/Azure/azure-c-shared-utility.git Previously the code assumed the 'path' was both the checkout location under .git/modules, as well as the path to extract the components. This proved to be incorrect as the .git/modules path needs to match the submodule 'name'. This causes the components that were fetched to be initialized in the wrong location, which later caused the 'git submodule update' process to skip not properly initialized modules. A test case was added for this specific case to ensure a regression does not appear in the future. (Bitbake rev: ffd7ed530a17d22df576d986ac78428a6979e79c) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

bitbake: bitbake: fix version comparison when one of the versions ends in .

2019-06-15T10:06:38+00:00

Previously, this would happen: ====================================================================== ERROR: test_vercmpstring (bb.tests.utils.VerCmpString) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/alexander/development/poky/bitbake/lib/bb/tests/utils.py", line 45, in test_vercmpstring result = bb.utils.vercmp_string('1.', '1.1') File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 143, in vercmp_string return vercmp(ta, tb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 135, in vercmp r = vercmp_part(va, vb) File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 124, in vercmp_part elif ca < cb: TypeError: '<' not supported between instances of 'NoneType' and 'int' ---------------------------------------------------------------------- (Bitbake rev: bd953d56d007a8bfa5ecb6e753da4abfb035f9f2) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: bitbake: cooker: Rename __depends in all multiconfigs

2019-06-11T12:28:21+00:00

The renaming of the __depends variable to __base_depends and file watches needs to occurs for all multiconfigs, not just the base config. Failing to do this for all multiconfigs will result in a huge increase in the size of the parsing cache (about 5x for a single mulitconfig) because all multiconfig caches will still depend on the base config files. This will also seen a similar jump in the amount of time required to load the parsing cache from memory, both because the cache is larger and because of explosion of additional existence checks that must be done for the base files. [YOCTO #13359] (Bitbake rev: da5d1560d4ad1c735f6166a5d9ce94f36c94186b) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: toaster: Fix Thud Bitbake release metadata

2019-06-10T08:44:41+00:00

Fix the Bitbake version ID for Thud (1.40). [YOCTO #13356] (Bitbake rev: f8a23ad4f8f0336470a5d5b074b7a2d02ec94631) Signed-off-by: David Reyna Signed-off-by: Richard Purdie

bitbake: gitsm: The fetcher did not process some recursive submodules properly.

2019-03-24T16:42:22+00:00

Move the submodule processing outside of the if statement to avoid any optimizations that may happen. Update the test cases to include the additional case, and split the other test cases into individual tests to make it easier to figure out what the failure may be. (Bitbake rev: 7c1eb51d1e8a4c5f39bf9dddf05fb0b3598da72b) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 0ec98c01ae50f95c9c74acf53013ac59e0e72b08) Signed-off-by: Scott Murray Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie