summaryrefslogtreecommitdiffstats
path: root/meta/classes
Commit message (Collapse)AuthorAgeFilesLines
* buildhistory: Fix srcrevs outputRichard Purdie2021-12-081-17/+13
| | | | | | | | | | | | | | | | | | | The code was assuming that the a recipe with only one srcrev wouldn't "name" it. This isn't the case as the glibc or bzip2 recipes show, you can have a single srcrev which is named. We can pull the data from the fetcher and in fact we already have it, we just need to handle the "default" case and make that code the default for all srcrev regardless of length. [YOCTO #14017] (From OE-Core rev: 9a7ea10660d0efd87e8cadf866e6dbed112b7f94) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 45ae567932ba52b758eb41754453e9828d9533a1) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible_build/package_XXX: Ensure SDE task is in dependency chainRichard Purdie2021-12-024-4/+8
| | | | | | | | | (From OE-Core rev: c8c0757f06a8cfa2159575e0b98530b899a889b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 938e4dfb68a465f61cb754b2cd28d0728616b5a9) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* mirrors: Add kernel.org sources mirror for downloads.yoctoproject.orgRichard Purdie2021-11-151-0/+1
| | | | | | | | | | | kernel.org now has a mirror of the downloads.yoctoproject.org sources archive so include this in our mirrors list. (From OE-Core rev: dbbec36638f035aefbaf22f152a2ca95836d08bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible_build: Remove BUILD_REPRODUCIBLE_BINARIES checkingMark Hatle2021-11-151-19/+34
| | | | | | | | | | | | | | | | | | | | | | Previously if BUILD_REPRODUCIBLE_BINARIES was set to 0, the system would fall back and select the default epoch (April 2011), but still perform the reproducible build actions. This resulted in binaries that had an unusually old date. Simplify the functions and remove the anonymous python as no longer necessary. Also improve the documentation to better explain what the class is doing and how a recipe can override the behavior if necessary. (From OE-Core rev: 814bedacac9f5c343c9888c0b14649189a84f817) Signed-off-by: Mark Hatle <mark.hatle@xilinx.com> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1976013b026cfba94de32a13e994d92d7e9e39e5) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Avoid deploy_source_date_epoch sstate when unneededRichard Purdie2021-11-151-0/+4
| | | | | | | | | | | | | | | This sstate task is only needed when depended upon, it can be skipped if there are no tasks running that directly depend upon it. This reduced the number of sstate tasks in something like an image build. (From OE-Core rev: 884e44701ada57abe4d8ad9ece424435be25c6a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 601cee016da5c7505915e26641a085714de175ce) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Ensure SDE is accounted for in package task timestampsRichard Purdie2021-11-151-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating packages we build them with --clamp-mtime and use SOURCE_DATE_EPOCH as the maximum mtime. This makes the end packages reproducible. The data stored in sstate for do_package and the package task doesn't benefit from this though and have varying timestamps. This means their outhash varies and means hash equivalance isn't effective at all and doesn't work as intended/desired. We could create the sstate archives with the same clamping however that would lead to different results depending on whether a task was installed from sstate or not. Making that differ is a path to madness. It also wouldn't fix the outhash of the task to be determninistic without clamping of the date in the hash calculation code. Instead, iterate over the files in sstate output and clamp them at the code level. This isn't ideal but does make the file timestamps determnistic everywhere and means we don't have to change the hash calculation code. This issue can be clearly seen looking at the do_package outhash for a recipe which you then re-run the package task for after adding something like whitespace to the install task. The outhash shouldn't change but currently does. (From OE-Core rev: 06b8f2a5a24be1a87f0eaf29fdba719ebe3bb06e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c3b3cc4745811b48b9193f83889946b2e1788932) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: another fix for touching files inside pseudoJose Quaresma2021-11-151-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is a fixup for 676757f "sstate: fix touching files inside pseudo" running the 'id' command inside the sstate_unpack_package function shows that this funcion run inside the pseudo: uid=0(root) gid=0(root) groups=0(root) The check for [ -w ${SSTATE_PKG} ] and [ -O ${SSTATE_PKG}.siginfo ] will always return true and the touch can fail when the real user don't have permission or in readonly filesystem. As the documentation refers: - the file test operator "-w" check if the file has write permission (for the user running the test). - the file test operator "-O" check if you are owner of file We can avoid this test running the touch and mask any return errors that we have. (From OE-Core rev: 29fc85997ade490ae46ffca37ef8e1a56957c876) (From OE-Core rev: e7d94a9cc5ab1b2c5d160fd06d643a4bc3409d26) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5b9210d66c78bb3f79056e5586cea7b0edd714a9) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* mirrors: Add uninative mirror on kernel.orgRichard Purdie2021-11-151-0/+1
| | | | | | | | | | | | | At the last nas outage, we realised that we don't have good mirrors of the uninative tarball if our main system can't be accessed. kernel.org mirrors some Yocto Project data so we've ensured uninative is there. Add the appropriate mirror url to make use of that. (From OE-Core rev: ebeb1458c7f24cd97978beb7cddf814cae43c6a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Add explict branch to git SRC_URIs, handle github url changesSteve Sakoman2021-11-111-1/+1
| | | | | | | | | | | | | | | | | | | | This update was made with the convert-scruri.py script in scripts/contrib This script handles two emerging issues: 1. There is uncertainty about the default branch name in git going forward. To try and cover the different possible outcomes, add branch names to all git:// and gitsm:// SRC_URI entries. 2. Github are dropping support for git:// protocol fetching, so remap github urls as needed. For more details see: https://github.blog/2021-09-01-improving-git-protocol-security-github/ (From OE-Core rev: 827a805349f9732b2a5fa9184dc7922af36de327) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* mirrors.bbclass: remove dead infozip mirrorsOleksandr Kravchuk2021-11-031-1/+0
| | | | | | | | | | | (From OE-Core rev: 615a25ea0d8d8e2134fa8ade03b3883ae0c2f593) Signed-off-by: Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0140df8724a1c73f7b62fbbbaee58c3eb119eeba) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: fix unclosed testdata fileRoss Burton2021-11-031-3/+4
| | | | | | | | | | (From OE-Core rev: a1e49456343a2be9adb6c0d1d970c2b0c070f53e) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0c192a97e3e1c015a48667d6903cc07a8b2620e4) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible_build: Drop obsolete sstate workaroundRichard Purdie2021-11-031-4/+0
| | | | | | | | | | | | sstate has been sufficiently invalidated since this change was made, drop the workaround now. (From OE-Core rev: 81b602eaab686fa5a523c023285f8ffd7050888d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7a497d8f30f21bafc78d0f22f3442a9cc99544cb) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* license.bbclass: implement ast.NodeVisitor.visit_ConstantRoss Burton2021-11-031-0/+4
| | | | | | | | | | | | | | | Since Python 3.8 visit_Num(), visit_Str() and so on are all deprecated and replaced with visit_Constant. We can't yet remove the deprecated functions until we require 3.8, but we can implement visit_Constant to silence the deprecation warnings. (From OE-Core rev: d91fe6ecb9fbb410b3bab6ced66b7fe5f869cf83) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 875a944392a3c93f40081a14af357f70b6b8264f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base: Use repr() for printing exceptionsRichard Purdie2021-11-031-2/+2
| | | | | | | | | | | | | | Exceptions print more clearly using repr() instead of str(), fix in fetch and unpack tasks. Drop part of the test which no longer makes sense after this change. (From OE-Core rev: 045124ce7ef7c53a1932848835f93abbe535f157) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c94816259cc1c09746353ad26ca0c811e0c962c2) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base: Clean up unneeded len() callsRichard Purdie2021-11-031-4/+4
| | | | | | | | | | | | This code pattern isn't very pythonic, improve it to drop the unneeded len() calls. (From OE-Core rev: e0cecf7b4a4b966f11527a88e632997bdf74af96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 69376ac1a6147b26fe1abaa4cf68414024814d63) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: don't silently handle all exceptions in sstate_checkhashesRoss Burton2021-11-031-3/+4
| | | | | | | | | | | | | | | If checkstatus returns an exception we should silently handle FetchError, as this means the fetch failed for 'normal' reasons such as file not found. However, other exceptions may be raised, and these should be made visible. (From OE-Core rev: f16803718a19ace7b582c5b73924fe3112502927) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 50d99faf88a1d82cbd939b9bd6e33ebed2b1ffd8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix modify with patches in override directoriesRoss Burton2021-11-031-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a recipe applies patches which are in machine-specific override directories, devtool will fail to fetch the patches that don't match the default configuration. For example where there are patches at qemux86/x86.patch and qemuarm/arm.patch: SRC_URI = "file://source" SRC_URI_append_qemuarm = " file://arm.patch" SRC_URI_append_qemux86 = " file://x86.patch" The patch apply phase sets OVERRIDES but does not set FILESOVERRIDES, so it cannot find the patch files as the search path isn't correct. Fix this by setting FILESOVERRIDES too. Also when iterating through the overrides we need to be sure that other overrides that are used are not enabled, so extend no_overrides instead of simply appending the current override. Fixes most but not all of [ YOCTO #14060 ]. (From OE-Core rev: a372cdf8e175423c47faeecc98ad076ee26bbec8) (From OE-Core rev: b20699229671ef37daac8b0ed1133aacb477f6a0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4a35bcc9d164ac038a31356a15a0f61ccdd38be2) Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: fix touching files inside pseudoJose Quaresma2021-11-031-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | running the 'id' command inside the sstate_create_package function shows that this funcion run inside the pseudo: uid=0(root) gid=0(root) groups=0(root) The check for touch files [ ! -w ${SSTATE_PKG} ] will always return true and the touch can fail when the real user don't have permission or in readonly filesystem. As the documentation refers, the file test operator "-w" check if the file has write permission (for the user running the test). We can avoid this test running the touch and mask any return errors that we have. (From OE-Core rev: 590de1dd89cfd5f0ca7395880ba88b27ee35470d) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f6e7445c94443544e92fda97a017ce93393c5f84) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/reproducible_build: Use atomic rename for SDE fileJoshua Watt2021-10-291-7/+6
| | | | | | | | | | | | | | | | | | | | If an existing source date epoch file was found during do_unpack, it was deleted and a new one would be written in its place. This causes a race with check-before-use code in get_source_date_epoch_value. Resolve the problem by making do_unpack write the new source date epoch to a temporary file, then do an atomic rename to ensure it's always present, and change the check-before-use code to use a EAFP exception instead of checking for file existence. [YOCTO #14384] (From OE-Core rev: b98d37da1554f524bd5b16287731d7b34945e92d) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0b5e3b33187bf78a2d62cc886463e4b27d6bd228) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible_build: Work around caching issuesRichard Purdie2021-10-291-4/+7
| | | | | | | | | | | | | | | | | | SOURCE_DATE_EPOCH can be expanded early in the parsing process before the class extensions are applied. This can mean the directory pointed to for the SDE can be incorrect until later in parsing. Cache the file name in the cached value and allow it to dynamically update. This isn't ideal but avoding expansion of the variable likely isn't possible and I'm not sure how else to handle this. This works around the issue until a better solution can be found. (From OE-Core rev: 257eb2ee73831afe84600235c967cbb4c2627e26) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 44dc97cd1223e4d2b635669627ec5f796838d42d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* externalsrc: Fix a source date epoch race in reproducible buildsRichard Purdie2021-10-231-10/+9
| | | | | | | | | | | | | | | When reproducible builds are enabled and externalsrc is in use, the source date epoch function is added. The conditions on the conditional code removing the unpack task need to match the deltask function, else the source date epoch function can end up running twice and the functions can race with each other causing build failures or corruption. (From OE-Core rev: e30c2be8b9534a8dbec5520b41a00c6dc8aa65a4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e82095c02881410035ca23dc12692f074d8ed39b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* externalsrc: Work with reproducible_buildMark Hatle2021-10-231-0/+10
| | | | | | | | | | | | | | | | | | | Externalsrc removes do_fetch, do_unpack, and do_patch. The system normally discovers the correct reproducible date as a postfuncs of do_unpack, so this date is never found, so it falls back to the default epoch. Instead we can move the discovery function to a prefuncs on the epoch deploy task. This task will run before do_configure, and since the source is already available can run safely at anytime. (From OE-Core rev: badef50c6c9821f1975483de0ba50a75f2eda52c) Signed-off-by: Mark Hatle <mark.hatle@xilinx.com> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0b7dd711a54e92ce54abe99f59fc67e683d52dfe) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Ensure pclist files are deterministic and don't use full pathsRichard Purdie2021-10-231-3/+3
| | | | | | | | | | | | | | Currently the pkgconfig pclist files contain full paths which are build host specific and the order of entries is not deterministic. Fix both these issues so the files are deterministic. (From OE-Core rev: 00615c9edb8b9b644e853453603846eddfff2054) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e422e29bca4af3ab4073e04490f38b05cd7c38c0) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* uninative: Improve glob to handle glibc 2.34Ross Burton2021-10-231-1/+1
| | | | | | | | | | | | | | With glibc 2.34, the libraries were renamed. Tweak the glob to support both as this is needed for newer uninative versions. [RP: tweak commit message] (From OE-Core rev: effa24ebf216687c97b9edc5d2ac4a74ff430f4c) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 98248306e4b5f023e96375293b60524574ebb686) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patch.bbclass: when the patch fails show more info on the fatal errorJose Quaresma2021-10-231-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are situations when the user have the 'patchdir' defined as a parameter on SRC_URI. However he doesn't know that with this the patch is applied relatively to the receipe source dir 'S'. - When user have 'patchdir' defined check if this directory exist. - If the patch fails show addition info to the user: - Import: show the striplevel - Resolver: show the expanded 'patchdir' to the user. The next example is from opencv in meta-oe layer, here the patch is applied on the target directory ${WORKDIR}/git/contrib. S = "${WORKDIR}/git" SRCREV_FORMAT = "opencv_contrib" SRC_URI = "git://github.com/opencv/opencv.git;name=opencv \ git://github.com/opencv/opencv_contrib.git;destsuffix=contrib;name=contrib \ file://0001-sfm-link-with-Glog_LIBS.patch;patchdir=../contrib \ " * When the patch fail there are no message that indicates the real reason. patchdir=../no-found-on-file-system ERROR: opencv-4.5.2-r0 do_patch: Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output: stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch can't find file to patch at input line 37 Perhaps you used the wrong -p or --strip option? * The check of the patchdir will add a new fatal error when the user specifies a wrong path than don't exist. patchdir=../no-found-on-file-system ERROR: opencv-4.5.2-r0 do_patch: Target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../no-found-on-file-system' not found, patchdir '../no-found-on-file-system' is incorrect in patch file '0001-sfm-link-with-Glog_LIBS.patch' * When we can't aplly the patch but the patchdir exist, show the expanded patchdir on fatal error. patchdir=../git ERROR: opencv-4.5.2-r0 do_patch: Applying patch '0001-sfm-link-with-Glog_LIBS.patch' on target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../git' Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output: stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch can't find file to patch at input line 37 Perhaps you used the wrong -p or --strip option? (From OE-Core rev: af46b360b364f6b50cd389d838dc5dd3099ecc41) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit c44bc7c0fb8b7c2e44dd93607a3bfd9733e1df80) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rm_work.bbclass: Fix for files starting with -William A. Kennington III2021-10-071-4/+4
| | | | | | | | | | | | | | This makes it possible to name files starting with a hyphen in the work directory. Without this change rm will fail due to an unexpected option being passed. (From OE-Core rev: eb65a5870a1f5c5a4fbce0df87a7dbfb38487dce) Signed-off-by: William A. Kennington III <wak@google.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5f1a63e0de4921ef970114a16d0827fcddcdaa0e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* libc_package/buildstats: Fix python regex quoting warningsRichard Purdie2021-10-071-1/+1
| | | | | | | | | (From OE-Core rev: dd705fccfb35f9d3fee1452a5145d4e7ad24999d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6a8077317ce12e13018ec4472f728dd24880bda9) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: symlink the task log and qemu console log to tmp/log/oeqaAlexander Kanavin2021-09-301-2/+10
| | | | | | | | | | | | | | | | This makes it easier for the AB scripts (particularly, collect-results) to access and archive these items, as they can contain useful information when ptests or other qemu tests fail (and also if they don't fail). [YOCTO #14518] (From OE-Core rev: bdad1bdfec7fa86a6550f8aeb4e74029599df6d5) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1965b344abcff0ba584136f929b4a14645f1585e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* useradd: Ensure preinst data is expanded correctly in pkgdataRichard Purdie2021-09-301-0/+4
| | | | | | | | | | | | | | | | | | | | | The preinst data in pkgdata will not expand out the ${XXX_PARAM} variables since they don't use a package suffix. It happens that the final expansion used for the packages is corrected by a second trip through the datastore. The first version is used for calculation of the task output hash and recent improvements in hash reuse showed this data wasn't using included in the hashes, meaning for example builds with dynamic IDs were mixing sstate with builds using static IDs. The result was a mess. Fix this by expanding the data in the preinst correctly to use the package specific _PARAM values. (From OE-Core rev: fbd5a1f877d805d810562985c7a5dac4a5761263) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 375430f249e7e0b6622e566e2478b40ba7e606ab) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdk: Decouple default install path from built in pathRichard Purdie2021-09-081-0/+1
| | | | | | | | | | | | | | | | | Add SDKPATHINSTALL which is used as the default install location of the SDK instead of SDKPATH. This means the default install path isn't encoded into every SDK binary, meaning if a date is used there the entire SDK doesn't have to rebuild. Most distros can switch to only customise SDKPATHINSTALL meaning more sstate reuse too. [YOCTO #14100] (From OE-Core rev: 33a5714592bf41a26fcd7e8ff595b45538082431) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bc4ee5453560dcefc4a4ecc5657df5cc1666e153) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image_types: Restore pre-btrfs-tools 4.14.1 mkfs.btrfs shrink behaviorMarek Vasut2021-09-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the mkfs.btrfs generates large images with a lot of wasted space. This happens since OE-core updated btrfs-tools from 4.13.3 to 4.15.1 in commit 94b645aa77 ("btrfs-tools: update to 4.15.1") . Note in mkfs.btrfs(8) manpage section -r says the following: " -r|--rootdir <rootdir> ... Note This option may enlarge the image or file to ensure it’s big enough to contain the files from rootdir. Since version 4.14.1 the filesystem size is not minimized. Please see option --shrink if you need that functionality. --shrink Shrink the filesystem to its minimal size, only works with --rootdir option. ... Note prior to version 4.14.1, the shrinking was done automatically. " Add the --shrink option to EXTRA_IMAGECMD_btrfs to reinstate the original behavior and un-waste the space. (From OE-Core rev: 6de30e62fc5815edc19c96b49c0ff9681298c56c) Signed-off-by: Marek Vasut <marex@denx.de> Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Cc: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c4a99d36967302c176b62fad840b5e79486ea356) Cc: Steve Sakoman <steve@sakoman.com> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Reduce the number of calls to the "dirname" commandOleksandr Popovych2021-09-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | utils.bbclass contains create_cmdline_wrapper() function that creates wrapper script with additional arguments for any passed "$cmd" command, and uses several calls to "dirname". Because "dirname" is an external command, in cases of lots of calls to wrapped "$cmd", each call of "dirname" will incur significant overhead. There are three same calls to "dirname": one for saving it`s output to "realdir" variable, and other two in "exec" command. So last two "dirname" calls can be replaced with cached value from "realdir" variable. (From OE-Core rev: 912c9eda653c45fee2f55092fbe281efba897bc0) Signed-off-by: Oleksandr Popovych <opopovyc@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4b9cf2c80fd14386e0b88a2e6c40a9fa3f1ae0f7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdk: Enable do_populate_sdk with multilibsKhem Raj2021-08-262-2/+1
| | | | | | | | | | | | | | | | | | | This patch enables building image based SDKs for multi-libbed images e.g. lib32-core-image-minimal and so on. Change the path to nativesdk tools to use recipe-sysroot since thats where the nativesdk components are installed and it will need access to qemu wrappers during build for processing intercepts [YOCTO #14444] (From OE-Core rev: eea71d274fe4d5f23ba0ab35d14be4fff2e64e2b) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6196a785eababb040ee1dee9f33cb6d6dad77eef) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate.bbclass: fix error handling when sstate mirrors is roJose Quaresma2021-08-181-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The commit dd555537fc35c5f934af09d601d70772eb5955ae 'sstate.bbclass: fix errors about read-only sstate mirrors' adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The exception handler checks the error type with the python module errno but this module needs to be imported as it don't exist. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 778, function: sstate_task_postfunc 0774: 0775: omask = os.umask(0o002) 0776: if omask != 0o002: 0777: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0778: sstate_package(shared_state, d) 0779: os.umask(omask) 0780: 0781: sstateinst = d.getVar("SSTATE_INSTDIR") 0782: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/home/builder/src/base/poky/meta/classes/sstate.bbclass', lineno: 708, function: sstate_package 0704: except PermissionError: 0705: pass 0706: except OSError as e: 0707: # Handle read-only file systems gracefully *** 0708: if e.errno != errno.EROFS: 0709: raise e 0710: 0711: return 0712: Exception: NameError: name 'errno' is not defined (From OE-Core rev: ce78c16409363741d59a2f787aca66077bec93cd) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 15f30ad144fbe25e9a5e71bc7e42e746d2039992) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: remove deprecated CVE_CHECK_CVE_WHITELISTMichael Opdenacker2021-08-181-3/+0
| | | | | | | | | | | | This variable has been deprecated since Yocto Project version 3.0. (From OE-Core rev: 3c4719eda2b0d8fe1b7cc4006c05ee8c4618e4c6) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f8ac58568b2dceef54a743369460019b3a3eeccd) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: improve comment about CVE patch file namesMichael Opdenacker2021-08-181-3/+6
| | | | | | | | | | (From OE-Core rev: 3f5270d1c96f623c27043159f47a952dc24a565b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8aa613480663e11ecc62278d8c57ca719eb23899) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: update link to NVD website for CVE detailsMichael Opdenacker2021-08-181-1/+1
| | | | | | | | | | | | | | | The old URL schema https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-ID now redirects to https://nvd.nist.gov/vuln/detail/CVE-ID (From OE-Core rev: b2c21e0d7230583e4751a124c015534e5cd16490) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 57adb57a9d9b08c08ab606ec7b561792e4f4ff2d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: fix commentsMichael Opdenacker2021-08-181-4/+4
| | | | | | | | | | | | | | This implements various fixes in comments in cve-check.bbclass In particular, the "whitlisted" typo is important as the "whitelisted" word is going to be replaced in a near future. (From OE-Core rev: 596859862409f83861665a7e282da8f9f7297878) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5eecd2bf942254d08c252388594e5ec7ae330f45) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: add include/exclude layersakuster2021-08-181-0/+17
| | | | | | | | | | | | | | | | | There are times when exluding or including a layer may be desired. This provide the framwork for that via two variables. The default is all layers in bblayers. CVE_CHECK_LAYER_INCLUDELIST CVE_CHECK_LAYER_EXCLUDELIST (From OE-Core rev: 76d5c60734c614b8224bef97022738f3a59244d0) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5fdde65ef58b4c1048839e4f9462b34bab36fc22) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* kernel-yocto: Simplify no git repo case in do_kernel_checkoutPaul Barker2021-08-181-15/+15
| | | | | | | | | | | | | | | | | | | | | If the kernel sources are not fetched via git, a local git repository is created in do_kernel_checkout. In this case we know that there will be no remote branches and we will already be on the correct branch (since only one branch will exist). So we can simplify things by skipping these steps. This also removes the assumption that the default git branch name will be "master". Prior to this change, the final git checkout command in do_kernel_checkout could fail if a local git repo was created and the user had changed init.defaultBranch in their gitconfig. (From OE-Core rev: 3a16f312e5942e8a3d6f77a550c5374da81eed37) Signed-off-by: Paul Barker <paul@pbarker.dev> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit af2a9c92d4498492ca23388c7b4bbed48abdc4d7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* license: Exclude COPYING.MIT from pseudoRichard Purdie2021-08-181-2/+2
| | | | | | | | | | | | | | | | | | | Along with the other license exclusions, we need to exclude the top level COPYING.MIT file else when: COPY_LIC_DIRS = "1" COPY_LIC_MANIFEST = "1" is set, we see eSDK failures from a pseudo abort. [YOCTO #14366] (From OE-Core rev: 46dd415720eb507a3d181a289b155ac4cfc2219c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3eb580843de3f055e42fcce60b0f15c4190c0542) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Fix rebuilds when changing layer configRichard Purdie2021-08-101-0/+1
| | | | | | | | | | | | | | | | When adding a layer which changed SSTATE_EXCLUDEDEPS_SYSROOT, the state hashes were changing when they should not. This was caused by wider use of setscene_depvalid which means the dependency on the variable was seen when it was previously not. Exclude the variable since this should be be included in the hashes. (From OE-Core rev: bae9c6482271d53dc28d3c801fba467e268003bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 09725a29365c69ccbd603fe3a1de72189f26d5ac) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: fix do_ar_configured failure for kernelChangqing Li2021-08-101-1/+4
| | | | | | | | | | | (From OE-Core rev: 224913ff067e6c26cabe0df0ecbf50fb03b85165) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d7776a23cbea836ddb8ac5ec77012af2449ab875) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* archiver.bbclass: Fix patch error for recipes that inherit dos2unix.leimaohui2021-08-101-0/+3
| | | | | | | | | | | | | do_unpack_and_patch error happens for these recipes inherit dos2unix. (From OE-Core rev: 3e8cb257546e00a988136dfa6d7f5e6b7f7f16d8) Signed-off-by: Lei Maohui <leimaohui@fujitsu.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2ceda7c90c0087f52693c54d5ccab143b27f4d21) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate: Drop pseudo exclusionRichard Purdie2021-07-201-2/+0
| | | | | | | | | | | | | | | | Now that pseudo-native always includes statx support and we have sanity checks on pseudo-native to ensure it always contains a minimum feature set, we no longer need to mark pseudo-native as distro specific. This fixes eSDK build problems. [YOCTO #14428] (From OE-Core rev: d7e87a5851d717da047f552be394d5712efa0402) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3279e43fcb469edb63c7c4eb60fdc565d5751f9d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* report-error: Drop pointless inheritRichard Purdie2021-07-151-2/+0
| | | | | | | | | | | The base class is always inherited, drop the unneeded inherit. (From OE-Core rev: cfd74f2bae51413d9c327e0f08ecf751325c2d74) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 837d62c066589e5a12a0bf894ae7b24e8b048665) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate/staging: Handle directory creation race issueRichard Purdie2021-07-102-6/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sstate code tries to be careful about racing around directory creation. In particular, the copyhardlinktree code creates the directory tree first allowing for "already exists" errors and ignoring them, then hardlinks the files in. Unfortunately the sstate removal code can race against this since it will try and remove empty directories. If there is some bad timing, a newly created directory can be removed before it was populated, leading to build failures. We could try and add locking but this would damage performance, we've been there before. It is also unclear where to actually place locks just based on the contents of a manifest file which may cover multiple sstate install locations for a given task. Instead, lets disable directory removal in the problematic "shared" core path. This could result in a few more empty directories being left on disk but those should be harmless and better than locking hurting performance or rare build races. [YOCTO #13999] [YOCTO #14379] (From OE-Core rev: fa49622521b6386d8031b1e7519f087aa9d99b19) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4f94d9296394bc7ce241439f00df86eb5912875f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate.bbclass: fix errors about read-only sstate mirrorsMichael Ho2021-07-101-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a read-only sstate mirror is used in conjunction with hash equiv, then OSError will be raised when an sstate-cache hit is achieved. This is because sstate_task_postfunc will try to "touch" the symlinks that point to the read-only sstate mirror when sstate_report_unihash has changed SSTATE_PKG. This commit adds an additional exception handler to silently mask read only rootfs errors thrown during the touch. The fix is also duplicated to sstate_eventhandler as the code is very similar but it may not be needed there. Example of the error: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_task_postfunc(d) 0003: File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function: sstate_task_postfunc 0770: 0771: omask = os.umask(0o002) 0772: if omask != 0o002: 0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" % omask) *** 0774: sstate_package(shared_state, d) 0775: os.umask(omask) 0776: 0777: sstateinst = d.getVar("SSTATE_INSTDIR") 0778: d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir']) File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package 0699: if not os.path.exists(siginfo): 0700: bb.siggen.dump_this_task(siginfo, d) 0701: else: 0702: try: *** 0703: os.utime(siginfo, None) 0704: except PermissionError: 0705: pass 0706: 0707: return Exception: OSError: [Errno 30] Read-only file system (From OE-Core rev: f2360f796016cce93610a080ff07c8047bda6ee8) Signed-off-by: Michael Ho <Michael.Ho@bmw.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 244b3be0358a66e0cca4016fe26144e3d7323390) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_pkgdata: Avoid task hash mismatches for generic task changesRichard Purdie2021-07-101-1/+1
| | | | | | | | | | | | | Add SSTATETASKS to package_prepare_pkgdata[vardepsexclude] since otherwise the task hashes vary depending upon which packaging backends are enabled and likely other changes which add/remove unrelated sstate tasks. (From OE-Core rev: fd94aa40e68189c41ef650d7fc9f4d4da686a4ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 4011d31d4372639fd72ee0eefae210bf59c90d13) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* kernel-devicetree: Fix interaction when packaging disabledRichard Purdie2021-07-101-4/+7
| | | | | | | | | | | | | When packaging is disabled using the nopackages class, ensure we don't add to PACKAGES. This fixes builds where we have an unpackaged kernel alongside a packaged kernel. (From OE-Core rev: 44756b6d92d922873fe1781e2cc1be3031cc5a86) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d6b114cf5a9b22688c0b59a3afc46a07450d87b5) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-11 20:24:38 +0000