summaryrefslogtreecommitdiffstats
path: root/meta/classes/package.bbclass
Commit message (Collapse)AuthorAgeFilesLines
* package: get_package_mapping: avoid dependency mapping if renamed package ↵Yann Dirson2020-09-101-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | provides original name Packages with a runtime dependency on a target package whose name is changed by the PKG_* mechanism must rebuild when that mapping changes, but we have no way of tracking this today, so eg. packagegroup-machine-base ends up with a relationship on a versioned kernel-image, and does not get rebuilt when that version changes, leading to unsatisfiable dependency and reproducibility issue. OTOH there is no reason for the dependency to get rewritten if the renamed package already has a RPROVIDES on the non-rewritten package name, and if the dependency relationship is an unversionned one. This is what this patch prevents. Note that this may not cover all cases of rewritten package names. Notably I had to let the rewrite be done in the case of versionned dependencies, as package managers usually can follow "Provides" in such case; this includes many dependencies against shared-lib packages renamed to their soname, and those at least are OK, since the dependent recipe should explicitly depend on the target recipe. (From OE-Core rev: 920beaaeef62b558e046f32c8ef0332250969ef1) (From OE-Core rev: 210ebed1e9c2285d6e457bf03d1f1a1f3ddc7fda) Signed-off-by: Yann Dirson <yann@blade-group.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: explode the RPROVIDES so we don't think the versions are ↵Ross Burton2020-09-101-1/+1
| | | | | | | | | | | | | | | | | | | | provides emit_pkgdata() creates symlinks for each of the RPROVIDES in pkgdata/MACHINE/runtime-rprovides. However this string can contain versions which results in directories called (=2.32), so pass the RPROVIDES string through bb.utils.explode_deps() to strip the versions out. Helps mitigate - but not solve - #13999. (From OE-Core rev: be88ca5c4bd0dd98d172a53727c7104d46349fdd) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6a5395dec34192db233bfb2a060e5ccc99708f03) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Sort shlib2 output for hash equivalencyMark Hatle2020-09-021-1/+1
| | | | | | | | | | | | | | The output was unsorted, so different versions of python, different input ordering could have have changed the files, and thus changed the hashes making the system think the output was different, even when unmodified. (From OE-Core rev: 07d15b83a998d6a80eba2cf8649486bfe302f94d) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 23cc846c92219c5cbe3fc6a0024579195d3cbd78) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/package: Use HOST_OS for runtime dependenciesJoshua Watt2020-07-311-5/+5
| | | | | | | | | | | | | | | | | | | | The code was using TARGET_OS to try and detect what type of OS the packages would be running on, but this is incorrect. TARGET_OS is the OS for which the package will generate output (e.g. GCC output target or GDB debugging target). The OS where the package will run is the HOST_OS. Note that HOST_OS only differs from TARGET_OS when doing a canadian cross compile, and even then in our case only when doing so for a non-Linux host (e.g. MinGW). Fix the code to use HOST_OS instead. (From OE-Core rev: 2541b5be1e9c2c33ca8017130461a19f8734d883) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f284c5b2df220b520b025a59874e04ef4becd829) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Correct variable dependeciesRichard Purdie2020-02-231-2/+4
| | | | | | | | | | | Ensure that the plain variable X is also added to the dependencies as well as X_<pkg>. Allow the funciton to be called with a different variable list too. (From OE-Core rev: c392401e10ff43d10b7e57c9d552522a02c91878) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Support stripping and debug copy of static librariesMark Hatle2020-02-081-3/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, we won't copy and strip static libraries. However, this functionality can be useful in some cases where people are doing development on the target, and don't generally want the larger debug capable static libraries. To enable the new functionality set: PACKAGE_DEBUG_STATIC_SPLIT = '1' Add a new function splitstaticdebuginfo. Thus function will copy the unmodified static library into the specific debug directory location. By keeping an unmodified version, it is possible for a user trying to debug something to use -L /usr/lib/.debug-static and their existing build commands to switch from stripped to full debug versions. The PACKAGE_DEBUG_SPLIT_STYLE will select between two different approaches, /usr/lib/debug-static or <path>/.debug-static. Additionally you can now choose to strip static libraries to conserve space. If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC' is set to 1, the static library will be stripped. (This is not on by default, as it could make diagnosing static library usage difficult in some cases.) Add to insane.bbclass a skip to the staticdev warning for the specific -dbg package versions. (From OE-Core rev: 17fa66c8199d73f0b59b2b3e609075933bf1e74b) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Allow INHIBIT_PACKAGE_STRIP_FILES to skip .ko and static libsMark Hatle2020-02-081-6/+7
| | | | | | | | | | | Change the order of the skip processing to happen before any .ko and static library processing. This will allow these types of files to be individually skipped if necessary. (From OE-Core rev: c50fcd54ffe60b63d042d05e6cf538a593dc410f) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Fix debug source processing for static librariesMark Hatle2020-02-081-1/+1
| | | | | | | | | | | | Format of the sources list is the [ (file, [source, ...]), ... ] before this change, the static libraries were processed but the items were included incorrectly causing no sources for static libraries to be included. (From OE-Core rev: fa356b23c2f4599681693bba50d36659b07a8125) Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Report only the licenses that are incompatiblePeter Kjellerstedt2019-12-161-2/+3
| | | | | | | | | | When excluding a package from being packaged due to incompatible licenses, report the license(s) that are actually incompatible. (From OE-Core rev: 7d5c07e4a32a0968942ae538023c2891b59d8ab5) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Always include ldconfig fragmentAndreas Oberritter2019-11-111-3/+1
| | | | | | | | | | Now that ldconfig may get installed from a feed, use it when it's available on the target. (From OE-Core rev: 0f09f19dc48edf2ac50b554c18c217b7f97d4ae3) Signed-off-by: Andreas Oberritter <obi@opendreambox.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Use with to manage file handle lifetimesOla x Nilsson2019-10-231-42/+34
| | | | | | | (From OE-Core rev: e90978056c4b49e138b3d422939bf995829ec3b0) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Multiple shlib_providers for the same file should errorJonathan Marler2019-09-271-10/+11
| | | | | | | | | | | | | | In the case where multiple packages provide the same file, we show an error. Otherwise, python will generate a different build depending on which provider appears first in the dictionary. On my system this order changes every time I run bitbake causing intermittent build differences. Add a sorted() to fix the determinism issue too. (From OE-Core rev: 61c41369003444bfbf1c45e7cfd2752a4b7bc22f) Signed-off-by: Jonathan Marler <johnnymarler@hp.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: allow shell-style wildcards in PRIVATE_LIBSAlexander Kanavin2019-09-061-2/+4
| | | | | | | | | | | | | | | | PRIVATE_LIBS is used to exclude 'private' libraries from getting added to automatic runtime dependency resolution. This variable currently has to list all libraries by name, which becomes a maintenance issue if the list of such libraries frequently changes, or is very large. This change allows using shell-style wildcards in the variable, similar to how FILES lists what gets packaged. (From OE-Core rev: 732db32714c208d8eeeb90308926dc886ef7b791) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Fix race between do_package and do_packagedataRichard Purdie2019-08-211-3/+7
| | | | | | | | | | | | | | | | do_package has PKGDESTWORK as a cleandir and do_packagedata has it as an sstate-input dir. This means do_package wipes out the directory at both do_package and do_package_setscene. do_package_setscene and do_packagedata_setscene can run in parallel when installing from sstate which means they can wipe out parts of each other leading to interesting build failures. We therefore have to add in a hardlink copy so that the directories can work independently of each other. (From OE-Core rev: 692b2046bb60806ee6bbde6ba15bd1637b00efb1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Improve determinismRichard Purdie2019-07-261-1/+1
| | | | | | | | | | Its possible in cases with multiple shlib providers we were not being deterministic. Add in a couple of sorted() calls to fix the shlibs and pkgconfig cases with this potential issue. (From OE-Core rev: 5b9a4214ee17e1a39dd5a1742f2ac5ed25a11310) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: check PKG_ variables before executing ontarget postinstBruce Ashfield2019-07-111-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a package uses PKG_ variables to map package names to version specific variants, on target postinstall functionality will be broken. i.e. something like the following casuses rootfs assembly errors: d.setVar('pkg_postinst_ontarget_linux-source', 'cd /usr/src/; ln -sf %s linux-source' % source_pkg) This breakage is due to the fact that the original package name (as specified by the PACKAGES variable) is logged by the intercept scripts, but the mapped / specific version is actually installed to the rootfs (and hence logged by the package manager). When the runtime listing of on-target scripts is performed, we get a package manager error due to a missing package, since it checks the generic version logged by the intercept scripts. We can fix this by ensuring that the PKG_ variable mapped package name is logged by the intercept phase, and hence the package manager can locate and execute the on target postinst script. This variable check is consistent with other places in the code, and has no impact if PKG_ variables are not used. (From OE-Core rev: a6af0886d1be584974086c0ddb4a5bc566eb7984) Signed-off-by: Bruce Ashfield <bruce.ashfield@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "pigz: Add debug for autobuilder errors"Richard Purdie2019-06-301-1/+1
| | | | | | This reverts commit b08976456c8ab7f29efd83644ce42746c0d6501b. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Build pkgdata specific to the current recipeRichard Purdie2019-06-301-13/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | This switches the code to build pkgdata specific to the current recipe which means that its filtered to the recipes dependencies and can perform better as we can drop the lockfile. It uses a similar method to the staging code to do this, using BB_TASKDEPDATA to construct a list of packagedata task output which this recipe should "see". The original pkgdata store is left unaltered so existing code works. The lock file was there to prevent files disappearing as they were read or as directories were listed. Since we have a copy of the data and only access output from completed tasks (as per their manifests), we can remove the lock. The lock was causing starvation issues on systems with parallelism. There was also a potential determinism problem as the current code could "see" data from recipes which it doesn't depend upon. [YOCTO #13412] (From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pigz: Add debug for autobuilder errorsRichard Purdie2019-06-301-1/+1
| | | | | | (From OE-Core rev: b08976456c8ab7f29efd83644ce42746c0d6501b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: fix directories setuid and setgid bitsJoël Esponde2019-06-281-1/+2
| | | | | | | | | | | | | | | populate_packages relies on ``mkdir`` to both create a directory and set its permissions. However, ``mkdir`` honors the ``umask`` value. Therefore, some bits may be lost in the operation. In our case, the setgid bit on the directories were lost. This commit fixes this by having a distinct call to create the directory and to set the permissions. (From OE-Core rev: 0f82b53a650e76e0129fae6ce7581a41d042315b) Signed-off-by: Jean-Tiare Le Bigot <jean-tiare.le-bigot@easymile.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Clean up writing of runtime pkgdata filesPeter Kjellerstedt2019-06-071-38/+18
| | | | | | | | | | This introduces a variable, PKGDATA_VARS, that contains the names of the variables that are to be output in the runtime pkgdata files. (From OE-Core rev: 43e55bfa040425cf93d94ac626a31f6fd00a7a74) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/package: Sort ELF file listJoshua Watt2019-06-041-0/+6
| | | | | | | | | | | Sorts the list of detected ELF files by path before processing. This ensures that when multiple files are hardlinked together the first one found is always the same. This is required to have reproducible builds. (From OE-Core rev: de86bfeda6e3845336a0b56c883b49219967128f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add source package to ${PACKAGES}Joshua Watt2019-03-061-13/+15
| | | | | | | | | | | | | | | | | | | | Permanently adds the -src source package to ${PACKAGES} instead of relying on creating it dynamically at packaging time. The source package is now defined in bitbake.conf, just like -dev and -dbg packages. For compatibility, the -src package is still added dynamically if it was missing, since some recipes have a tendency to completely override PACKAGES and do so without manually adding back the -src package. This allows RDEPENDS on the -src packages, which wasn't previously possible. [YOCTO #13203] (From OE-Core rev: b25e1edf0204fc2f64aa8d66e09b8e2d67b90e17) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Rework debug source file handlingRichard Purdie2019-02-171-23/+27
| | | | | | | | | | | | | | | | | Currently we parallel process the files we install running dwarfsrcfiles over each one in parallel threads but requiring a lock to write the results to one file. This is not ideal for performance and means we can't then use per file data for other purposes such as source code license processing. Rework the code so that the list of source files is generated per installed file and is reusable. The code still generates a null separated debugsources.list file since this is used by a shell pipeline but it no longer needs locking. (From OE-Core rev: 95de93988eb725c14102f642ebabff3920ae194f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Add pkg_postinst_ontarget to PACKAGEVARSRichard Purdie2019-01-261-1/+1
| | | | | | | | | | | Changes to pkg_postinst_ontarget were not triggering rebuilds, this fixes that. [YOCTO #13127] (From OE-Core rev: 432d5a5481bd8efb848b95fbe6500a72fba9ac65) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-11/+11
| | | | | | | | | | | | | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. Note that some show up as: """ meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.   """ where the problem isn't on 1293 in package.bbclass but in some _prepend to a package.bbclass function in a different file like mesa.inc, often from do_package_split() calls. (From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Make package output files more deterministicJacob Kroon2019-01-141-6/+6
| | | | | | | | | | | | | | | | | | | | | | | Observing depsig.do_package for packages inbetween rebuilds indicated that the following variables/files content was changing order randomly. Make them deterministic by sorting the output: RDEPENDS_<pkg> RRECOMMENDS_<pkg> FILERDEPENDSFLIST_<pkg> packages-split/<pkg>.shlibdeps The following variable was not observed to change, but it is assumed that the same situation can occur, so do the same sorting for consistency: FILERPROVIDESFLIST_<pkg> (From OE-Core rev: c99cb0bbb78089d1d15c4c8563a71db0df1cb0da) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Do not hide cpio's errorRobert Yang2019-01-111-1/+2
| | | | | | | | | | | | | | | | | | | | | We use subprocess.check_output() to run the command, which means that we need care about the error, so the 2>/dev/null should not be used, otherwise it is hard to debug when the error happens. I guess it was copied from previous lines, but that command's error can be ignored (excpet: pass): try: subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) except subprocess.CalledProcessError: # Can "fail" if internal headers/transient sources are attempted pass But we don't do this in the current location, so remove "2>/dev/null" (From OE-Core rev: 017a53d2743be2b5d4965a39b4e126fb74f700ad) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Sort FILES_INFO by keyJacob Kroon2019-01-111-1/+1
| | | | | | | | | | | | Observing depsig.do_package for a package inbetween rebuilds indicated that FILES_INFO was changing content order randomly. Force it to be deterministic by sorting with respect to the keys when serializing. Suggested-by: Joshua Watt <jpewhacker@gmail.com> (From OE-Core rev: b0364be65bf0b3dee44f81379f4062e9f707c128) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: fix python unclosed file ResourceWarningChen Qi2019-01-031-1/+2
| | | | | | | | | | | | Fix the following warning. ResourceWarning: unclosed file <_io.TextIOWrapper name='/.../systemd/1_239-r0/debugsources.list' mode='a' encoding='UTF-8'> (From OE-Core rev: 91810a57f0edd8b37c5f3f989a5aca69d9a40b37) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* allarch: only enable allarch when multilib is not usedKai Kang2018-09-131-3/+6
| | | | | | | | | | | | | | | | | | | | | Some allarch packages rdepends non-allarch packages. when multilib is used, it doesn't expand the dependency chain correctly, e.g. core-image-sato -> ca-certificates(allarch) -> openssl we expect dependency chain for lib32-core-image-sato: lib32-core-image-sato -> ca-certificates(allarch) -> lib32-openssl it should install lib32-openssl for ca-certificates but openssl is still wrongly required. Only enable allarch when multilib is not used to fix the issue. (From OE-Core rev: a23c482cab4f874f4a6a6889716123569eb5ece9) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: warn if perms conf file does not existChen Qi2018-09-041-1/+5
| | | | | | | | | | | | | | It's possible that the perms conf file specified by FILESYSTEM_PERMS_TABLES does not exist. Currently, this situation is silently ignored, which is likely to lead to further do_rootfs failures. So fix to output a warning, telling user that the specified file in FILESYSTEM_PERMS_TABLES cannot be found. (From OE-Core rev: e14b9f2a667889092251053933dc2f3c51f01ef0) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: only one hardlink of separated debug info file in each ↵Hongxu Jia2018-08-281-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | directory While multiple hardlinks of binary located in different dirs, there are also multiple hardlinks of separated debug info file with the same binary name in same debug dirs. But in each dir, only one debug file with original name works. Because all of binary hardlinks have one `.gnu_debuglink' which is added in `splitdebuginfo'. It caused gdb could not find debugging symbols. [Before the patch] $ find . ./usr/bin/foo ./usr/bin/foo-hd1 ./usr/bin/.debug ./usr/bin/.debug/foo ./usr/bin/.debug/foo-hd1 ./usr/libexec/foo-hd2 ./usr/libexec/.debug ./usr/libexec/.debug/foo-hd2 $ readelf --debug-dump usr/libexec/foo-hd2 Contents of the .gnu_debuglink section: Separate debug info file: foo $ gdb usr/libexec/foo-hd2 Reading symbols from usr/libexec/foo-hd2...(no debugging symbols found)...done. [Before the patch] [Apply the patch] $ find . ./usr/bin/foo ./usr/bin/foo-hd1 ./usr/bin/.debug ./usr/bin/.debug/foo ./usr/libexec/foo-hd2 ./usr/libexec/.debug ./usr/libexec/.debug/foo $ gdb usr/libexec/foo-hd2 Reading symbols from usr/libexec/foo-hd2...Reading symbols from usr/libexec/.debug/foo...done. [Apply the patch] (From OE-Core rev: d4eaf42f7708f8d3a31a04d958bd7420dd7dd6b9) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: use bb.utils.break_hardlinks helperRasmus Villemoes2018-08-231-1/+1
| | | | | | | | | | | | This does the same thing, but is more efficient in case st_nlinks is (already) 1. Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c . (From OE-Core rev: 38180b5c1044be13458fb927ad1babae61e4c51f) Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/package: Clean up getstatusoutputJoshua Watt2018-08-231-8/+3
| | | | | | | | | | Replaces usage of the deprecated oe.utils.getstatusoutput() with Python subprocess calls. (From OE-Core rev: 7f9d2d16b8cdff9cbba2b3965c74d1c5b8ab1106) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: improve -dbg and -src package orderingRasmus Villemoes2018-08-151-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | nativesdk-gpgme fails package_qa when setting PACKAGE_DEBUG_SPLIT_STYLE = "debug-with-srcpkg". ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug package contains .debug directory: nativesdk-python3-gpg path /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so [debug-files] This turns out to be because the automatic moving of the -dbg package to the beginning of the package list is disabled in that case, so the python3-gpg packages that the recipe prepends to PACKAGES ends up before the -dbg package. It's not clear why the "and not split_source_package" was added when debug-with-srcpkg was introduced. Presumably the intention was to prevent the -dbg package to end up before the -src package, which we of course need to. But at the same time, we still need -dbg packages to end up before all other packages. Using list.insert(0, ...) also means that if there happens to more than one -dbg package, their relative ordering gets inverted in the new list. This tries to fix these issues by sorting the packages by (priority, original position), where priority is 10 for -src, 30 for -dbg and 50 for everything else. That guarantees that packages of the same "type" preserve their relative ordering, while also ensuring that -dbg always preceed other packages. This scheme is also quite extensible, and, should the need arise, one could even expose the priorities as a knob the recipe author could use to ensure specific orderings of packages instead of the somewhat fragile and coarse-grained method of "prepend or append, and ensure you do that in a proper order". Probably the autodebug condition needs to stay, but I think the split_source_package condition in the preceding elif should be removed, so that that logic applies to all packages called -src, not just the one we might have created a few lines above. (From OE-Core rev: 805edbc7dc9ceae00d991f9b4e185bbbe1d3ba45) Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Fix file copying to preserve sparse filesRichard Purdie2018-08-141-1/+1
| | | | | | | | | We want to preserve sparse files when building the system, add the option to tar to ensure we do this when copying files. (From OE-Core rev: d86da5ae386a8129a966a53901de160823f4d250) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: Fix hardlink preservation issueRichard Purdie2018-07-311-1/+1
| | | | | | | | | Recent changes broke the preservation of hardlinks during processing due to a missing index. Fix this, reducing the size of the git recipe packages in particular (it contains many hardlinks). (From OE-Core rev: 28eeada955762f38ccbd1d26c53768364dbd1a5e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package.bbclass: append to FILERPROVIDES_*/FILERDEPENDS_* variables instead ↵Martin Jansa2018-07-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | of setting them * this allows to work around some of the issues with file-rdeps * e.g. in my case I have /usr/lib/libmali.so.0.1 which according to rpmdeps provides only following libmali* libmali.so(LIBMALI_1.0) libmali.so.0.1 but many users of this link with libmali.so directly and according to rpmdeps require libmali.so which causes file-rdeps for a lot of recipes * I was using simple work around to just set: RPROVIDES_${PN} = "libmali.so libGLESv2.so libEGL.so" but that doesn't work well with multilib, because the value gets expanded with MLPREFIX and RPROVIDES_${PN} = "lib32-libmali.so lib32-libGLESv2.so lib32-libEGL.so" doesn't help at all. [YOCTO #9217] Many unsolveable QA warnings from build-deps and file-rdeps (From OE-Core rev: 01b9810ff1cea53ae321f6fbe760b83573ed78c0) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/package: fix variable name in commentPaul Eggleton2018-07-261-1/+1
| | | | | | | | | | The variable is PACKAGE_WRITE_DEPS as seen in the default below the comment. (From OE-Core rev: 364449251ffe4ff2c11acaa258edcec244c38818) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Use subprocess to be consistent with the rest of the classRichard Purdie2018-07-251-4/+4
| | | | | | | | | Using 'sub' in one function just confuses things, standardise and fix formatting of the parameters too. (From OE-Core rev: a740c638148ed7236c49eed55ae9a15b94e55b9f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package/package_manager: multiprocess_exec -> multiprocess_launchRichard Purdie2018-07-241-2/+2
| | | | | | | | | | | | After this replacement, the parent exception handling works so we don't need subprocess wrapping with bb.error in the underlying functions. The underlying contexts also have better module handling so the imports can be cleaned up. (From OE-Core rev: aa8260adf53139d776a2affe6118d28b295c1fab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Clean up getstatusoutput usageRichard Purdie2018-07-241-13/+11
| | | | | | | | | | | | | | | | Replace usage of oe.utils.getstatusoutput() with direct subprocess calls. The call is just a wraper to the subprocess call of the same name and requires the caller to handle exceptions themselves. We usually do this badly, failing to show the output or the command or the return code. Its much safer to rely on a call like subprocess.check_output() instead. This also makes it easier to spot and remove cases where shell=True isn't needed in a later cleanup. (From OE-Core rev: 9f058857fb692f1251deb43bcaa7ed0120140093) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Call file to determine elf status in parallelRichard Purdie2018-07-241-35/+50
| | | | | | | | | | This allows the calls to is_elf (which calls file) to happen in parallel allowing a speedup of do_package and do_populate_sysroot for native recipes. (From OE-Core rev: bbe0d3e26484f3f347262d40a8a9d415ce21fb43) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Refactor to remove isElf/is_elf function duplicationRichard Purdie2018-07-241-36/+4
| | | | | | | | | | | There are probably further cleanups needed here but this at least removes the major code duplication between these two similar funcitons, keeping the kernel module ".ko" extension check for efficiency to avoid opening and reading file contents in the general case. (From OE-Core rev: 7ad0c0d6ab12bebeac097fc0f5210c876dcfe9be) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Allow parallel processing of shlib analysisRichard Purdie2018-07-241-17/+30
| | | | | | | | | | | This function is a bit more invasive to add parallelism to but allows the shlibs analysis to happen in multiple threads. In order to return values correctly/safely the data types needed tweaking to avoid lists and use immutable objects. (From OE-Core rev: b5788fb1f795f2f35d1788d8311e12984ffb2122) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Allow parallel processing of debug splittingRichard Purdie2018-07-241-11/+13
| | | | | | | | | | Adjust the code so that the splitting of debug symbols from files happens in parallel. To to this we need to move some path handling code into the main function and pass more parameters in. (From OE-Core rev: 9f0c2ed5d44a16e8268ac521236c4752f930f26a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Don't use subshell to execute fileRichard Purdie2018-07-241-6/+2
| | | | | | | | | We don't need any functionality from the shell here, its just extra fork overhead. Therefore remove it and use subprocess directly. (From OE-Core rev: bcc03ea19e103f6aa93bada2f49fcc5cc7bc0790) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Drop subshell usage for dwarfsrcfile generation.Richard Purdie2018-07-241-2/+10
| | | | | | | | | | | | The command for running dwarfsrcfiles is simple and does not need a subshell for each execution. By expanding out this function to use check_output() from subprocess and a list of arguments, the shell overhead can be dropped. For recipes with lots of files this gives a significant saving. (From OE-Core rev: 6334129dfbe266602fab70ce445641053a05be6c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Rework PACKAGELOCK based upon sstate for do_packagedataRichard Purdie2018-07-241-14/+9
| | | | | | | | | | | | | | | | | | I think this lock dates from before we had sstate for do_packagedata. Since WORKDIR is recipe specific and we write into WORKDIR, we no longer need any write locks in the do_packagedata code itself, its handled by the sstate task lock for the final copy in at the end. The final write lock can be simply removed. The only time we need read locking is when actually reading data from the shared directory. We can therefore reduce the window the lock is held significantly as well, hence improving the speed of packagedata tasks running in parallel. (From OE-Core rev: f7106cdf2190d9ec59132a1cb2bb431d653cd9c5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-24 12:19:22 +0000