summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe
Commit message (Collapse)AuthorAgeFilesLines
* sstatesig: Optimise get_taskhash for hashequivRichard Purdie2020-06-021-2/+11
| | | | | | | | | | | | | | | | With hashequiv the get_taskhash function is called much more regularly and contains expensive operations. This these don't change based upon hash in a given build, improve the caching within the function to reduce overhead. (From OE-Core rev: de98cfe3cde4b8d5f4b163b5fba3f129651ef06a) (From OE-Core rev: 4c7e12ee42ff6ab228c2d8aa23a8153ff0debd4b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* prservice.py: fix do_package with newer Python in Ubuntu 20.04Martin Jansa2020-05-071-0/+4
| | | | | | | | | | | | | | | | | | * with Ubuntu 20.04 which is using python 3.8 I'm seeing a lot of errors like: ERROR: libxml2-2.9.10-r0 do_package: Can NOT get PRAUTO, exception No module named '_sysconfigdata' not sure what caused this from python 3.8, but this seems to work * PRserv is enabled with: PRSERV_HOST = "localhost:0" (From OE-Core rev: 4b26eaf7152fb712aba47a0c746333578f58ee8d) (From OE-Core rev: c314c6695acca9ea68e38359fb03a94afa37bb2b) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c23d6f77994698e71d9a011cddec1237158b15ca) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: don't try to rm /var/lib/opkgJan Luebbe2020-04-171-2/+1
| | | | | | | | | | | | | | | | As opkglibdir starts with a /, os.path.join will ignore self.target_rootfs, leading to an attempt to remove /var/lib/opkg. This only fails if it exists on the host, explaining why this remained undiscovered for long. (From OE-Core rev: 71711f5f20fc3744be4c9188a75606f60d524ff9) Signed-off-by: Jan Luebbe <jlu@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fc974977cea389f54e7fc7de7b1c8fd3d8bafe58) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: avoid installing provided packages via aptJan Luebbe2020-04-171-2/+12
| | | | | | | | | | | | | | | | | | | | If there already is a package providing (and conflicting against) packages what should be installed, apt will try remove the conflicting package (target-sdk-provides-dummy) and any that depend on it (like apt and dpkg). This usually fails because of the protection of essential packages. In that case, no -dev/-dbg packages are installed to the SDK. Avoid this problem by checking which packages are already provided and removing them from the list to be installed. Also sort the list to make it easier to read when debugging. (From OE-Core rev: 978eeeb7c975441e5b05253a63b9d954af2b903b) Signed-off-by: Jan Luebbe <jlu@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3ffb339dd55f8ca7c952fd3390608510f772e19f) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: collect provided package names when using debsJan Luebbe2020-04-171-2/+8
| | | | | | | | | | | | | This is needed for a later change to avoid installing packages which are already provided by an installed package. (From OE-Core rev: 1bedfdf58d3ebd06126aa45c7bdc84e66c750725) Signed-off-by: Jan Luebbe <jlu@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ad72dfaaa2caf2c39d033dc1682f0bbbbe45dbbd) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: fix handling of last packageJan Luebbe2020-04-171-7/+1
| | | | | | | | | | | | | | | | | In commit 7d214b34e11dc57316ed5c1c7747c4601286f6d2, only the code in the for loop was modified to store the pkgarch value. The code used if there was no empty line at the end was not modified. Instead of fixing the duplicated code, remove it and just make sure that a final empty line is processed. (From OE-Core rev: 7dba11373af742d4e4924e14e8fd386d53ffddbd) Signed-off-by: Jan Luebbe <jlu@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a7b93c695b23d015607b179d98526b9b14c03d45) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: make sure to not remove packages in apt installJan Luebbe2020-04-171-1/+1
| | | | | | | | | | | | | | | apt install can decide to remove already installed packages if there are conflicts. Avoid this by explicitly specifying --no-remove. This will then cause a "E: Packages need to be removed but remove is disabled." message. (From OE-Core rev: 15790fa224f405652e8ccc93c01dee04a7259246) Signed-off-by: Jan Luebbe <jlu@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9605a488b55042add012e9aeef13ab3f4e70e6e5) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Improve debug output if getpwuid() failsRichard Purdie2020-01-281-2/+6
| | | | | | | | | | | | If getpwduid fails, we don't see which file it failed on which is key information to aid debugging. Print this information when exceptions are raised. (From OE-Core rev: 2bba0052597020ea887c84419440df11f9859283) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9d7a5219713af8117eda145052c6d9abdf577d8f) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Test cross/native hashserv method extensionRichard Purdie2020-01-281-0/+9
| | | | | | | | | | | | | | | | | We can have one taskhash which represents multiple native/cross sstate objects since they're stored by BUILD_ARCH or possibly host distro (or host gcc version). We need to put these into separate namespaces on hashserv since their outhashes will never match and we need deterministic lookups for the different namespaces. Use this extramethod option to handle this. This fixes several problematic failures on the autobuilder. (From OE-Core rev: 766b57a1f9030429b546276a537fbce3a175cc25) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2a76082363d189880613765ad339718e3614049d) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/terminal.py: fix gnome-terminal start behaviorTrevor Gamblin2019-10-081-1/+1
| | | | | | | | | | | | | | | | | | | [Bugzilla Bug 13201] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=13201 Newer versions of gnome-terminal (3.32.0 and up) are not starting as expected for commands e.g. "bitbake -c devshell zlib". This manifests as the instance appearing as a new tab rather than a new window. Fix this (and maintain new window preferred behavior) by changing the "-x" option to "--" as per the warning message, avoiding deprecated options: # Option “--command” is deprecated and might be removed in a later version of gnome-terminal. # Use “-- ” to terminate the options and put the command line to execute after it. (From OE-Core rev: d5fd205239c8e3a1d68649562a7e91c8fbbc805e) Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/lsb: Make sure the distro ID is always lowercasedPeter Kjellerstedt2019-09-301-4/+4
| | | | | | | | | | | | | | | | In commit 8689e561 (lib/oe/lsb: attempt to ensure consistent distro id regardless of source), the distro ID returned by oe.lsb.distro_identifier() was lowercased, but only if a release version is also present. This changes the code to always lowercase the distro ID, including the default distro ID "unknown", which is used if no other ID can be identified. (From OE-Core rev: 4ba7ef79d23a4cf688d7a794064893fe5f2f473b) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/sstatesig: Fix class inheritance problemsRichard Purdie2019-09-301-7/+8
| | | | | | | | | | | | The locked sigs class needs to be inherited after the hashequiv mixin so that get_unihash can correctly wrap the underlying hashequiv function. To do this turn the locked sigs class into a second mixin, then the order can be correctly handled. Tweak the get/set_taskdata to match. (From OE-Core rev: 73d3414ecf594e426e6024b71931038a201eca52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Fix hash equivlanency locked signature issuesRichard Purdie2019-09-301-4/+10
| | | | | | | | | | | | | | | | | | | | | | Using locked signatures with the hash equivalency server ran into problems. We need to: a) Ensure the lockedhashes data object is passed from the core to any individual tasks using the get/set_taskdata methods b) Return a locked singature instead of a unihash c) Write the unihash being used to locked signature lists rather than the calculated taskhash d) Skip warnings of hash mismatch if the hash is a unihash These changes fix esdk builds (which use locked sigs) when a hash equivalence server is in use. (From OE-Core rev: 25dc3d78de01dffa77a3a2452d6a97d741b446d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdk: Install nativesdk locales for all TCLIBC variantsKhem Raj2019-09-271-4/+0
| | | | | | | | | | | | | | | | | | | | | install_locales() here is actually operating on nativesdk and only glibc is the default library for nativesdk, since thats what most of desktop/server distros use, therefore bailing out based on TCLIBC is not needed here, since nativesdk-glibc would be required for all non-glibc targetting SDKs as well. Fixes SDK install time error ERROR: OE-core's config sanity checker detected a potential misconfiguration. Either fix the cause of this error or at your own risk disable the checker (see sanity.conf). Following is the list of potential problems / advisories: Your system needs to support the en_US.UTF-8 locale. ERROR: SDK preparation failed (From OE-Core rev: ba2df1fa43b79c4959911c1b0bcad996f255b7a8) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdkJaewon Lee2019-09-191-2/+6
| | | | | | | | | | | | | | | | | | | | | | When doing a devtool build-sdk from within an esdk all nativesdk components would be rebuilt. This patch introduces SDK_INCLUDE_NATIVESDK flag to toggle the inclusion of nativesdk packages when creating the esdk sstate Currently locked-sigs.inc is generated during do_sdk_depends which doesn't pull in nativesdk packages. Generating another locked-sigs.inc in do_populate_sdk_ext and pruning it to only nativesdk* packages by using a modified version of the already existing function prune_locked_sigs and merging it with the current locked-sigs.inc Also adding SDK_INCLUDE_NATIVESDK tasklistfn to the logic surrounding setting tasklist file to not prune esdk sstate during creation [YOCTO #13261] (From OE-Core rev: d046afd12e1c209b29dca6ba402b9aa14680c5ce) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Remove SSTATE_HASHEQUIV_SERVERJoshua Watt2019-09-181-6/+2
| | | | | | | | | | Removes all references to the SSTATE_HASHEQUIV_SERVER variable. This variable is redundant now that BB_HASHSERVE is present. (From OE-Core rev: 54b3adb6bc90a8e4b9e92952688772ee074d36e5) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Update server URIJoshua Watt2019-09-181-1/+1
| | | | | | | | | | The server no longer uses a "http://" URI, since it has been updated to use a different protocol. (From OE-Core rev: 519561172c48bc7f7a61a3d02edd418fc0895b7b) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classextend.py: don't extend file for file dependencyChangqing Li2019-09-161-0/+2
| | | | | | | | | | | | | | | | | | | | | | | Fix error like: lib32-e2fsprogs-1.45.3-r0 do_package_qa: QA Issue: /usr/sbin/e2scrub_all contained in package lib32-e2fsprogs-e2scrub requires /bin/bash, but no providers found in RDEPENDS_lib32-e2fsprogs-e2scrub For some lib32 packages(eg: lib32-bash, lib32-sed) which probvides files, extend is not needed Eg: RPROVIDES of lib32-bash expects to have /bin/bash, with original extend, it will become lib32-/bin/bash, then will cause above error Fix by don't extend file dependency, and skip multilib check for file dependency in do_package_qa to avoid error like: WARNING: lib32-bash-5.0-r0 do_package: QA Issue: lib32-bash package lib32-bash - suspicious values '/bin/bash /bin/sh' in RPROVIDES [multilib] (From OE-Core rev: a9163120ed52534e7dbf4db50dc2b03bbf69f06b) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/gpg_sign.py: Remove unwanted importVijai Kumar K2019-09-061-1/+0
| | | | | | | | | | Remove unwanted import of oe.utils. (From OE-Core rev: 42e9de68384fb24d5c9a5f161810be90c45942d2) Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Fix leftover splitting issue from siggen changeRichard Purdie2019-08-121-1/+1
| | | | | | (From OE-Core rev: 8b76048da99ec3be7d763b7ccc81f767c7015bc7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Updates to match bitbake siggen changesRichard Purdie2019-08-062-33/+23
| | | | | | | | | | | | | Update the metadata to correspond to the bitbake siggen task specification format change. This standardises on "<fn>:<task>" everywhere rather than the "." delimiter that was being used in some places. This is an API breaking change but means we now have a consistent format being used throughout the codebase without compatibility APIs. (From OE-Core rev: 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstate/sstatesig: Update to new form of BB_HASHCHECK_FUNCTIONRichard Purdie2019-08-061-6/+7
| | | | | | | | | Bitbake has updated to a cleaned up form of BB_HASHCHECK_FUNCTION, adapt to this cleanup. This is an API breaking change. (From OE-Core rev: a4d413d8d809132b0e0a5dd673a36e2bd0e0be4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Update to handle BB_HASHSERVERichard Purdie2019-08-061-2/+6
| | | | | | | | | Update the code to handle BB_HASHSERV, an autostarted bitbake internal hash equivalency server suited to local developer usage. (From OE-Core rev: 9b3d7b0b0ff27281391c2b8f2511d312d6995ed6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Adpat to recent bitbake hash equiv runqueue changesRichard Purdie2019-08-061-2/+0
| | | | | | | | | | Upstream bitbake now hanes preserving the unihash data itself, drop this usage of persist_data which was extremely problematic due to concurrent task access issues, particulary on loaded systems. (From OE-Core rev: 034d91c2c94b201797a7830b0af6141132f9bad1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Add debug for incorrect hash server settingsRichard Purdie2019-08-061-0/+4
| | | | | | | | | If the hash server settings are incorrect, show the user useful error messages instead of tracebacks. (From OE-Core rev: 2df5d95b9b63e30ddaa7c72a4173b9a05b3f15f9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Move unihash siggen code to bitbakeRichard Purdie2019-08-061-162/+1
| | | | | | | | | | | | This code is closely tied with the hash server in bitbake and also means we can't relibably test the hashserv runqueue functionality without OE metadata. Moving this to bitbake as a MixIn class makes most sense and encourages code collaboration and reuse as well as enabling easier and more accurate testing of the APIs. (From OE-Core rev: a2a9c6092d4dde706ed071b08a972d1d87184295) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Remove bb.build.FuncFailedRichard Purdie2019-08-063-5/+4
| | | | | | | | | | | | | Whilst seemingly a good idea, this exception doesn't really serve any purpose that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't pythonic. Its not used in many places, lets clean up those and remove usage of it entirely. It may ultimately be dropped form bitbake entirely. (From OE-Core rev: efe87ce4b2154c6f1c591ed9d8f770c229b044ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package: Improve determinismRichard Purdie2019-07-261-1/+1
| | | | | | | | | | Its possible in cases with multiple shlib providers we were not being deterministic. Add in a couple of sorted() calls to fix the shlibs and pkgconfig cases with this potential issue. (From OE-Core rev: 5b9a4214ee17e1a39dd5a1742f2ac5ed25a11310) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager: Ensure the base-feed directory existsAlistair Francis2019-07-181-0/+2
| | | | | | | | | | Ensure that the /etc/opkg directory exists before we try to create a file there. (From OE-Core rev: 30ff50223cd0b79fd3b8aa393ea1e621282773ac) Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/copy_buildsystem: move layer into layers directoryAndrej Valek2019-07-181-1/+7
| | | | | | | | | | | | | | | | | | | | | | | Layers could be located outside from poky but inside the build directory. This case should be covered in eSDK. meta-abc meta-def/meta-ghi meta-def/poky meta-def/meta-oe/meta-oe ... It should take all enabled layers and put them into 'layers' dir during build-time with respecting new relative path to poky. layers/meta-abc layers/meta-ghi layers/poky layers/meta-oe/meta-oe ... (From OE-Core rev: 55ecf6988d3e3c0935cb6324a6ad2c75f1191a1d) Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory_analysis: ignore ownership for sysroot diffsRoss Burton2019-07-051-7/+11
| | | | | | | | | The sysroot isn't populated under pseudo, so ownership differences should be ignored. (From OE-Core rev: 01b816be4adff8f3992c1369810bdcf11a26fd6c) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: report sysroot changesRoss Burton2019-06-281-1/+10
| | | | | | | | | Now that the sysroot is written into the build history, write it out. (From OE-Core rev: e9df98ba63e6d2baefee550170dbdd11ed2ad03a) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rootfs: Fix dependency for every dpkg runRicardo Ribalda Delgado2019-06-191-0/+1
| | | | | | | | | | | | | | | | | Avoid getting a warning on do_rootfs. Fixes: NOTE: Installing the following packages: libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb WARNING: Unable to install packages. Command '/workdir/build/tmp/work/qt5122-poky-linux/bottlecam-image/1.0-r0/recipe-sysroot-native/usr/bin/apt-get install --force-yes --allow-unauthenticated libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb' returned 100: Reading package lists... Building dependency tree... Reading state information... (From OE-Core rev: 4e315dd23c52b75fd64dca89e7bffe25a0b0e6f9) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* multiconfig: Adapt to bitbake switch 'multiconfig' -> 'mc'Richard Purdie2019-06-101-1/+1
| | | | | | (From OE-Core rev: 8a6f7c1e455156966f467008645fef14db679ccf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gpg_sign/selftest: Fix secmem parameter handlingRichard Purdie2019-06-071-17/+22
| | | | | | | | | | | | | | | | | | | | | | | | | We keep seeing "cannot allocate memory" errors from rpm when signing packages on the autobuilder. The following were tried: * checking locked memory use (isn't hitting limits) * Restricting RPM_GPG_SIGN_CHUNK to 1 * Limiting to 10 parallel do_package_write_rpm tasks * Allowing unlimied memory overcommit * Disabling rpm parallel compression and the test still failed. Further invetigation showed that the --auto-expand-secmem wasn't being passed to gpg-agent which meant the secmem couldn't be expanded hence the errors when there was pressure on the agent. The reason this happens is that some of the early gpg commands can start the agent without the option and it sticks around in memory so a version with the correct option may or may not get started. We therefore add the option to all the key gpg calls. (From OE-Core rev: c7e131a76e522503df55e211dd261829feacfa28) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/reciputils.py: support character `+' in git pvHongxu Jia2019-06-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | While pv contains char `+' (such as ncurses 6.1+20181013), it was incorrectly converted to `6.1' In this commit: - Convert [^\+]* to .*? Support pv to contain char `+' and not greedy match (Such as ncurses 6.1+20181013) - Add [^\+] to sfx match Support sfx contains extra chars between `+' and `git' (such as asciidoc "8.6.9+py3-gitAUTOINC+618f6e6f6b") - Make sfx and rev greedy match Run `devtool check-upgrade-status --all' in poky and compare results, only one difference on ncurses version: Without the commit: INFO: ncurses 6.1 6.1+20181013 Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 With the commit: INFO: ncurses 6.1+20181013 MATCH Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 (From OE-Core rev: 8049bd34b89e710f7bb20883813ba3f929d9e997) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patch: add minver and maxver parametersRoss Burton2019-05-091-0/+11
| | | | | | | | | Add minver/maxver parameters to limit patch application by comparing PV. (From OE-Core rev: 3fac9f884ac6bcc0280e1bb5d0f0e397bb53678f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-0930-1/+105
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe.path: Add copyhardlink() helper functionPaul Barker2019-05-071-1/+12
| | | | | | | | | | | | | | This function creates hard links if possible, falling back to copying the file if the destination is on a different volume to the source. The docstring for copyhardlinktree() is also updated to make the difference between the two functions a little clearer. (From OE-Core rev: 5437efa16f9bec914e417c6c939a39c247084f52) Signed-off-by: Paul Barker <paul@betafive.co.uk> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils/multiprocess_launch: Improve failing subprocess outputRichard Purdie2019-04-261-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Output before this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.: Traceback (most recent call last): File "/media/build1/poky/meta/lib/oe/utils.py", line 272, in run ret = self._target(*self._args, **self._kwargs) File "/media/build1/poky/meta/classes/package_ipk.bbclass", line 230, in ipk_write_pkg shell=True) File "/usr/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Note how stdout/stderr from the failing command isn't shown. After this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:Foobar *** Error: Package name Foobar contains illegal characters, (other than [a-z0-9.+-]) opkg-build: Please fix the above errors and try again. We suddenly get a much more usable error message. The traceback is supressed as its distracting from the real problem in this case. Ideally python itself would handle this but it doesn't so we have to wrap the exception. We already do this in bitbake itself for the same reason. (From OE-Core rev: 09276dc76a8bda237b0b0b6d117a1980ae9dbfcc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: call a dependency parser only on actual dependency listsAlexander Kanavin2019-04-251-1/+1
| | | | | | | | | | Previously it was also called on filelists and possibly other items which broke the parser. (From OE-Core rev: 90bbe1bbc1667bf836d93df1e1ecca0c43315d06) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patch/insane: Rework patch fuzz handlingAndreas Müller2019-04-091-15/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently there are three issues which can be enhanced: 1. Fuzz warnings cannot be configured as errors for hardening. It happened often to me that these warnings were overseen and detected after commits were already out. 2. The output is too verbose - particularly when more than one file is affected. Meanwhile all users should know why patch fuzz check is performed. So move links with background information to insane.bbclass. 3. Reduce copy & paste effort slightly by printing PN (nit: <recipe> was not a correct suggestion e.g for native extended recipe - see example below) To achieve patch.py drops patch-fuzz info encapsulated by a header- and footer- string into log.do_patch. With this insane.bbclass can drop warnings/errors depending on 'patch-fuzz' in ERROR_QA or WARN_QA. Default remains unchanged: Spit out warnings only. A message for two fuzzed patches and 'pact-fuzz' in ERROR_QA now looks like: | ERROR: autoconf-native-2.69-r11 do_patch: Fuzz detected: | | Applying patch autoreconf-exclude.patch | patching file bin/autoreconf.in | Hunk #1 succeeded at 73 with fuzz 1 (offset -3 lines). | Hunk #2 succeeded at 143 (offset 6 lines). | Hunk #3 succeeded at 167 (offset 6 lines). | Hunk #4 succeeded at 177 (offset 6 lines). | Hunk #5 succeeded at 281 (offset 15 lines). | Hunk #6 succeeded at 399 (offset 15 lines). | Hunk #7 succeeded at 571 (offset 20 lines). | Hunk #8 succeeded at 612 (offset 20 lines). | Hunk #9 succeeded at 636 (offset 20 lines). | Hunk #10 succeeded at 656 (offset 20 lines). | Hunk #11 succeeded at 683 (offset 20 lines). | | Applying patch autoreconf-gnuconfigize.patch | patching file bin/autoreconf.in | Hunk #1 succeeded at 55 with fuzz 1 (offset -3 lines). | Hunk #3 succeeded at 663 (offset 18 lines). | | The context lines in the patches can be updated with devtool: | | devtool modify autoconf-native | devtool finish --force-patch-refresh autoconf-native <layer_path> | | Don't forget to review changes done by devtool! | | ERROR: autoconf-native-2.69-r11 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz] (From OE-Core rev: c762c0be43a3854a43cb4b9db559b03126d50706) Signed-off-by: Andreas Müller <schnitzeltony@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Make prune_suffix prune a suffixAndre Rosa2019-04-091-3/+3
| | | | | | | | | | | ... instead of replacing a substring that could happen more than once and not only when it ends with it. Do the same for the prefix. See related https://github.com/openembedded/bitbake/pull/24 . There it stops replacing sufixes once first one is matched but not here. (From OE-Core rev: 610ac84170f8a91cc3321edfc336a9e39f24ebe3) Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: added sh_quote() functionEnrico Scholz via Openembedded-core2019-04-041-0/+3
| | | | | | | | | | | This function is a wrapper around "shlex.quote()" and can be used in "${@...}" context where shlex (or pipes, which provides similar functionality) is unavailable. (From OE-Core rev: 127141f5023a7e3fc3963dc7d76cfce9067a9e8a) Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Support kmeta directory usage with devtool modify/finishJaewon Lee2019-03-061-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using Kmeta directories, devtool finish will add every single file in the directory to the bbappend. This is because in the current implementation, the get_recipe_local_files function treats the kmeta directory like a file. Modifying the function to loop through the provided directories and return all included files instead of just the top level directory. This will enable correct file to file comparison when determing which files are new/changed and need to be added to the bbappend. Adding an extra check in devtool-source.bbclass to not copy the cfg file if its already included somewhere in the kmeta directory Also during 'modify', when moving necessary files in the kmeta directory from the workdir to oe-local-files, the dangling parent directories are left behind. This in itself is not an issue as the temporary devtool workspace is automatically deleted, but this causes an incorrect include directory to be added in kernel-yocto.bbclass. Changing the order of the if statements to catch the correct conditional. This is safe to do as when not in the devtool context, there will be no oe-local-files directory. (From OE-Core rev: 4671011b5b02d0989aa0fdcb50e18cb385a0e95e) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager.py: turn nativesdk postinst warnings into notesAlexander Kanavin2019-02-251-2/+2
| | | | | | | | | | | | | | | | | | | | | The warnings deal with two specific cases of failure: 1) unable to execute nativesdk postinsts for mingw SDKs because they need to be run under wine 2) unable to execute target postinsts when there is no qemu usermode support for the target Neither of these should be a big problem as mingw issue was never found to problematic, and target postinsts deal with things that are needed at runtime and not at build time which is the purpose of SDKs. The specific reason to do this is to reduce the amount of warnings shown by the Yocto autobuilder, to zero eventually. (From OE-Core rev: cbc32fcd9b52e750600cce9dd84b33e3ce612eae) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Fix hang in multiprocess_launch()Richard Purdie2019-02-171-0/+4
| | | | | | | | | | | | | | If large results values are returned by the subprocesses, we can hit a deadlock where the subprocess is trying to write data back to the parent, the pipe is full and the parent is waiting for the child to exit. Avoid this by calling the update() method which would trigger reading a result from the child, avoiding the deadlock. The issue is described in https://bugs.python.org/issue8426 (From OE-Core rev: 0035e8066ecbbff94d6a1994a9f72b1368d660d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe,oeqa/selftest: Fix DeprecationWarning: invalid escape sequenceRichard Purdie2019-02-161-4/+4
| | | | | | | | | Fix another load of regex escape sequence warnings for newer python versions. (From OE-Core rev: bd2c125bb9c362b6122e99dfdf4e1cfe12c26a90) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: add -src to system package name blacklistRoss Burton2019-02-151-1/+1
| | | | | | | | | | | oe.utils.packages_filter_out_system() returns PACKAGES after removing "system" packages but it doesn't handle ${PN}-src as generated by PACKAGE_DEBUG_SPLIT_STYLE=debug-with-srcpkg. (From OE-Core rev: 162632d3d1e40c83ed9c5d49a026edf3912860a0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONSAlejandro del Castillo2019-02-082-41/+2
| | | | | | | | | | | | Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to deinstalled and pinned). This is brittle, and not consistent across the different solver backends. Use new --add-ignore-recommends flag instead. (From OE-Core rev: 0d11e813ba9b4e8de9e6e5099ff85f5d914243bc) Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-10 14:46:05 +0000