summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: utils: Update to use exec_module() instead of load_module()Richard Purdie2022-01-171-2/+5
| | | | | | | | | | | | This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. (Bitbake rev: e84c9aa77b61e48f1507edcba7bde65295f4d3ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 68a18fbcb5959e334cf307d7fa8dc63832edb942) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/command: Add a dummy event for tinfoil testingRichard Purdie2022-01-072-0/+15
| | | | | | | | | | | | | We need a command genetating an event to test through the tinfoil API. The current test has IO load issues so add a dummy version which won't have the IO constraints. (Bitbake rev: c5969eedd035648e3258bade386cc67ce3bb0e03) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a144178584394ea0700ffc2f2bfac94e8690effc) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bblayers/action: When adding layers, catch BBHandledExceptionRichard Purdie2021-12-051-2/+2
| | | | | | | | | | | | | | When adding a layer, parse error can occur, raising BBHandledException. Catch this and error, aborting the layer add to meet user expectations. [YOCTO #14054] (Bitbake rev: f18b65d0b9a6b983d53bde491e1bf2ca56949444) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ceddb5b3d229b83c172656053cd29aeb521fcce0) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure reparsing is handled correctlyRichard Purdie2021-12-051-0/+2
| | | | | | | | | | | | | | | | | | >From tinfoil, if you edit bblayers.conf and break it, then call parseConfiguration (e.g. by adding a bad layer with bitbake-layers), the system doens't show any parse error yet it should. Add in a call to the updateCache function so that things really are reparsed when requested. Partially fixes [YOCTO #14054] (Bitbake rev: a6ad6eaf4cddbe45077d1e6cb667f71602b8a49d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e655f9361b9c3b77906b8e06b5cc76bc5180640e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process/knotty: Improve early exception handlingRichard Purdie2021-12-022-5/+23
| | | | | | | | | | | | | | | | | | | | | | | The new server startup code means exceptions can happen when we aren't setup to show them to the user correctly, leading to ugly tracebacks. Add in some special case handling of BBHandledException to at least ensure that common case doesn't traceback and the user sees meaningful output. In the future, the logging setup can likely be improved, as can the way runCommand handles exceptions, they all should likely become real exceptions again on the UI side. [YOCTO #14022] [YOCTO #14033] (Bitbake rev: 7fdd43c5cbde38daa013076de2fdedcf3c3d3107) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6059d0e77f60ddb679049bd34478f41b1ab7995d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: When sending commands we need to process eventsRichard Purdie2021-12-021-1/+10
| | | | | | | | | | | | | | | | The server may be displaying useful information for the user through log messages so we should display anything that has been sent. Its either this or expecting every UI to implement this code around every command call which isn't good API. [YOCTO #14054] (Bitbake rev: f20da5247dea524e837c5b6fdeccc79cbafedf90) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 64ae9d7e2fad804dd9e12706c6d76b4b22f9586b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: Ensure exceptions inheriting from BBHandledException are ↵Richard Purdie2021-12-021-1/+5
| | | | | | | | | | | | | | | | | visible Previous changes allowed BBHandledException to be detected but not exceptions which inherit from it. Fix this. The code really needs totally reworking to preserve the exceptions. [YOCTO #14054] (Bitbake rev: 80348b68a34b7ec45a0496a4af7f2ae0c26488f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ef762d92df6c2554c6248e80212f984d9ec4c651) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: Add timeout for checkstatus calls (30s)Richard Purdie2021-11-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | We had an issue where a webserver serving sstate had filesystem issues so would accept connections but effectively not do anything with them. This causes bitbake to hang whilst processing things like sstate objects inside the checkstatus() calls. It can be replicated by setting up a server like: socat -u TCP4-LISTEN:NNN,fork OPEN:/dev/null and pointing SSTATE_MIRRORS in OE at that address. Adding a timeout to the checkstatus calls of 30s means that whilst the system will pause, it will then continue and not hang entirely. Since there isn't a large transfer here, 30s should be a reasonable response time after which we should fall back to building things ourselves. [YOCTO #13716] (Bitbake rev: c0348de8121c3a842bf44906f7e2f79e93f7275b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update pcre.org address after github changesRichard Purdie2021-11-091-1/+1
| | | | | | | | | | | | | vcs.pcre.org was a redirect to github which we use for subversion testing. With the protocol changes at github and the removal of the redirect, use a direct address for github. (Bitbake rev: fa471399d41efdf61e95e0be541b45f0621756f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6230ca71eb7eb2a6db162e28a01727d00af5299b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update github urlsRichard Purdie2021-11-081-13/+13
| | | | | | | | | (Bitbake rev: f1a3e9d22b5f4fb01c4a0e4ba03afb1afbba47f1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 07fca7e3ab696ba985b3ef86ab9031d688bf2df2) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch2: Fix quoting warningRichard Purdie2021-11-081-6/+6
| | | | | | | | | | | | | | | Fix: lib/bb/tests/fetch.py:1288: DeprecationWarning: invalid escape sequence for several lines of the fetch tests. (Bitbake rev: bd8883d756328ca4c8f6bf97f77e17133a6bfb45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9d84fd557a3fcbae2cdd70b24e69325ad737a01e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Handle github dropping git:// supportRichard Purdie2021-11-031-0/+4
| | | | | | | | | github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: c222eddcebe892ae209aea7776cfc1147ac1df6e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Use os.rename instead of mvRichard Purdie2021-10-231-4/+4
| | | | | | | | | | | | os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: c5b8a2fce98c362ea77d74a8bc472d01b739a98a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Avoid races over mirror tarball creationRichard Purdie2021-10-231-2/+19
| | | | | | | | | | | | | | | | | There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: e3da0ecbd282da060b52a4bcf3ed36497295fde0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: let asyncio discover the running loopJustin Bronder2021-10-231-2/+2
| | | | | | | | | | | | | | | | | | | >From 3.10 documentation [1]: Deprecated since version 3.8, removed in version 3.10: The loop parameter. This function has been implicitly getting the current running loop since 3.7 This is fixed in master as a side-effect of cf9bc0310b0092bf52b61057405aeb51c86ba137 which is more intrusive but likewise drops the loop parameter. 1. https://docs.python.org/3/library/asyncio-stream.html#asyncio.open_connection (Bitbake rev: 74a1e71b1e677a482fdedc685a71a1798ad63920) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: correct deprecation warning in process.pyAlexander Kanavin2021-10-231-1/+1
| | | | | | | | | | | (Bitbake rev: 1c422329c3b88a91a6faeacbba3b6104d66ba7a1) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit aff52fe21a0b27f6302555c1e52a864550eb46ce) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: adjust parser error check for python 3.10 compatibilityAlexander Kanavin2021-10-231-1/+1
| | | | | | | | | | | | | | The change was introduced in https://github.com/python/cpython/commit/a698d52c3975c80b45b139b2f08402ec514dce75 (Bitbake rev: e0d6af2f6885c93f94c7fbcaeb7b0336c05f02b0) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8d3c6cbbe6ee734495713ae3b99c609527842506) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: do not import imp in layerindexlibAlexander Kanavin2021-10-231-1/+0
| | | | | | | | | | | | | The module is deprecated and unused. (Bitbake rev: 2dca6a22cb7b911c905a10158f1a3c3258de5c11) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 76c9030d6c91cd776a1aa732fb92b7cd4539b503) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fix regexp deprecation warningsAlexander Kanavin2021-10-231-3/+3
| | | | | | | | | | | | | | See here for details: https://docs.python.org/3/library/re.html (Bitbake rev: d5ac4af74ba62a27232ff16931e8f3b22fc43112) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 660e6ad4abb77c6f3c1d48bd64777dd76c05d7e2) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: correct the collections vs collections.abc deprecationAlexander Kanavin2021-10-232-3/+4
| | | | | | | | | | | | | This becomes a hard error in python 3.10. (Bitbake rev: 4b66d498eab22eca29c9e3a7fdfa5c6e54055e78) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ae219e1f7460077f4492b31ac91cef4cf9b17277) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: compat.py: remove file since it no longer actually implements anythingChris Laplante2021-10-234-30/+21
| | | | | | | | | | | | | | Now that compat.py just imports Python standard library stuff, get rid of the layer of indirection. (Bitbake rev: 2a40a776efd7c28fa93a8556accccb79e3a7d0f0) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e2be6defbb9fcf25f9df04c3b452d0dba48dfd03) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: Update urls to match upstream branch name changesRichard Purdie2021-10-191-3/+3
| | | | | | | | | | (Bitbake rev: 93094705c1c66688e0730a1efb17805778c9fa9f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 473e2a5486bd972ad0f808db089abcb8945d3a48) Signed-off-by: Justin Bronder <jsbronder@cold-front.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/msg: Cleanup verbose option handlingRichard Purdie2021-10-026-18/+15
| | | | | | | | | | | | | | | The levels of indirection to set these verbose logging options is rather crazy. This attempts to turn things into two specific options with much more specific meanings. For now its all still controlled by the commandline verbose option and should funciton as previously, with the addition that the BB_VERBOSE_LOGS option can now be task specific. (Bitbake rev: b4117231bf070703b9375af4411bcd160e07fdae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 423c046f2173aaff3072dc3d0882d01b8a0b0212) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Show a readable error for invalid multiconfig nameRichard Purdie2021-10-021-0/+2
| | | | | | | | | | | | | | | | | If a multiconfig starts with a digit, users would see pages of errors as we use the multiconfig as a python function name prefix and python functions cannot start with a digit. We could avoid doing that but it is easier just to ask users to name multiconfigs not starting with digits. This tweak ensures the user sees an easier to understand error. (Bitbake rev: 34301f8a38078c2329e460051a1193c0314bcfd2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f9cddaeef35b2ea0dadf717101ed896f6b857abd) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Improve error handlingRichard Purdie2021-10-021-4/+6
| | | | | | | | | | | | | | | If bitbake-worker fails, return an error code showing that. Also make the thread cleanup code explict in a finally clause as it would otherwise hang. [YOCTO #14393] (Bitbake rev: 97541440e982848ef8bdbced22decdc24eda855b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7e0af70fb53fb13f824ca954b8cc1dffee730233) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Show error for no BBLAYERS in bblayers.confRichard Purdie2021-10-021-0/+3
| | | | | | | | | | | | | | If there is no BBLAYERS set in bblayers.conf show a more helpful error and exit. [YOCTO #14340] (Bitbake rev: 7f7034a6a0893debd8a5288a5765146a8b2ab0a0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 97183e10faf9862b5d9489d6e2c27ac77c3b697d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Improve missing core layer error messageRichard Purdie2021-10-021-0/+2
| | | | | | | | | | | | | | If the core layer is missing from bblayers.conf, the message the user sees is hard to understand. Improve it. [YOCTO #14340] (Bitbake rev: 0b08c0ed89951a2fc1f052d1bcae8b8bc5552c8e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 5815a7258ebb8a989e0c6f5798853559d9413f02) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve error display for handled exceptionsRichard Purdie2021-10-021-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't need tracebacks for BBHandledException. Reduces confusing output like: ERROR: /meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio ERROR: ExpansionError during parsing /meta/recipes-core/images/core-image-tiny-initramfs.bb Traceback (most recent call last): File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_qa', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_cpio', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1038, in follow_chain(task='do_image_complete', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in chain: > bb.fatal("Circular task dependencies as %s depends itself via the chain %s?!" % (task, " -> ".join(chain))) chain.append(task) File "/bitbake/lib/bb/__init__.py", line 165, in fatal: mainlogger.critical(''.join(args), extra=kwargs) > raise BBHandledException() to the real error: ERROR: /media/build1/poky/meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio (Bitbake rev: a189283ba9dba8fdf11436d5e74cd2b95c2500b4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 551d4c0576a0a0c3406000029df9238b312f2263) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Catch and error upon circular task referencesRichard Purdie2021-10-021-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | If there are circular task references, error on them rather than show a recursion error. A simple reproducer is: """ do_packageswu () { : } addtask do_packageswu after do_image_complete before do_image_qa """ into image_types.bbclass. There is code in runqueue to detect these but we never get that far with the current codebase. [YOCTO #13140] (Bitbake rev: f613d8d601be75e624e46cfe2351d1a067a9c341) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 339d4d6be515a71311b81fb9e99742af0d8a5130) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Avoid duplicating logs in verbose modeRichard Purdie2021-10-021-0/+4
| | | | | | | | | | | | With "bitbake -v", for task failures you'd see the log output twice. Avoid this by using the existing "did we print info" switch. (Bitbake rev: 53d05a7b4a6380bd2bf8dd7bb0681e8c961bed5d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Don't include logs in error message if piping themRichard Purdie2021-10-021-0/+3
| | | | | | | | | | | | | If the caller is piping the logs, they likely don't want them in the error exception as well. This removes duplicate output from the build output allowing the UI level controls on whether to show logs to work correctly. (Bitbake rev: f84a2f8d8bcc2fa4cd9ab6ef80ae638d0df47965) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fc58ad84a9deb2620ad90611684dad65dafedb11) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Handle SystemExit in python tasks correctlyRichard Purdie2021-10-021-1/+1
| | | | | | | | | | | | | | | | If a python task fails with sys.exit(), we currently see no TaskFailed event. The high level code does detect the exit code and fail the task but it can leave the UI inconsistent with log output. Fix this be intercepting SystemExit explicitly. This makes python task failures consistent with shell task failures. (Bitbake rev: 8dec1a58ff176b82006a084537156f65ad81def9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9eee9fd4f2f96789ad2b037e74d561bdc1426856) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Match markup to real function nameRichard Purdie2021-10-021-2/+2
| | | | | | | | | | | | The point of the injected text is to identify where the function comes from. Using the correct function name would therefore be better. (Bitbake rev: 7919f1f94e15bb5d3d124062bdcfdbe44aceb81a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 30c6ff8551c235254ab90663ab88f66bb0c71edb) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: bitbake-layers: add skip reason to outputMarco Felsch2021-10-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Currently we inform the user that some package/layer is skipped but we don't print the reason albeit bitbake knows the reason. So currently it looks like: gtk+: meta-oe 2.24.32 (skipped) With this change the output prints the skip reason which is very helpful for debugging: gtk+: meta-oe 2.24.32 (skipped: one of 'x11 directfb' needs to be in DISTRO_FEATURES) (Bitbake rev: 253aa584b04f4e5c6e7b16904a9e242a4f5cbd35) Signed-off-by: Marco Felsch <m.felsch@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d43e72db4f7c8b47d91d99ed54ce30e9ee898de1) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ui/taskexp: Fix to work with empty build directoriesRichard Purdie2021-10-021-0/+1
| | | | | | | | | | | | | | | If run on an empty build directory, taskexp wasn't working as it didn't send the current environment to the server. This means HOSTTOOLS in oe-core couldn't be built and gave an error. Add the missing updateToServer call in. [YOCTO #14408] (Bitbake rev: 6737fba707917db16e317bb738c47c096454f816) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 06a0bbe746f879ae539223e7fdb6f07d55d13719) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ui/taskexp: Improve startup exception handlingRichard Purdie2021-10-021-0/+4
| | | | | | | | | | | | | When an exception occurs at startup, show it to the user. [YOCTO #14408] (Bitbake rev: 2dc35a3ef95da594db2051369e98e8f678848849) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit cc1df1af67cfd3e223b39e2b7ea5f86b8cf78aee) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server: Fix early parsing errors preventing zombie bitbakeJoshua Watt2021-10-021-1/+2
| | | | | | | | | | | | | | | | | | | | If the client process never sends cooker data, the server timeout will be 0.0, not None. This will prevent the server from exiting, as it is waiting for a new client. In particular, the client will disconnect with a bad "INHERIT" line, such as: INHERIT += "this-class-does-not-exist" Instead of checking explicitly for None, check for a false value, which means either 0.0 or None. (Bitbake rev: 77f62ec8d45cf639d5030d0743778b9bc496a25c) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch2: Use our own git server for dtc test repoRichard Purdie2021-09-221-1/+1
| | | | | | (Bitbake rev: 3c1799b2576f80b6dcb310e03f77105a58b9fa8e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: providers: replace newly added logger.warn() with logger.warning()Denys Dmytriyenko2021-08-021-2/+2
| | | | | | | | | | | | | Commit https://git.openembedded.org/bitbake/commit/?id=78cd63285713fde59506eb2e71a7b7ee59a594ff converted logger.info() to logger.warn(), which is deprecated and instead should use logger.warning(): https://lists.openembedded.org/g/bitbake-devel/topic/82742194#12377 (Bitbake rev: c2a3bda3a29e12472ef7862e424ea1552fab2959) Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Allow colon in variable expansion regexRichard Purdie2021-07-302-3/+3
| | | | | | | | | Now that ":" is a valid character in variable key names, it needs to be allowed by the variable expansion code too, to match. (Bitbake rev: c5418eae56cc50dbae7951c055434a0c456c53a4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart/parse: Allow ':' characters in variable/function namesRichard Purdie2021-07-304-2/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is becomming increasingly clear we need to find a way to show what is/is not an override in our syntax. We need to do this in a way which is clear to users, readable and in a way we can transition to. The most effective way I've found to this is to use the ":" charater to directly replace "_" where an override is being specified. This includes "append", "prepend" and "remove" which are effectively special override directives. This patch simply adds the character to the parser so bitbake accepts the value but maps it back to "_" internally so there is no behaviour change. This change is simple enough it could potentially be backported to older version of bitbake meaning layers using the new syntax/markup could work with older releases. Even if other no other changes are accepted at this time and we don't backport, it does set us on a path where at some point in future we could require a more explict syntax. I've tested this patch by converting oe-core/meta-yocto to the new syntax for overrides (9000+ changes) and then seeing that builds continue to work with this patch. (Bitbake rev: a6d5fb7554e3cf071e453db56a1e7469ac44277c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Don't classify shell functions that names start with ↵Tomasz Dziendzielski2021-07-301-1/+1
| | | | | | | | | | | | | | | | | | "python*" as python function If shell function name starts with 'python' or 'fakeroot' parser wrongly assumes it's python/fakeroot function. [YOCTO #14204] Use regex lookahead assertions to check if 'python' expression is followed by whitespace or '(' and if 'fakeroot' is followed by whitespace. (Bitbake rev: 9df61675f0e9bb67a78bfa1a16b1cf9fa4c333f1) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Avoid parser deadlocksRichard Purdie2021-06-191-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If you make parsing fail (e.g. add something like: X := "${@d.getVar('MCMACHINES').split()[1]}" to meson.bbclass, then run "while true; do bitbake -g bash; done" it will eventually hang. It appears the cancel_join_thread() call the parsing failure triggers, breaks the results_queue badly enough that it sits in read() indefintely (called from self.result_queue.get(timeout=0.25)). The timeout only applies to lock aquisition, not the read call. I've tried various other approaches such as using cancel_join_thread() in other places but the only way things don't lock up is to avoid cancel_join_thread() entirely for results_queue. I do have a concern that this may adversely affect Ctrl+C handling but equally, its broken now already and this appears to improve things. [YOCTO #14034] (Bitbake rev: 0e0af15b84e07e6763300dcd092b980086b9b9c4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure parser is cleaned upRichard Purdie2021-06-191-0/+1
| | | | | | | | | | | | During cooker shutdown, its possible the parser isn't cleaned up. Fix this (which may partially explain why threads were left hanging around at exit). (Bitbake rev: fe1b57f4d39c287526353387e164c863eb9df421) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Explictly shut down the sync threadRichard Purdie2021-06-191-1/+7
| | | | | | | | | | | | | | | | | | Hongxu Jia reported a problem where the bb_cache files were not always being written out correctly. This was due to the sync thread being terminated prematurely. Whilst the preceeding changes mean the exit handler for this thread is now correctly called since we switch to using sys.exit() instead of os._exit(), this write can happen after we drop the bitbake lock, leading to potential races. Avoid that headache by adding in explicit thread join() calls before we drop the lock (which atexit or Finalize can't do). (Bitbake rev: 6d2dd16b87ce62086f956912e9a7335b2adfcc94) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure parse_quit thread is closed downRichard Purdie2021-06-191-6/+8
| | | | | | | | | | | | | Each run through the parser would leak a thread from the queue created to shut the parser down. Close this down correctly and clean up the code flow slightly whilst in the area, making sure this thread does shut down correctly (we don't care if it loses data). (Bitbake rev: 1dcc4e29dee3fd0dd4d6dc2fc9ccdc5888fcfcf3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: providers: selected version not available should be a warningRoss Burton2021-05-221-2/+2
| | | | | | | | | | | | | If the selected version if not available, bitbake will happily attempt to build something else. This should be a loud warning not a small note. (Bitbake rev: 078f3164dcb1de7a141bec3a8fd52631d0362631) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 78cd63285713fde59506eb2e71a7b7ee59a594ff) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: Fix crash when using git LFS and submodulesNiels Avonds2021-05-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: b4acaa314cd8bc86c50e14464ff01c2d110122c4) Signed-off-by: Niels Avonds <niels@codebits.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e05d79a6ed92c9ce17b90fd5fb6186898a7b3bf8) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix deferred task issuesRichard Purdie2021-05-131-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In a multiconfig situation there are circumstances where firstly, tasks are deferred when they shouldn't be, then later, tasks can end up as both covered and not covered. This patch fixes two related issues. Firstly, the stamp validity checking is done up front in the build and not reevaulated. When rebuilding the deferred task list after scenequeue hash change updates, we need therefore need to check if a task was in notcovered *or* covered when deciding to defer it. This avoids strange logs like: NOTE: Running setscene task X of Y (mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch_setscene) NOTE: Deferring mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch after mc:host:/A/alsa-state.bb:do_deploy_source_date_epoch where tasks have run but are then deferred. Since we're recalculating the whole list, we also need to clear it before iterating to rebuild it. By ensuring covered tasks aren't added to the deferred queue, the covered + notcovered issue should also be avoided. in the task deadlock forcing code. [YOCTO #14342] (Bitbake rev: fa068b5a3430b1b580cacfaf9011cdc3324d5844) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3c8717fb9ee1114dd80fc1ad22ee6c9e312bdac7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: tests/fetch: remove write protected files tooMikko Rapeli2021-05-131-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For some reason several git-annex files in Debian 10 buster are read-only and removing them with "rm -rf" fails. Fixes test failures like: $ bitbake-selftest ... rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied EE..................................ssss.sssssssssssssss.sssss....................................................................................................... ====================================================================== ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex fetcher, ud = self.fetch_shallow(uri) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow bb.utils.remove(ud.clonedir, recurse=True) File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path)) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1. Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod the whole temporary directory which should cover any directory name differences between different git-annex versions. Fixes tests failing after chmod call: Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/ (Bitbake rev: 1fae1c812138f35c35ea4c0586e21f022524c5f1) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-14 21:07:28 +0000