summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/build.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: build: Make python output print to stdout when running with -v ↵Mark Asselstine2023-03-211-2/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (verbose) When tasks are run with -v (verbose) on the bitbake commandline, shell tasks print their stdout, python tasks do not. This change redirects the python task's print output to an in memory buffer. After the task is executed the output is printed to stdout via the logger. This makes the python task behavior match the shell task behavior when running with -v. The contents of the task's log files remain unchanged after this change. This approach should keep the correct order in most cases, however, if the python task accesses the logger directly, that content will appear before other output. On the other hand, this change should negate the need for python tasks to access the logger directly. Special care is taken to save/restore the existing stdout and stderr and preventing sending output directly to the logger when there are "recursive" calls, for instance when a python function calls a shell function, avoiding printing things potentially out of order and/or multiple times. The logging-test.bb in meta-selftest can be used to review this change. This has been tested with the full bblogging oeqa tests. [Yocto #14544] (Bitbake rev: 81a58647b2f4fc0a2589b2978fc9d81b2bfe6aec) Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/siggen: Rework stamps functionsRichard Purdie2022-12-171-105/+44
| | | | | | | | | | | | | | | The current method of passing either a task's datastore, or dataCaches and a filename into the stamp functions is rather horrible. Due to the different contexts, fixing this is hard but we do control the bitbake side of the API usage so we can migrate those to use other functions and then only support a datastore in the public bb.build API which is only called from task context in OE-Core. (Bitbake rev: c79ecec580e4c2a141ae483ec0f6448f70593dcf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/siggen/runqueue: Drop do_setscene referencesRichard Purdie2022-12-111-3/+3
| | | | | | | | | | do_setscene was from a different era before our modern setscene per task code. It hasn't been used for years so remove some old obsolete references to it. (Bitbake rev: ef72282298f7c4db74383c23bb0251dd06d3c6d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/data_smart/build: Clean up datastore finalize/update_data ↵Richard Purdie2022-11-141-1/+0
| | | | | | | | | | | | | references We dropped the update_data calls a while ago. Clean up the code to match the reality and drop the remaining no-op pieces. Update the comments to reflect the slowest operations and let the cookie monster's spirit live on! (Bitbake rev: 584989ed2b5af4e8799571dece0cf94f995ef14e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: prefix the tasks with a timestamp in the log task_orderJose Quaresma2022-08-121-1/+3
| | | | | | | | | | This is useful when debugging as it helps understand possible race conditions between tasks of diferent recipes. (Bitbake rev: 950a2ea4c91d6e13d7587104367fa85cc7efe01c) Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add clean_stamp API function to allow removal of task stampsRichard Purdie2022-05-161-5/+9
| | | | | | | | | | | | | | | We currently have no API to be able to remove all the potential stamps of a task. It is unusual to need to do this, particularly as you could race against other things happening in the system but we do have a use case for this in cleaning up sysroots in OE-Core. The alternative is to mess with CLEANMASK in OE-Core but that is just going to add potential for errors. We need the first part of the make_stamp() function so separate that out so it can be called seperately. (Bitbake rev: 4d671504a25863018ac51c21c005cef0a4d8f05c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add missing parameter to TaskInvalidRichard Purdie2022-03-301-1/+1
| | | | | | | | | | | | Fixes: File "bitbake/lib/bb/build.py", line 585, in _exec_task event.fire(TaskInvalid(task, d), d) TypeError: __init__() missing 1 required positional argument: 'metadata' (Bitbake rev: 16ea4429344e36ff022616abd7fa74e5964b6d6a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Tweak exception handling for setscene tasksRichard Purdie2022-01-191-5/+9
| | | | | | | | | | | | | | | If an unexpected exception occurs in a setscene task, it is currently hidden from the user and not recorded in any logs. This isn't helpful to debug such failures. Change the code so that even in the "silent" or "quiet" task case (setscene tasks), a warning is shown with the traceback unless it was an "handled" exception. This means the failing function can show it's own warning/error instead if it wants to and then raise a handled event. (Bitbake rev: 41dcdc61eb40def8c14a42e8d7bb9ce5a34afa57) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Set BB_CURRENTTASK earlierRichard Purdie2021-10-141-1/+0
| | | | | | | | | | | For some debugging, BB_CURRENTTASK is set too late to be useful as it isn't present in some event handlers for example. There is no other way to know which task is actually running so set the value earlier. (Bitbake rev: 7daaaaa27f55b5a458656857c6d61a51b34a62fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Fix log flushing raceRichard Purdie2021-09-231-38/+38
| | | | | | | | | | | | | | There is a race between sending the TaskFailed event which could trigger the UI to look at the logs and flushing and closing the log files. Reverse the order of the finally clause and the exception handling to ensure we've handled the logfiles before sending the task events. This should fix a race seen in bblogging.BitBakeLogging.test_python_exit_logging (Bitbake rev: 032190aac31604d37740d8aecf6e74a5448de358) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Make exception printing clearerRichard Purdie2021-09-231-1/+1
| | | | | | | | | | | | | | Shows: ERROR: SystemExit(1) instead of: ERROR: 1 (Bitbake rev: df89e37c33e4398a5f8ece9a8b973be3fe2ff361) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure python stdout/stderr is logged correctlyRichard Purdie2021-09-231-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 | This is python stdout Whilst after the change we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | This is python stdout | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 since the output is now correctly mixed with the log messages. In some cases the logging tests indicate the output is being lost entirely which is bad for debugging and makes things rather confusing. (Bitbake rev: 8966b43761500e0505333d8c9a3f0f2c3dbe7559) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Catch and error upon circular task referencesRichard Purdie2021-09-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | If there are circular task references, error on them rather than show a recursion error. A simple reproducer is: """ do_packageswu () { : } addtask do_packageswu after do_image_complete before do_image_qa """ into image_types.bbclass. There is code in runqueue to detect these but we never get that far with the current codebase. [YOCTO #13140] (Bitbake rev: 339d4d6be515a71311b81fb9e99742af0d8a5130) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Avoid duplicating logs in verbose modeRichard Purdie2021-09-111-0/+4
| | | | | | | | | With "bitbake -v", for task failures you'd see the log output twice. Avoid this by using the existing "did we print info" switch. (Bitbake rev: e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Handle SystemExit in python tasks correctlyRichard Purdie2021-09-111-1/+1
| | | | | | | | | | | | | If a python task fails with sys.exit(), we currently see no TaskFailed event. The high level code does detect the exit code and fail the task but it can leave the UI inconsistent with log output. Fix this be intercepting SystemExit explicitly. This makes python task failures consistent with shell task failures. (Bitbake rev: 9eee9fd4f2f96789ad2b037e74d561bdc1426856) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Match markup to real function nameRichard Purdie2021-09-111-2/+2
| | | | | | | | | The point of the injected text is to identify where the function comes from. Using the correct function name would therefore be better. (Bitbake rev: 30c6ff8551c235254ab90663ab88f66bb0c71edb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: warn on setting noexec/nostamp/fakeroot flag to any value ↵Chris Laplante2021-06-061-0/+5
| | | | | | | | | | | | | | | | | | | besides '1' Currently setting those flags to even the empty string "" causes it to be set, which is contrary to the documentation. In a future version of BitBake, we'd like to change the behavior so that setting the flag to "" does not set it. This will allow conditionally setting noexec, using variable expansion or inline Python. I found no places in poky or meta-openembedded where this warning would trigger. [YOCTO #13808] (Bitbake rev: 1e7655c4f765ba7b4791c4cca048a69bf8d9c93d) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add find_stale_stamps functionRichard Purdie2021-03-231-0/+17
| | | | | | | | | | Add a new function which compares the stamp filename we want (including taskhash) with what is in the stamp directory (using the clean mask). This tells us which stamp files are stale and are due to be rerun. (Bitbake rev: b126a53882d202e4df0f9661303355c9fe9ec80e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: logging: Make bitbake logger compatible with python loggerJoshua Watt2021-02-101-3/+3
| | | | | | | | | | | | | | | | | The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/msg: Cleanup verbose option handlingRichard Purdie2020-08-251-2/+5
| | | | | | | | | | | | The levels of indirection to set these verbose logging options is rather crazy. This attempts to turn things into two specific options with much more specific meanings. For now its all still controlled by the commandline verbose option and should funciton as previously, with the addition that the BB_VERBOSE_LOGS option can now be task specific. (Bitbake rev: 423c046f2173aaff3072dc3d0882d01b8a0b0212) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: make shell traps less chatty when 'bitbake -v' is useduninative-2.9Chris Laplante2020-08-211-2/+3
| | | | | | | | | | | | This shuts up both the DEBUG and EXIT handlers. Also, remove an unneeded "set -o errtrace" (i.e. set -E), since we no longer use an ERR trap. (Bitbake rev: 89e851fa0403d1e98aeed69990101e3f84f0b283) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace with the original metadata locations of ↵Chris Laplante2020-08-151-0/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash shell funcs Leverage the comments that emit_var writes and the backtrace that the shell func writes to generate an additional metadata-relative backtrace. This will help the user troubleshoot shell funcs much more easily. Example: | WARNING: /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955:171 exit 1 from 'exit 1' | WARNING: Backtrace (BB generated script): | #1: myclass_do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 171 | #2: do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 166 | #3: actually_fail, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 153 | #4: my_compile_extra, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 155 | #5: do_compile, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 141 | #6: main, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 184 | | Backtrace (metadata-relative locations): | #1: myclass_do_something, /home/laplante/repos/oe-core/meta/classes/myclass.bbclass, line 2 | #2: do_something, autogenerated, line 2 | #3: actually_fail, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 36 | #4: my_compile_extra, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 38 | #5: do_compile, autogenerated, line 3 ERROR: Task (/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 542 tasks of which 541 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. (Bitbake rev: ae1aa4ea79826c32b20e1e7abdf77a15b601c6f2) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace when a Bash shell function failsChris Laplante2020-08-151-11/+50
| | | | | | | | | | | | | The trick here is to use a DEBUG trap to record the line number of each command as they execute. This is so we can report the real line number of the failing command, which is otherwise not possible in a Bash EXIT trap. See http://gnu-bash.2382.n7.nabble.com/trap-echo-quot-trap-exit-on-LINENO-quot-EXIT-gt-wrong-linenumber-td3666.html (Bitbake rev: 8366e2ad5fdad3972766b40b23188908330067ee) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: create_progress_handler: fix calling 'get' on NoneTypeChris Laplante2020-07-291-1/+1
| | | | | | | | | Just use the |resolve| function which already takes care of it. (Bitbake rev: 91b809a0902ffd42be4edf7f0a7d25e6d354d822) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print traceback if progress handler can't be createdChris Laplante2020-07-291-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ERROR: expat-native-2.2.9-r0 do_compile: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_compile.90289 Log data follows: | ERROR: 'NoneType' object has no attribute 'get' ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_compile) failed with exit code '1' After: ERROR: expat-native-2.2.9-r0 do_configure: Failed to create progress handler ERROR: expat-native-2.2.9-r0 do_configure: Traceback (most recent call last): File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 400, in exec_func_shell logfile = create_progress_handler(func, progress, logfile, d) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 344, in create_progress_handler cls_obj = functools.reduce(resolve, cls.split("."), bb.utils._context) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 343, in resolve return x.get(y) AttributeError: 'NoneType' object has no attribute 'get' ERROR: expat-native-2.2.9-r0 do_configure: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_configure.22982 ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_configure) failed with exit code '1' (Bitbake rev: ba0472a672eacbbbb2295252a0d2056d3399c6a9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: augment TaskBase to capture PN and PVChris Laplante2020-03-071-0/+2
| | | | | | | | | | PF is already captured, but it's a pain to try to parse out PN and PV, which can be helpful to have available. (Bitbake rev: 2885034f42a1dce7586e081a0461b8932a3593bf) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: remove unused importsFrazer Clews2020-01-191-1/+0
| | | | | | | | | | removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty/uihelper: Switch from pids to tids for Task event managementRichard Purdie2019-12-061-12/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | We've seen cases where a task can execute with a given pid, complete and a new task can start using the same pid before the UI handler has had time to adapt. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/knotty.py", line 484, in main helper.eventHandler(event) File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/uihelper.py", line 30, in eventHandler del self.running_tasks[event.pid] KeyError: 13490 This means using pids to match up events on the UI side is a bad idea. Change the code to use task ids instead. There is a small amount of fuzzy matching for the progress information since there is no task information there and we don't want the overhead of a task ID in every event, however since pid reuse is unlikely, we can live with a progress bar not quite working properly in a corner case like this. [YOCTO #13667] (Bitbake rev: e427eafa1bb04008d12100ccc5c862122bba53e0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Drop bb.build.FuncFailedRichard Purdie2019-08-061-47/+10
| | | | | | | | | | | | | Its hard to see what this exception adds in the current codebase. The logfile attribute is effectively ignored, the exception doesn't serve a defined purpose and mostly seems to be worked around. Remove it entirely. If this does cause output problems, we'll figure out better ways to address those. (Bitbake rev: cfeffb602dd5319f071cd6bcf84139ec77f2d170) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: implement custom progress handlers injected via OE_EXTRA_IMPORTSChris Laplante2019-06-111-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | A separate patch to base.bbclass (in poky) will add the OE_EXTRA_IMPORTS variable. The contents are appended into OE_IMPORTS. This provides a mechanism by which layers (in their layer.conf) can make custom progress handlers available. As a backup, individual recipes can inject progress handlers into __builtins__. Custom handlers are expected to have this __init__ signature: def __init__(self, d, outfile=None, otherargs=None): Recipes can then use the handlers like this: do_task[progress] = "custom:mylayer.util.ProgressHandler[:args]" The last part (everything after and including the second colon) is optional. If provided, it is passed to HandlerClass's __init__ as otherargs="args". Otherwise, otherargs=None. (Bitbake rev: 20289d62c84c393990dd3deb0cca1b17c09092e6) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/progress: use context managers for progress handlersChris Laplante2019-06-111-4/+27
| | | | | | | | | | | It seems context management support was half-implemented, but never finished. For example, LogTee has __enter__ and __exit__ but they haven't been exercised until now. (Bitbake rev: bf522ad3e0c52cdb69b406226840d870ff4f2766) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: extract progress handler creation logic into its own methodChris Laplante2019-06-111-11/+16
| | | | | | | (Bitbake rev: a841efa50d3aaf7c57446806327b2b687371cb29) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Strip old editor directives from file headersRichard Purdie2019-05-041-2/+0
| | | | | | | | | | There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. (Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie2019-05-041-13/+0
| | | | | | | | | | With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-041-0/+2
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Disable warning about dependent tasks for nowRichard Purdie2019-05-011-2/+2
| | | | | | | | | This breaks with rm_work so disable the warning until we find a better solution (and change the test accordingly too). (Bitbake rev: 93e94c06baf013e3d072465a55bddd1fe61c0772) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure warning for invalid task dependencies is usefulRichard Purdie2019-05-011-1/+1
| | | | | | | | | | | | WARNING: elfutils: dependent task do_rm_work does not exist is much less useful than WARNING: elfutils: dependent task do_rm_work for do_deploy does not exist (Bitbake rev: e034c6f75e3d7730ff16a8d1bd0cba03beda0af8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add verbnote to shell log commandsStefan Kral2019-04-301-0/+2
| | | | | | | | Add the missing bbverbnote fifo cmd for logging from shell. (Bitbake rev: cf11fdb30c405e1a4521a7299f84816c0e13a881) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: check dependendent task for addtaskRobert Yang2019-04-301-0/+3
| | | | | | | | | | | | | | | | | The following command is incorrect, but was ignored silently, that may suprise users: addtask task after task_not_existed This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it. [YOCTO #13282] (Bitbake rev: b1ad36169c2ad189d84c53c707b66c12d5ff812f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add unhandled exception information to the logger when ↵Alexander Kanavin2019-02-141-1/+2
| | | | | | | | | | | | | | | executing python tasks Previously this information was simply discarded, which in some cases resulted in generic, unhelpful failures. With this change the user would see what the exception is, but without ugly tracebacks or bulky, irrelevant information from the log file. (Bitbake rev: 8da9242702fbef8cf3156b95a1076802e0f653c7) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize/build: Clean up /dev/null fd handlingRichard Purdie2018-09-221-5/+2
| | | | | | | | | | | | | | | | At the end of bitbake selftest we see: sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r+' encoding='UTF-8'> Clean up the /dev/null handling to drop the unused entry in build.by and ensure the other open() calls are cleaned up. NULL was unused since http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/build.py?id=4a081b5a52e3d27da8d4b062f3fda292e8d8fb0a back in 2012. (Bitbake rev: e72be96cfa9f05fda5f420c7cfa8bcfa9304b884) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add support for multiconfig dependenciesAlejandro Enedino Hernandez Samaniego2018-08-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the capability for tasks from different multiconfigs to depend on one another. These dependencies can be enabled using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to: - cooker: To glue both multiconfigs in one place and make sure the dependencies can be provided. - taskdata: To parse and add a new kind of dependency (mcdepends) to the taskdata object. - runqueue: To differentiate tasks from different multiconfigs, add the specified dependencies to the corresponding tasks, and create a working runqueue that contains tasks from both multiconfigs. - siggen: To avoid looking for tasks from different multiconfigs on objects where they dont belong. The taskdata objects are still not aware of the concept of multiconfig, so each object doesnt know which multiconfig its building, hence why the mcdepends are added to all taskdata objects equally (we really dont expect many of these), but the actual dependencies are added only to the required tasks by the runqueue. (Bitbake rev: da8cb8633504bdc815bdcefc538340b9bce5065d) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: Don't return dependencies for tasks which don't existRichard Purdie2017-12-281-0/+6
| | | | | | | | | | If for example you deltask do_build but it has recrdeps set, its confusing to have that list returned when the task no longer exists (same would apply to deps too if it was set after the deltask). (Bitbake rev: b7a81cb91e82ba64b63c9153dc161a2ab4696715) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: add tasksbetween() functionPaul Eggleton2017-02-071-0/+27
| | | | | | | | | | | | | | | Add a utility function that gives the list of dependent tasks between two specified tasks (just within the same recipe). This is intended to be able to be executed from recipe context so it uses the datastore rather than having access to the runqueue. This will be used in OpenEmbedded-Core's populate_sdk_ext.bbclass to get the list of tasks between do_image_complete and do_build. (Bitbake rev: 433379bf12cf31fdf46defdf66695cf8be9994b1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: avoid exception when function is not definedPatrick Ohly2017-01-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | exc_func() fails with a hard to debug exception when the function does not exist, for example due to a typo: ERROR: ...: Traceback (most recent call last): File "/work/bitbake/lib/bb/build.py", line 644, in exec_task return _exec_task(fn, task, d, quieterr) File "/work/bitbake/lib/bb/build.py", line 584, in _exec_task exec_func(func, localdata) File "/work/bitbake/lib/bb/build.py", line 198, in exec_func cleandirs = flags.get('cleandirs') AttributeError: 'NoneType' object has no attribute 'get' There is code further down which will print a warning, but we don't reach that unless we allow None as result of of d.getVarFlags() first. The warning is further down intentionally and has to stay there, to ensure that 'cleandirs' gets executed also for empty functions. (Bitbake rev: 25df3db5eeda9dbf417e67e96845f376e5c8e6b3) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add preceedtask() APIPatrick Ohly2017-01-201-0/+16
| | | | | | | | | | | | | | | | | | | The API is required by the revised rm_work.bbclass implementation, which needs to know all tasks that do_build depends so that it can properly inject itself between do_build and those tasks. The new API primarily hides the internal implementation of the "after" and "before" dependency tracking. Because tasks defined as precondition via "recrdeptask" may or may not be relevant (they are for rm_work.bclass), the API also includes support for that. There's no default value for including recrdeptasks, so developers have to think about what they need. (Bitbake rev: 9289ab40e77906e983a2f79cd7602ee95be5025a) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/uihelper: Show better information about multiconfig tasks on UIRichard Purdie2016-12-221-0/+1
| | | | | | | | | | Currently the UI shows X is building, possibly multiple times but doesn't say which of the multilibs that might be. This adds a prefix to the task name so the mulitconfig being built can be identified. (Bitbake rev: d93ddd185dde53f50aea06ada094bb0d9b5ba4f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: enable access to logger within tasksPaul Eggleton2016-12-141-0/+1
| | | | | | | | | | | | | | | | | | In certain circumstances it can be useful to get access to BitBake's logger within a task; the main example is in OpenEmbedded's image construction code where we want to be able to check the log file for errors and warnings, but we don't want to see any errors or warnings that were emitted through the logger; so we need a way to exclude those. In order to do this, pass the logger object into the task via a BB_TASK_LOGGER variable, and add a logging handler class to bb.utils that can be added to it in order to keep a list of warnings/errors that have been emitted. (Bitbake rev: f1cd6fab604f14d8686b1d783cbfe012d923ee42) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-301-3/+3
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-20/+20
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-07 19:14:47 +0000