summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/build.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: build: Make exception printing clearerRichard Purdie2021-10-251-1/+1
| | | | | | | | | | | | | | | | | Shows: ERROR: SystemExit(1) instead of: ERROR: 1 (Bitbake rev: 16bb44e00fcad8daefd3b79c5564d9c8a0d03a6c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit df89e37c33e4398a5f8ece9a8b973be3fe2ff361) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure python stdout/stderr is logged correctlyRichard Purdie2021-10-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 | This is python stdout Whilst after the change we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | This is python stdout | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 since the output is now correctly mixed with the log messages. In some cases the logging tests indicate the output is being lost entirely which is bad for debugging and makes things rather confusing. (Bitbake rev: 3fbc46735ee3679ef1b7466810e4345d78ed05ea) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8966b43761500e0505333d8c9a3f0f2c3dbe7559) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Handle SystemExit in python tasks correctlyRichard Purdie2021-10-021-1/+1
| | | | | | | | | | | | | | | | If a python task fails with sys.exit(), we currently see no TaskFailed event. The high level code does detect the exit code and fail the task but it can leave the UI inconsistent with log output. Fix this be intercepting SystemExit explicitly. This makes python task failures consistent with shell task failures. (Bitbake rev: fdc2d3df35ad1282c4ad85d76519e395b023a563) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9eee9fd4f2f96789ad2b037e74d561bdc1426856) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Avoid duplicating logs in verbose modeRichard Purdie2021-10-021-0/+4
| | | | | | | | | | | | With "bitbake -v", for task failures you'd see the log output twice. Avoid this by using the existing "did we print info" switch. (Bitbake rev: 69c622b744d987b48f7dd805ead973ad90479e4e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Catch and error upon circular task referencesRichard Purdie2021-09-161-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | If there are circular task references, error on them rather than show a recursion error. A simple reproducer is: """ do_packageswu () { : } addtask do_packageswu after do_image_complete before do_image_qa """ into image_types.bbclass. There is code in runqueue to detect these but we never get that far with the current codebase. [YOCTO #13140] (Bitbake rev: 4d83d9103d9225a2ac42c0f3dfada2f5cf140b68) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 339d4d6be515a71311b81fb9e99742af0d8a5130) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add find_stale_stamps functionRichard Purdie2021-03-231-0/+17
| | | | | | | | | | Add a new function which compares the stamp filename we want (including taskhash) with what is in the stamp directory (using the clean mask). This tells us which stamp files are stale and are due to be rerun. (Bitbake rev: b126a53882d202e4df0f9661303355c9fe9ec80e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: logging: Make bitbake logger compatible with python loggerJoshua Watt2021-02-101-3/+3
| | | | | | | | | | | | | | | | | The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/msg: Cleanup verbose option handlingRichard Purdie2020-08-251-2/+5
| | | | | | | | | | | | The levels of indirection to set these verbose logging options is rather crazy. This attempts to turn things into two specific options with much more specific meanings. For now its all still controlled by the commandline verbose option and should funciton as previously, with the addition that the BB_VERBOSE_LOGS option can now be task specific. (Bitbake rev: 423c046f2173aaff3072dc3d0882d01b8a0b0212) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: make shell traps less chatty when 'bitbake -v' is useduninative-2.9Chris Laplante2020-08-211-2/+3
| | | | | | | | | | | | This shuts up both the DEBUG and EXIT handlers. Also, remove an unneeded "set -o errtrace" (i.e. set -E), since we no longer use an ERR trap. (Bitbake rev: 89e851fa0403d1e98aeed69990101e3f84f0b283) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace with the original metadata locations of ↵Chris Laplante2020-08-151-0/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash shell funcs Leverage the comments that emit_var writes and the backtrace that the shell func writes to generate an additional metadata-relative backtrace. This will help the user troubleshoot shell funcs much more easily. Example: | WARNING: /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955:171 exit 1 from 'exit 1' | WARNING: Backtrace (BB generated script): | #1: myclass_do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 171 | #2: do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 166 | #3: actually_fail, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 153 | #4: my_compile_extra, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 155 | #5: do_compile, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 141 | #6: main, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 184 | | Backtrace (metadata-relative locations): | #1: myclass_do_something, /home/laplante/repos/oe-core/meta/classes/myclass.bbclass, line 2 | #2: do_something, autogenerated, line 2 | #3: actually_fail, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 36 | #4: my_compile_extra, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 38 | #5: do_compile, autogenerated, line 3 ERROR: Task (/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 542 tasks of which 541 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. (Bitbake rev: ae1aa4ea79826c32b20e1e7abdf77a15b601c6f2) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace when a Bash shell function failsChris Laplante2020-08-151-11/+50
| | | | | | | | | | | | | The trick here is to use a DEBUG trap to record the line number of each command as they execute. This is so we can report the real line number of the failing command, which is otherwise not possible in a Bash EXIT trap. See http://gnu-bash.2382.n7.nabble.com/trap-echo-quot-trap-exit-on-LINENO-quot-EXIT-gt-wrong-linenumber-td3666.html (Bitbake rev: 8366e2ad5fdad3972766b40b23188908330067ee) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: create_progress_handler: fix calling 'get' on NoneTypeChris Laplante2020-07-291-1/+1
| | | | | | | | | Just use the |resolve| function which already takes care of it. (Bitbake rev: 91b809a0902ffd42be4edf7f0a7d25e6d354d822) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print traceback if progress handler can't be createdChris Laplante2020-07-291-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ERROR: expat-native-2.2.9-r0 do_compile: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_compile.90289 Log data follows: | ERROR: 'NoneType' object has no attribute 'get' ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_compile) failed with exit code '1' After: ERROR: expat-native-2.2.9-r0 do_configure: Failed to create progress handler ERROR: expat-native-2.2.9-r0 do_configure: Traceback (most recent call last): File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 400, in exec_func_shell logfile = create_progress_handler(func, progress, logfile, d) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 344, in create_progress_handler cls_obj = functools.reduce(resolve, cls.split("."), bb.utils._context) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 343, in resolve return x.get(y) AttributeError: 'NoneType' object has no attribute 'get' ERROR: expat-native-2.2.9-r0 do_configure: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_configure.22982 ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_configure) failed with exit code '1' (Bitbake rev: ba0472a672eacbbbb2295252a0d2056d3399c6a9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: augment TaskBase to capture PN and PVChris Laplante2020-03-071-0/+2
| | | | | | | | | | PF is already captured, but it's a pain to try to parse out PN and PV, which can be helpful to have available. (Bitbake rev: 2885034f42a1dce7586e081a0461b8932a3593bf) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: remove unused importsFrazer Clews2020-01-191-1/+0
| | | | | | | | | | removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty/uihelper: Switch from pids to tids for Task event managementRichard Purdie2019-12-061-12/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | We've seen cases where a task can execute with a given pid, complete and a new task can start using the same pid before the UI handler has had time to adapt. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/knotty.py", line 484, in main helper.eventHandler(event) File "/home/pokybuild/yocto-worker/qemux86-alt/build/bitbake/lib/bb/ui/uihelper.py", line 30, in eventHandler del self.running_tasks[event.pid] KeyError: 13490 This means using pids to match up events on the UI side is a bad idea. Change the code to use task ids instead. There is a small amount of fuzzy matching for the progress information since there is no task information there and we don't want the overhead of a task ID in every event, however since pid reuse is unlikely, we can live with a progress bar not quite working properly in a corner case like this. [YOCTO #13667] (Bitbake rev: e427eafa1bb04008d12100ccc5c862122bba53e0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Drop bb.build.FuncFailedRichard Purdie2019-08-061-47/+10
| | | | | | | | | | | | | Its hard to see what this exception adds in the current codebase. The logfile attribute is effectively ignored, the exception doesn't serve a defined purpose and mostly seems to be worked around. Remove it entirely. If this does cause output problems, we'll figure out better ways to address those. (Bitbake rev: cfeffb602dd5319f071cd6bcf84139ec77f2d170) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: implement custom progress handlers injected via OE_EXTRA_IMPORTSChris Laplante2019-06-111-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | A separate patch to base.bbclass (in poky) will add the OE_EXTRA_IMPORTS variable. The contents are appended into OE_IMPORTS. This provides a mechanism by which layers (in their layer.conf) can make custom progress handlers available. As a backup, individual recipes can inject progress handlers into __builtins__. Custom handlers are expected to have this __init__ signature: def __init__(self, d, outfile=None, otherargs=None): Recipes can then use the handlers like this: do_task[progress] = "custom:mylayer.util.ProgressHandler[:args]" The last part (everything after and including the second colon) is optional. If provided, it is passed to HandlerClass's __init__ as otherargs="args". Otherwise, otherargs=None. (Bitbake rev: 20289d62c84c393990dd3deb0cca1b17c09092e6) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/progress: use context managers for progress handlersChris Laplante2019-06-111-4/+27
| | | | | | | | | | | It seems context management support was half-implemented, but never finished. For example, LogTee has __enter__ and __exit__ but they haven't been exercised until now. (Bitbake rev: bf522ad3e0c52cdb69b406226840d870ff4f2766) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: extract progress handler creation logic into its own methodChris Laplante2019-06-111-11/+16
| | | | | | | (Bitbake rev: a841efa50d3aaf7c57446806327b2b687371cb29) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Strip old editor directives from file headersRichard Purdie2019-05-041-2/+0
| | | | | | | | | | There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. (Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie2019-05-041-13/+0
| | | | | | | | | | With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-041-0/+2
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Disable warning about dependent tasks for nowRichard Purdie2019-05-011-2/+2
| | | | | | | | | This breaks with rm_work so disable the warning until we find a better solution (and change the test accordingly too). (Bitbake rev: 93e94c06baf013e3d072465a55bddd1fe61c0772) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure warning for invalid task dependencies is usefulRichard Purdie2019-05-011-1/+1
| | | | | | | | | | | | WARNING: elfutils: dependent task do_rm_work does not exist is much less useful than WARNING: elfutils: dependent task do_rm_work for do_deploy does not exist (Bitbake rev: e034c6f75e3d7730ff16a8d1bd0cba03beda0af8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add verbnote to shell log commandsStefan Kral2019-04-301-0/+2
| | | | | | | | Add the missing bbverbnote fifo cmd for logging from shell. (Bitbake rev: cf11fdb30c405e1a4521a7299f84816c0e13a881) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: check dependendent task for addtaskRobert Yang2019-04-301-0/+3
| | | | | | | | | | | | | | | | | The following command is incorrect, but was ignored silently, that may suprise users: addtask task after task_not_existed This patch can check and warn for it. It would be better to also check "before" tasks, but there is no easier way to do it. [YOCTO #13282] (Bitbake rev: b1ad36169c2ad189d84c53c707b66c12d5ff812f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add unhandled exception information to the logger when ↵Alexander Kanavin2019-02-141-1/+2
| | | | | | | | | | | | | | | executing python tasks Previously this information was simply discarded, which in some cases resulted in generic, unhelpful failures. With this change the user would see what the exception is, but without ugly tracebacks or bulky, irrelevant information from the log file. (Bitbake rev: 8da9242702fbef8cf3156b95a1076802e0f653c7) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize/build: Clean up /dev/null fd handlingRichard Purdie2018-09-221-5/+2
| | | | | | | | | | | | | | | | At the end of bitbake selftest we see: sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r+' encoding='UTF-8'> Clean up the /dev/null handling to drop the unused entry in build.by and ensure the other open() calls are cleaned up. NULL was unused since http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/build.py?id=4a081b5a52e3d27da8d4b062f3fda292e8d8fb0a back in 2012. (Bitbake rev: e72be96cfa9f05fda5f420c7cfa8bcfa9304b884) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add support for multiconfig dependenciesAlejandro Enedino Hernandez Samaniego2018-08-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the capability for tasks from different multiconfigs to depend on one another. These dependencies can be enabled using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to: - cooker: To glue both multiconfigs in one place and make sure the dependencies can be provided. - taskdata: To parse and add a new kind of dependency (mcdepends) to the taskdata object. - runqueue: To differentiate tasks from different multiconfigs, add the specified dependencies to the corresponding tasks, and create a working runqueue that contains tasks from both multiconfigs. - siggen: To avoid looking for tasks from different multiconfigs on objects where they dont belong. The taskdata objects are still not aware of the concept of multiconfig, so each object doesnt know which multiconfig its building, hence why the mcdepends are added to all taskdata objects equally (we really dont expect many of these), but the actual dependencies are added only to the required tasks by the runqueue. (Bitbake rev: da8cb8633504bdc815bdcefc538340b9bce5065d) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: Don't return dependencies for tasks which don't existRichard Purdie2017-12-281-0/+6
| | | | | | | | | | If for example you deltask do_build but it has recrdeps set, its confusing to have that list returned when the task no longer exists (same would apply to deps too if it was set after the deltask). (Bitbake rev: b7a81cb91e82ba64b63c9153dc161a2ab4696715) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: add tasksbetween() functionPaul Eggleton2017-02-071-0/+27
| | | | | | | | | | | | | | | Add a utility function that gives the list of dependent tasks between two specified tasks (just within the same recipe). This is intended to be able to be executed from recipe context so it uses the datastore rather than having access to the runqueue. This will be used in OpenEmbedded-Core's populate_sdk_ext.bbclass to get the list of tasks between do_image_complete and do_build. (Bitbake rev: 433379bf12cf31fdf46defdf66695cf8be9994b1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: avoid exception when function is not definedPatrick Ohly2017-01-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | exc_func() fails with a hard to debug exception when the function does not exist, for example due to a typo: ERROR: ...: Traceback (most recent call last): File "/work/bitbake/lib/bb/build.py", line 644, in exec_task return _exec_task(fn, task, d, quieterr) File "/work/bitbake/lib/bb/build.py", line 584, in _exec_task exec_func(func, localdata) File "/work/bitbake/lib/bb/build.py", line 198, in exec_func cleandirs = flags.get('cleandirs') AttributeError: 'NoneType' object has no attribute 'get' There is code further down which will print a warning, but we don't reach that unless we allow None as result of of d.getVarFlags() first. The warning is further down intentionally and has to stay there, to ensure that 'cleandirs' gets executed also for empty functions. (Bitbake rev: 25df3db5eeda9dbf417e67e96845f376e5c8e6b3) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add preceedtask() APIPatrick Ohly2017-01-201-0/+16
| | | | | | | | | | | | | | | | | | | The API is required by the revised rm_work.bbclass implementation, which needs to know all tasks that do_build depends so that it can properly inject itself between do_build and those tasks. The new API primarily hides the internal implementation of the "after" and "before" dependency tracking. Because tasks defined as precondition via "recrdeptask" may or may not be relevant (they are for rm_work.bclass), the API also includes support for that. There's no default value for including recrdeptasks, so developers have to think about what they need. (Bitbake rev: 9289ab40e77906e983a2f79cd7602ee95be5025a) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/uihelper: Show better information about multiconfig tasks on UIRichard Purdie2016-12-221-0/+1
| | | | | | | | | | Currently the UI shows X is building, possibly multiple times but doesn't say which of the multilibs that might be. This adds a prefix to the task name so the mulitconfig being built can be identified. (Bitbake rev: d93ddd185dde53f50aea06ada094bb0d9b5ba4f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: enable access to logger within tasksPaul Eggleton2016-12-141-0/+1
| | | | | | | | | | | | | | | | | | In certain circumstances it can be useful to get access to BitBake's logger within a task; the main example is in OpenEmbedded's image construction code where we want to be able to check the log file for errors and warnings, but we don't want to see any errors or warnings that were emitted through the logger; so we need a way to exclude those. In order to do this, pass the logger object into the task via a BB_TASK_LOGGER variable, and add a logging handler class to bb.utils that can be added to it in order to keep a list of warnings/errors that have been emitted. (Bitbake rev: f1cd6fab604f14d8686b1d783cbfe012d923ee42) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-301-3/+3
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-20/+20
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure we preserve sigbasedata files as well as sigdata onesRichard Purdie2016-11-041-1/+1
| | | | | | | | | We don't remove sigdata files, we also shouldn't remove sigbasedata files as this hinders debugging. (Bitbake rev: 8b879fd81fdcf86645cfabad0f54454ba573df52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.build: in _exec_task, catch errors from TaskStartedChristopher Larson2016-10-051-16/+24
| | | | | | | | | | | | | | | We don't always want a traceback when an exception is raised by the TaskStarted event handler. Silently return if we get a SystemExit or HandledException, and print the error and return for FuncFailed. This is done via a separate try/catch block, to avoid firing TaskFailed if all the TaskStarted event handlers didn't complete, otherwise the bitbake UIs get unhappy. (Bitbake rev: ca5b788280ad4303cc08a376e847cbbeda31970c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: fix os.getcwd() exceptionJuro Bystricky2016-09-281-2/+7
| | | | | | | | | | | | | When trying to obtain the current directory from a directory that does not exist anymore, an exception is raised. This patch handles such exception. [YOCTO #10331] (Bitbake rev: 4bcf77589312d9936340d8c308006c2fc9baf67c) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.build: in _exec_task, catch BBHandledExceptionChristopher Larson2016-09-211-0/+3
| | | | | | | | | | We don't want a traceback for this exception, we need to catch it, fire TaskFailed, and return failure. (Bitbake rev: 63966ada459d44d3dc7817ad2a026a22e8f6700f) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Allow dirs/cleandirs to work for empty functionsRichard Purdie2016-09-151-6/+7
| | | | | | | | | | | | | Users are surprised when dirs/cleandirs aren't acted upon for empty functions. This reorders the code slightly so that those flags are acted upon for empty functions as there are cases where this is expected. [YOCTO #10256] (Bitbake rev: 5bf874673d75b5f4ff2b34f0ab8502558ee84d00) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/runqueue: Add noextra stamp file parameter to fix multiconfig ↵Richard Purdie2016-09-021-3/+5
| | | | | | | | | | | | | | | | | | builds We can't execute the same task for the same package_arch multiple times as the current setup has conflicting directories. Since these would usually have the same stamp/hash, we want to execute in sequence rather than in parallel, so for the purposes of task execution, don't consider the "extra-info" on the stamp files. We need to add a parameter to the stamp function to achieve this. This avoids multiple update-rc.d populate_sysroot tasks executing in parallel and breaking multiconfig builds. (Bitbake rev: a9041fc96a14e718c0c1d1676e705343b9e872d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: handle incomplete message fragments in log FIFORoss Burton2016-07-261-31/+36
| | | | | | | | | | | | | | | | | | It's possible that the logging FIFO doesn't do a complete read (or the sender a complete write) with the result that an incomplete message is read in bitbake. This used to result in silently truncated lines but since 42d727 now also results in a warning as the start of the rest of the message isn't a valid logging command. Solve this by storing incoming bytes in a bytearray() across reads, and parsing complete messages from that. [ YOCTO #9999 ] (Bitbake rev: 508112793ee7ace613f07695222997309a2ca58f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: don't use $B as the default cwd for functionsRoss Burton2016-07-191-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | When bitbake executes a shell or Python function it can cd/chdir() into a directory before executing the task. If no directory is specified then the default of $B is used. However $B is an OpenEmbedded variable and BitBake shouldn't be aware of it. To solve this change the semantics slightly so that if no directory is specified, the current working directory isn't changed. There's also a sanity check that emits a warning if a Python task does os.chdir() without restoring the old path, and the previous working directory is restored. This does change semantics: whereas before a function in OE would have $B as the working directory unless specified, now the working directory is the top of the build tree. Any breakage this causes can be solved by either adding do_some_task[dirs] = "${B}" or by using absolute paths in the task. [ YOCTO #4634 ] (Bitbake rev: 67a7b8b021badc17d8fdf447c250e79d291e75f7) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build.py: decode the command as UTF-8Ross Burton2016-07-191-2/+6
| | | | | | | | | | | | | | | The messaging FIFO is UTF-8, so decode the command as UTF-8 as well as the value as otherwise "bberror" != b("bberror") and none of the messages from shell functions are ever displayed. Also add an else to the command parser so unhandled commands are noticed. [ YOCTO #9947 ] (Bitbake rev: 42d727743fa599e0a3c5ad2c29a1e6ede1a918bb) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: implement basic task progress supportPaul Eggleton2016-07-081-0/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | For long-running tasks where we have some output from the task that gives us some idea of the progress of the task (such as a percentage complete), provide the means to scrape the output for that progress information and show it to the user in the default knotty terminal output in the form of a progress bar. This is implemented using a new TaskProgress event as well as some code we can insert to do output scanning/filtering. Any task can fire TaskProgress events; however, if you have a shell task whose output you wish to scan for progress information, you just need to set the "progress" varflag on the task. This can be set to: * "percent" to just look for a number followed by a % sign * "percent:<regex>" to specify your own regex matching a percentage value (must have a single group which matches the percentage number) * "outof:<regex>" to look for the specified regex matching x out of y items completed (must have two groups - first group needs to be x, second y). We can potentially extend this in future but this should be a good start. Part of the implementation for [YOCTO #5383]. (Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build.py: remove task flag in deltask()Robert Yang2016-06-231-0/+1
| | | | | | | | | | Otherwise the function like d.getVarFlag(e, 'task', True) which is used by do_listtasks will still get it, and list the deleted tasks. (Bitbake rev: 779d73619daf59f76f5b0313e7fb5409f6e82553) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Convert to python 3Richard Purdie2016-06-021-7/+6
| | | | | | | | | Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-27 23:21:54 +0000