summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/build.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: daemonize/build: Clean up /dev/null fd handlingRichard Purdie2018-09-221-5/+2
| | | | | | | | | | | | | | | | At the end of bitbake selftest we see: sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r+' encoding='UTF-8'> Clean up the /dev/null handling to drop the unused entry in build.by and ensure the other open() calls are cleaned up. NULL was unused since http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/build.py?id=4a081b5a52e3d27da8d4b062f3fda292e8d8fb0a back in 2012. (Bitbake rev: e72be96cfa9f05fda5f420c7cfa8bcfa9304b884) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add support for multiconfig dependenciesAlejandro Enedino Hernandez Samaniego2018-08-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the capability for tasks from different multiconfigs to depend on one another. These dependencies can be enabled using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to: - cooker: To glue both multiconfigs in one place and make sure the dependencies can be provided. - taskdata: To parse and add a new kind of dependency (mcdepends) to the taskdata object. - runqueue: To differentiate tasks from different multiconfigs, add the specified dependencies to the corresponding tasks, and create a working runqueue that contains tasks from both multiconfigs. - siggen: To avoid looking for tasks from different multiconfigs on objects where they dont belong. The taskdata objects are still not aware of the concept of multiconfig, so each object doesnt know which multiconfig its building, hence why the mcdepends are added to all taskdata objects equally (we really dont expect many of these), but the actual dependencies are added only to the required tasks by the runqueue. (Bitbake rev: da8cb8633504bdc815bdcefc538340b9bce5065d) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: Don't return dependencies for tasks which don't existRichard Purdie2017-12-281-0/+6
| | | | | | | | | | If for example you deltask do_build but it has recrdeps set, its confusing to have that list returned when the task no longer exists (same would apply to deps too if it was set after the deltask). (Bitbake rev: b7a81cb91e82ba64b63c9153dc161a2ab4696715) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: add tasksbetween() functionPaul Eggleton2017-02-071-0/+27
| | | | | | | | | | | | | | | Add a utility function that gives the list of dependent tasks between two specified tasks (just within the same recipe). This is intended to be able to be executed from recipe context so it uses the datastore rather than having access to the runqueue. This will be used in OpenEmbedded-Core's populate_sdk_ext.bbclass to get the list of tasks between do_image_complete and do_build. (Bitbake rev: 433379bf12cf31fdf46defdf66695cf8be9994b1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: avoid exception when function is not definedPatrick Ohly2017-01-261-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | exc_func() fails with a hard to debug exception when the function does not exist, for example due to a typo: ERROR: ...: Traceback (most recent call last): File "/work/bitbake/lib/bb/build.py", line 644, in exec_task return _exec_task(fn, task, d, quieterr) File "/work/bitbake/lib/bb/build.py", line 584, in _exec_task exec_func(func, localdata) File "/work/bitbake/lib/bb/build.py", line 198, in exec_func cleandirs = flags.get('cleandirs') AttributeError: 'NoneType' object has no attribute 'get' There is code further down which will print a warning, but we don't reach that unless we allow None as result of of d.getVarFlags() first. The warning is further down intentionally and has to stay there, to ensure that 'cleandirs' gets executed also for empty functions. (Bitbake rev: 25df3db5eeda9dbf417e67e96845f376e5c8e6b3) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add preceedtask() APIPatrick Ohly2017-01-201-0/+16
| | | | | | | | | | | | | | | | | | | The API is required by the revised rm_work.bbclass implementation, which needs to know all tasks that do_build depends so that it can properly inject itself between do_build and those tasks. The new API primarily hides the internal implementation of the "after" and "before" dependency tracking. Because tasks defined as precondition via "recrdeptask" may or may not be relevant (they are for rm_work.bclass), the API also includes support for that. There's no default value for including recrdeptasks, so developers have to think about what they need. (Bitbake rev: 9289ab40e77906e983a2f79cd7602ee95be5025a) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/uihelper: Show better information about multiconfig tasks on UIRichard Purdie2016-12-221-0/+1
| | | | | | | | | | Currently the UI shows X is building, possibly multiple times but doesn't say which of the multilibs that might be. This adds a prefix to the task name so the mulitconfig being built can be identified. (Bitbake rev: d93ddd185dde53f50aea06ada094bb0d9b5ba4f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: enable access to logger within tasksPaul Eggleton2016-12-141-0/+1
| | | | | | | | | | | | | | | | | | In certain circumstances it can be useful to get access to BitBake's logger within a task; the main example is in OpenEmbedded's image construction code where we want to be able to check the log file for errors and warnings, but we don't want to see any errors or warnings that were emitted through the logger; so we need a way to exclude those. In order to do this, pass the logger object into the task via a BB_TASK_LOGGER variable, and add a logging handler class to bb.utils that can be added to it in order to keep a list of warnings/errors that have been emitted. (Bitbake rev: f1cd6fab604f14d8686b1d783cbfe012d923ee42) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-301-3/+3
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-20/+20
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure we preserve sigbasedata files as well as sigdata onesRichard Purdie2016-11-041-1/+1
| | | | | | | | | We don't remove sigdata files, we also shouldn't remove sigbasedata files as this hinders debugging. (Bitbake rev: 8b879fd81fdcf86645cfabad0f54454ba573df52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.build: in _exec_task, catch errors from TaskStartedChristopher Larson2016-10-051-16/+24
| | | | | | | | | | | | | | | We don't always want a traceback when an exception is raised by the TaskStarted event handler. Silently return if we get a SystemExit or HandledException, and print the error and return for FuncFailed. This is done via a separate try/catch block, to avoid firing TaskFailed if all the TaskStarted event handlers didn't complete, otherwise the bitbake UIs get unhappy. (Bitbake rev: ca5b788280ad4303cc08a376e847cbbeda31970c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: fix os.getcwd() exceptionJuro Bystricky2016-09-281-2/+7
| | | | | | | | | | | | | When trying to obtain the current directory from a directory that does not exist anymore, an exception is raised. This patch handles such exception. [YOCTO #10331] (Bitbake rev: 4bcf77589312d9936340d8c308006c2fc9baf67c) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.build: in _exec_task, catch BBHandledExceptionChristopher Larson2016-09-211-0/+3
| | | | | | | | | | We don't want a traceback for this exception, we need to catch it, fire TaskFailed, and return failure. (Bitbake rev: 63966ada459d44d3dc7817ad2a026a22e8f6700f) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Allow dirs/cleandirs to work for empty functionsRichard Purdie2016-09-151-6/+7
| | | | | | | | | | | | | Users are surprised when dirs/cleandirs aren't acted upon for empty functions. This reorders the code slightly so that those flags are acted upon for empty functions as there are cases where this is expected. [YOCTO #10256] (Bitbake rev: 5bf874673d75b5f4ff2b34f0ab8502558ee84d00) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/runqueue: Add noextra stamp file parameter to fix multiconfig ↵Richard Purdie2016-09-021-3/+5
| | | | | | | | | | | | | | | | | | builds We can't execute the same task for the same package_arch multiple times as the current setup has conflicting directories. Since these would usually have the same stamp/hash, we want to execute in sequence rather than in parallel, so for the purposes of task execution, don't consider the "extra-info" on the stamp files. We need to add a parameter to the stamp function to achieve this. This avoids multiple update-rc.d populate_sysroot tasks executing in parallel and breaking multiconfig builds. (Bitbake rev: a9041fc96a14e718c0c1d1676e705343b9e872d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build: handle incomplete message fragments in log FIFORoss Burton2016-07-261-31/+36
| | | | | | | | | | | | | | | | | | It's possible that the logging FIFO doesn't do a complete read (or the sender a complete write) with the result that an incomplete message is read in bitbake. This used to result in silently truncated lines but since 42d727 now also results in a warning as the start of the rest of the message isn't a valid logging command. Solve this by storing incoming bytes in a bytearray() across reads, and parsing complete messages from that. [ YOCTO #9999 ] (Bitbake rev: 508112793ee7ace613f07695222997309a2ca58f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: don't use $B as the default cwd for functionsRoss Burton2016-07-191-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | When bitbake executes a shell or Python function it can cd/chdir() into a directory before executing the task. If no directory is specified then the default of $B is used. However $B is an OpenEmbedded variable and BitBake shouldn't be aware of it. To solve this change the semantics slightly so that if no directory is specified, the current working directory isn't changed. There's also a sanity check that emits a warning if a Python task does os.chdir() without restoring the old path, and the previous working directory is restored. This does change semantics: whereas before a function in OE would have $B as the working directory unless specified, now the working directory is the top of the build tree. Any breakage this causes can be solved by either adding do_some_task[dirs] = "${B}" or by using absolute paths in the task. [ YOCTO #4634 ] (Bitbake rev: 67a7b8b021badc17d8fdf447c250e79d291e75f7) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build.py: decode the command as UTF-8Ross Burton2016-07-191-2/+6
| | | | | | | | | | | | | | | The messaging FIFO is UTF-8, so decode the command as UTF-8 as well as the value as otherwise "bberror" != b("bberror") and none of the messages from shell functions are ever displayed. Also add an else to the command parser so unhandled commands are noticed. [ YOCTO #9947 ] (Bitbake rev: 42d727743fa599e0a3c5ad2c29a1e6ede1a918bb) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: implement basic task progress supportPaul Eggleton2016-07-081-0/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | For long-running tasks where we have some output from the task that gives us some idea of the progress of the task (such as a percentage complete), provide the means to scrape the output for that progress information and show it to the user in the default knotty terminal output in the form of a progress bar. This is implemented using a new TaskProgress event as well as some code we can insert to do output scanning/filtering. Any task can fire TaskProgress events; however, if you have a shell task whose output you wish to scan for progress information, you just need to set the "progress" varflag on the task. This can be set to: * "percent" to just look for a number followed by a % sign * "percent:<regex>" to specify your own regex matching a percentage value (must have a single group which matches the percentage number) * "outof:<regex>" to look for the specified regex matching x out of y items completed (must have two groups - first group needs to be x, second y). We can potentially extend this in future but this should be a good start. Part of the implementation for [YOCTO #5383]. (Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/build.py: remove task flag in deltask()Robert Yang2016-06-231-0/+1
| | | | | | | | | | Otherwise the function like d.getVarFlag(e, 'task', True) which is used by do_listtasks will still get it, and list the deleted tasks. (Bitbake rev: 779d73619daf59f76f5b0313e7fb5409f6e82553) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Convert to python 3Richard Purdie2016-06-021-7/+6
| | | | | | | | | Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Update logger.warn() -> logger.warning()Richard Purdie2016-05-111-1/+1
| | | | | | | | | | python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.build: handle __builtins__ as a moduleChristopher Larson2016-05-061-2/+7
| | | | | | | | | Fixes pypy support. (Bitbake rev: e90cfc655affeec8f5519f7078dad5f99db3c461) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Allow python functions to execute with real exception ↵Richard Purdie2016-03-311-4/+11
| | | | | | | | | | | | | | | | | | handling With the code as it stands today it not possible to execute a python function and get "normal" python exception handling behaviour. If a python function raises an exception, it forces a traceback to be printed and the exception becomes a FuncFailed exception. This adds in a parameter 'pythonexception' which allows standard python exceptions to be passed unchanged with no traceback. Ultimately we may want to change to this convention in various places but at least it means we can start to add sane functions now. (Bitbake rev: 85cf22fd0ed26bb7dc7738ef2a10441891f38ae2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Improve python execution tracebacksRichard Purdie2016-02-101-7/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If something fails in a exec_func_python() the current stack trace shows incorrect filenames and linenumbers. For example: The stack trace of python calls that resulted in this exception/failure was: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module> 0196: chksum = bb.utils.sha256_file(fn) 0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath))) 0198: 0199: *** 0200:copy_buildsystem(d) 0201: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem 0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs" 0006: 0007:LICENSE = "MIT" 0008: *** 0009:inherit core-image 0010: 0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games" File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] The problem is the use of "FILE" to obtain the current filename. Instead, we therefore inject the function being executed into the methodpool which allows us to correct its linenumber and filename information. We can then clearly mark the initial piece as autogenerated and the rest of the linenumber and filename information should be correct. Afterwards the trace starts: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:copy_buildsystem(d) 0003: File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem 0062: import glob 0063: import oe.copy_buildsystem 0064: import subprocess 0065: *** 0066: subprocess.check_call("foo") 0067: 0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True) 0069: 0070: conf_bbpath = '' File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] We can't inject into methodpool at parsing time, since there may be _append or other override operations against the function before its execution. (Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/data: Don't expand python functions before execution [API change]Richard Purdie2016-02-101-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now, if you have some python code like: X = "a" def somefunction(d): d.setVar("X", "b") d.setVar("Y", "${X}") then any sane person would expect that Y = "b" at the end of the function. This is not the case, Y = "a". This is due to the python function being expanded before execution, the executed code would read d.setVar("Y", "a"). This understandably confuses people, it also makes it near impossible to write ${} in a python function without unintended things happening. I think there is general agreement we should fix this and standardise on non-expansion of python functions. We already don't expand anonymous python (mostly). I've checked OE-Core with buildhistory before and after this change and there were a small number of issues this exposed which I've sent patches for. I propose we default to not expanding python code and then deal with any consequences from that if/as/where identified. This will improve new user understanding and usability of the system, it also allows several long standing weird expansion issues to be fixed. (Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add expansion parameter to getVarFlagRichard Purdie2016-02-041-7/+7
| | | | | | | | | | | | | | | This sets the scene for removing the default False for expansion from getVarFlag. This would later allow True to become the expand default. On the most part this is an automatic translation with: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` There should be no functional change from this patch. (Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: minor shell_trap_code() formatting tweaksAndre McCurdy2016-01-301-3/+2
| | | | | | | | | | | Fix quoting of $BASH_COMMAND and avoid wrapping at 80 columns (the script which follows is likely to contain some very long lines, so line wrapping in bb_exit_handler() looks somewhat out of place). (Bitbake rev: 8e12c8f8441a7c6a03e603c5789d6037945704c1) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: clean up stamp-base related codesChen Qi2016-01-111-4/+4
| | | | | | | | | | | | The 'stamp-base' and 'stamp-base-clean' related codes are no longer useful, clean them up. [YOCTO #8468] (Bitbake rev: 7b4c42b315d4a26dd8f2ceb874a94737bf9f183e) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Add BB_TASK_IONICE_LEVEL supportRichard Purdie2015-10-291-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Similarly to BB_TASK_NICE_LEVEL, add BB_TASK_IONICE_LEVEL which allows the ioprio of tasks to be adjusted. This is in response to various qemu runtime timeouts which have been witnessed on the autobuilder, seemingly due to IO starvation (we already use NICE_LEVEL to adjust tasks). This has a fairly urgent need to deal with certain 'random' failures we're seeing on the autobuilders in testing. The format of the data in the variable is BB_TASK_IONICE_LEVEL = "<class>.<prio>". For <class>, 2 is best effort (the default), 1 is real time and 3 is idle. You'd need superuser privileges to use realtime. The <prio> value is a default of 4, and can be set between 0 and 7 with 7 being lowest priority and 0 the highest. The user can set this freely with normal privileges Note that in order for this to take effect, you need the cfq scheduler selected for the backing block device. We could use nice wrapper functions for ioprio from modules like psutil however that would complicate bitbake dependencies. This version has some magic numbers but works on the main 32 and 64 bit x86 build architectures and can easily be extended if ever needed. When we move to python 3.x, we can likely replace this with standard calls. (Bitbake rev: b9471ad147b102c45d65f5ffd9521864df7ff9c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: delete tasks thoroughlyChristopher Larson2015-09-031-10/+13
| | | | | | | | | | | | | | | We want addtask to be able to bring back a deleted task, but we don't want its previous dependencies to come back with it, so rather than marking a task as deleted and then skipping tasks marked as such, actually delete the task and its dependency information in deltask. While we're in that part of the code, also fix a couple 'not foo in bar' instances to 'foo not in bar', which is preferred in python. (Bitbake rev: 94b3f3d6bdfbfa47f7eb3c3de64940a145b2ddd1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Exit scripts with result of last shell functionAndre McCurdy2015-07-271-1/+1
| | | | | | | | | | | | | Since shell scripts run with 'set -e' the final exit at the end of the script can only be returning 0. However, for correctness and to follow the original intention of the 'cleanup' commands, let's fix the typo and return the success of the last shell function rather than the success of unhooking the exit trap handler. (Bitbake rev: bef724057f1ea81571dd3ac5a9cf862f818434b5) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: provide mechanism to bypass UI log suppressionPaul Eggleton2015-07-161-1/+6
| | | | | | | | | | | | | | | | | | The recent change to connect through the shell logging functions had an unexpected side-effect - bb.error() and bb.fatal() cause a flag to be set internally such that BitBake's UI will not print the full task log on failure; unfortunately we have in places within the OpenEmbedded metadata called these shell logging functions under error situations where we still want to see the full log (i.e., the message we're sending doesn't include the full error). Thus, provide a mechanism to fatally exit with an error but unset the flag, utilising the built-in python logging functionality that allows extra values to be passed in the log record. (Bitbake rev: e561b997c55e8537d82aa1339adfff4505cc38b7) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: set up infrastructure for shell message functionsPaul Eggleton2015-07-121-7/+43
| | | | | | | | | | | | | | | | | Create a fifo under ${T} for each running task that can be used by the task to send events back to BitBake. The purpose of this is to allow OpenEmbedded's logging.bbclass (which provides shell function equivalents for bb.warn(), bb.note() etc.) to have those functions trigger the appropriate events within BitBake so that messages are shown through the BitBake UI rather than just in the task log; to do that we just call the python functions. Part of the fix for [YOCTO #5275]. (Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add explict getVar param for (non) expansionRichard Purdie2015-06-231-4/+4
| | | | | | | | | | | | | | Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: reset build mtime cache before the buildEd Bartosh2015-04-291-1/+16
| | | | | | | | | | | | | Introduced build mtime cache structure. Reset it before the build to prevent bitbake from crashing when build/tmp/stamps hierarchy is removed. [YOCTO: #7562] (Bitbake rev: f8590547a198a78334debdf14bf40acb50c22ecc) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/data: Write out more complete python run filesRichard Purdie2014-08-281-1/+1
| | | | | | | | | | | | | Currently the output in the python task/function run files is rather incomplete and effectively useless. This enhances the code to take advantage of the bitbake's dependency tracking and extend the output to include dependencies. This makes the files more usable for debugging purposes. Since this only happens at python function execution time, the overhead is minimal in the grand scheme of things. (Bitbake rev: 02667e048c3e632f857c87177c0022eaf5481802) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/*.py: Typo fixes/grammar/comment fixes, nothing functional.Robert P. J. Day2014-08-251-6/+5
| | | | | | | (Bitbake rev: 587b144ee409d444494d8d7f2d1c53ede8f7c953) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: Ensure shared work siginfo files are written to the ↵Richard Purdie2014-06-191-2/+6
| | | | | | | | | | | | | correct location Right now shared work signature data is saved to the non-shared directory which is confusing to everyone including bitbake. Whilst its messy, extra the stampbase data instead, which ensures the sig data is written to the correct location alongside its corresponding stamp file. (Bitbake rev: 7ae1d4844d9d3a76f86ef32c5a794e51e334e588) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Fix broken exception handlingRichard Purdie2014-06-011-3/+2
| | | | | | | | | | Checking for explicit exception names is bad, we also want to be able top rely on inheritance. Fix these checks to be part of the real except clauses so SkipPackage is recognised as being inherited from SkipRecipe. (Bitbake rev: b131229145e1f2c372d6230a7b554e436c13c3f9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Add SkipRecipe event to replace SkipPackageRichard Purdie2014-06-011-1/+1
| | | | | | | | | | | | In the depths of time we were rather confused about naming. bb files are recipes, the event to skip parsing them should be SkipRecipe, not SkipPackage. This changes bitbake to use the better name but leaves the other around for now. We can therefore start removing references to it from the metadata. (Bitbake rev: 98d9e6e0f514a7cb7da1d99bf4bd5602b89426d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: measure task duration with server-side timestampsAlexandru DAMIAN2014-03-211-0/+2
| | | | | | | | | | | | | | | | | | | | | The buildstats and toaster use separate time markers to measure the duration of task execution. This causes a mismatch in the time measured by buildstats class and the time measured in toaster. The solution implemented here is to timestamp the creation of every TaskBase event on the bitbake server side and calculate the execution duration as the difference between creation time of TaskSucceeded and TaskStarted events. Based on an original patch by Marius Avram. [YOCTO #5485] (Bitbake rev: 7a08282c074c264f414cf7665dc873f51245072c) Signed-off-by: Marius Avram <marius.avram@intel.com> Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: filter out expanded empty strings for lockfiles flagStefan Stanacar2014-02-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | If we have something similar to: LOCKFILES = "${TMPDIR}/my.lock" LOCKFILES_qemuall = "" do_task[lockfiles] += "${LOCKFILES}" when expanded, lockfiles will be empty for qemu, resulting in File "/home/stefans/yocto/poky/bitbake/lib/bb/utils.py", line 630, in mkdirhier raise e OSError: [Errno 2] No such file or directory: '' This should filter out the empty expansions. (Bitbake rev: 7813e1bfd08cd48871f8c03cae2810265590105d) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: fix handling of task override for tasks with underscores in ↵Paul Eggleton2014-02-171-1/+1
| | | | | | | | | | | | | | their names Tasks whose names contain underscores (such as do_populate_sdk in OE) when converted to a task override do not function properly. If we replace underscores with hyphens we can still have a working override for these tasks. (Bitbake rev: cf90bd6b2a0ab7dce922bffb500d6a2ff2ff10e2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: build.py: create separate function for shell trap creation ↵Laurentiu Palcu2014-02-091-17/+20
| | | | | | | | | | | | | | | code Currently, the shell trap code was created in exec_func_shell(). Split the function so that we can create the code separately. Also, some whitespaces were automatically deleted by my editor. Since this is not necessarily a bad thing, leave these changes too. (Bitbake rev: c712e622d20c61a07c9c172b60e9dc6beae14197) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/lib/bb/build.py: fix the task flags cleandirsRobert Yang2014-01-071-0/+1
| | | | | | | | | | | | | | | The user manual said: 'cleandirs' - directories which should created before the task runs but should be empty But it only removes the dir, doesn't create it [YOCTO #5703] (Bitbake rev: 0636797d75874ce4577f29011d69c56a4c6b9e89) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/ast: Create strong task add/del API in bb.buildRichard Purdie2013-12-181-5/+32
| | | | | | | | | | | | | | | Currently its near impossible to control task addition/deletion from metadata context. This adds stong add/deltask API to bb.build which is traditionally where it resided. The rather broken remove_tasks function was removed, it didn't appear to do anything useful or have any users. This allows us to clean up hacks currently in use in metadata and use standard API for it instead. (Bitbake rev: bf7138dd38fc1f8efca80891198e3422fef64093) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/BBHandler/build: Add support for removing tasks (deltask)Richard Purdie2013-12-181-1/+5
| | | | | | | | | | | | | | | | | Back in the depths of time we did support task removal. In the pre AST days it was nearly impossible to continue supporting it, it wasn't used so it was dropped. With the modern codebase we can easily now support deltask and it would be very useful within the metadata since it can massively simplify dependency trees. As an example, a core-image-sato had 47703 inter task dependencies before this patch and a patch to native.bbclass, afterwards with the noexec tasks deleted, we had 29883. Such a significant simplification is worthwhile and justifies adding a deltask operation to the system. (Bitbake rev: acecbde6fb70ff3c96deab3cdf819d8442e87ed4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build.py: add single-quotes around already-expanded directory namePeter Seebach2013-11-201-1/+1
| | | | | | | | | | | | | | | | | | | If the computed name of a directory contains an undefined variable reference, bitbake dutifully creates a directory with a name that has ${...} in it. However, the actual task script created then tries to cd to that directory, and the cd command fails, because no such directory exists -- because the shell has helpfully removed the ${...} which did not match any actual variables. Since we want the name to be used exactly-as-is, add single quotes around the name so this doesn't cause strange failures running tasks, which allows us to progress past such failures and get to a point where they can be diagnosed. (Bitbake rev: 2809c2e6f2f35f9b08058950be896947ab5a0284) Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-14 20:22:44 +0000