Mirror of git.yoctoproject.org/poky https://git.enea.com/cgit/linux/poky.git/atom?h=2.2_M2 2016-07-19T07:56:50+00:00 2016-07-19T07:56:50+00:00 Ross Burton ross.burton@intel.com 2016-07-14T18:56:22+00:00 urn:sha1:2afcbfef2cd1ca568e5225884a8021df38ee3db0 When bitbake executes a shell or Python function it can cd/chdir() into a directory before executing the task. If no directory is specified then the default of $B is used. However $B is an OpenEmbedded variable and BitBake shouldn't be aware of it. To solve this change the semantics slightly so that if no directory is specified, the current working directory isn't changed. There's also a sanity check that emits a warning if a Python task does os.chdir() without restoring the old path, and the previous working directory is restored. This does change semantics: whereas before a function in OE would have $B as the working directory unless specified, now the working directory is the top of the build tree. Any breakage this causes can be solved by either adding do_some_task[dirs] = "${B}" or by using absolute paths in the task. [ YOCTO #4634 ] (Bitbake rev: 67a7b8b021badc17d8fdf447c250e79d291e75f7) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-07-19T07:47:06+00:00 Ross Burton ross.burton@intel.com 2016-07-15T10:25:42+00:00 urn:sha1:faa726824dfb5bc0141919bb16c0d0069a3da69e The messaging FIFO is UTF-8, so decode the command as UTF-8 as well as the value as otherwise "bberror" != b("bberror") and none of the messages from shell functions are ever displayed. Also add an else to the command parser so unhandled commands are noticed. [ YOCTO #9947 ] (Bitbake rev: 42d727743fa599e0a3c5ad2c29a1e6ede1a918bb) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-07-08T08:57:26+00:00 Paul Eggleton paul.eggleton@linux.intel.com 2016-06-23T10:59:05+00:00 urn:sha1:ac5e720575dd3c8f86514a7a646de82c8cc28e17 For long-running tasks where we have some output from the task that gives us some idea of the progress of the task (such as a percentage complete), provide the means to scrape the output for that progress information and show it to the user in the default knotty terminal output in the form of a progress bar. This is implemented using a new TaskProgress event as well as some code we can insert to do output scanning/filtering. Any task can fire TaskProgress events; however, if you have a shell task whose output you wish to scan for progress information, you just need to set the "progress" varflag on the task. This can be set to: * "percent" to just look for a number followed by a % sign * "percent:<regex>" to specify your own regex matching a percentage value (must have a single group which matches the percentage number) * "outof:<regex>" to look for the specified regex matching x out of y items completed (must have two groups - first group needs to be x, second y). We can potentially extend this in future but this should be a good start. Part of the implementation for [YOCTO #5383]. (Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-06-23T13:26:33+00:00 Robert Yang liezhi.yang@windriver.com 2016-06-15T02:42:10+00:00 urn:sha1:c6d50b272936b61b98a056ea2e1de5d0f4ef53ce Otherwise the function like d.getVarFlag(e, 'task', True) which is used by do_listtasks will still get it, and list the deleted tasks. (Bitbake rev: 779d73619daf59f76f5b0313e7fb5409f6e82553) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-06-02T07:24:02+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2016-05-12T07:30:35+00:00 urn:sha1:0f2c59367a649de5f57acdccfb4f1fdba9cde730 Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-05-11T09:34:30+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2016-05-09T13:01:12+00:00 urn:sha1:654eadfa30624c62018665da618287b6cb5c7e3c python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-05-06T09:18:12+00:00 Christopher Larson chris_larson@mentor.com 2016-04-30T19:43:52+00:00 urn:sha1:2d2d3121880d65e52cdffe6b6f869aa3f2dd6410 Fixes pypy support. (Bitbake rev: e90cfc655affeec8f5519f7078dad5f99db3c461) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-03-31T08:13:04+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2016-03-30T19:52:56+00:00 urn:sha1:d73da22b0e332411ed5552d36910fe15f92c8fac With the code as it stands today it not possible to execute a python function and get "normal" python exception handling behaviour. If a python function raises an exception, it forces a traceback to be printed and the exception becomes a FuncFailed exception. This adds in a parameter 'pythonexception' which allows standard python exceptions to be passed unchanged with no traceback. Ultimately we may want to change to this convention in various places but at least it means we can start to add sane functions now. (Bitbake rev: 85cf22fd0ed26bb7dc7738ef2a10441891f38ae2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-02-10T15:47:11+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2016-02-09T15:08:42+00:00 urn:sha1:400f53003e430e1a5f7302a0a40ed5afbf80904e If something fails in a exec_func_python() the current stack trace shows incorrect filenames and linenumbers. For example: The stack trace of python calls that resulted in this exception/failure was: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 200, function: <module> 0196: chksum = bb.utils.sha256_file(fn) 0197: f.write('%s\t%s\n' % (chksum, os.path.relpath(fn, baseoutpath))) 0198: 0199: *** 0200:copy_buildsystem(d) 0201: File: '/media/build1/poky/meta/recipes-sato/images/core-image-sato.bb', lineno: 9, function: copy_buildsystem 0005:IMAGE_FEATURES += "splash package-management x11-base x11-sato ssh-server-dropbear hwcodecs" 0006: 0007:LICENSE = "MIT" 0008: *** 0009:inherit core-image 0010: 0011:IMAGE_INSTALL += "packagegroup-core-x11-sato-games" File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] The problem is the use of "FILE" to obtain the current filename. Instead, we therefore inject the function being executed into the methodpool which allows us to correct its linenumber and filename information. We can then clearly mark the initial piece as autogenerated and the rest of the linenumber and filename information should be correct. Afterwards the trace starts: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:copy_buildsystem(d) 0003: File: '/media/build1/poky/meta/classes/populate_sdk_ext.bbclass', lineno: 66, function: copy_buildsystem 0062: import glob 0063: import oe.copy_buildsystem 0064: import subprocess 0065: *** 0066: subprocess.check_call("foo") 0067: 0068: oe_init_env_script = d.getVar('OE_INIT_ENV_SCRIPT', True) 0069: 0070: conf_bbpath = '' File: '/usr/lib/python2.7/subprocess.py', lineno: 535, function: check_call 0531: The arguments are the same as for the Popen constructor. Example: 0532: 0533: check_call(["ls", "-l"]) 0534: """ *** 0535: retcode = call(*popenargs, **kwargs) 0536: if retcode: 0537: cmd = kwargs.get("args") 0538: if cmd is None: 0539: cmd = popenargs[0] We can't inject into methodpool at parsing time, since there may be _append or other override operations against the function before its execution. (Bitbake rev: fae153095d23157dd7e72c29f683f86149ee33a8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> 2016-02-10T15:47:11+00:00 Richard Purdie richard.purdie@linuxfoundation.org 2016-02-02T14:07:58+00:00 urn:sha1:aece74876271f692296c5f792104c627e15b5c3e Right now, if you have some python code like: X = "a" def somefunction(d): d.setVar("X", "b") d.setVar("Y", "${X}") then any sane person would expect that Y = "b" at the end of the function. This is not the case, Y = "a". This is due to the python function being expanded before execution, the executed code would read d.setVar("Y", "a"). This understandably confuses people, it also makes it near impossible to write ${} in a python function without unintended things happening. I think there is general agreement we should fix this and standardise on non-expansion of python functions. We already don't expand anonymous python (mostly). I've checked OE-Core with buildhistory before and after this change and there were a small number of issues this exposed which I've sent patches for. I propose we default to not expanding python code and then deal with any consequences from that if/as/where identified. This will improve new user understanding and usability of the system, it also allows several long standing weird expansion issues to be fixed. (Bitbake rev: 8bf33a8e92c0e188fa392030025756196c96fcbb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>