linux/poky.git/bitbake/lib/bb/data_smart.py, branch 2.7_M2

bitbake: bb.data_smart: only try to expand refs to valid variable names

2019-01-21T23:44:34+00:00

This aligns the behavior of expansion with the recipe parser, only attempting to expand references to valid variable names. This avoids adding references for things like `${foo#${TOPDIR}}` to our vardeps without imposing much additional processing overhead beyond the change to the expansion regexp. YOCTO #12987 (Bitbake rev: df2ac65370aa86cdbc1574fdede25e3519410e45) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: data/siggen: Switch md5 -> sha256

2019-01-08T11:16:03+00:00

Similarly to the codeparser change, change to sha256 hashes due to worries over collisions. The main impact of this change is slightly slower parsing time as well as longer sstate file names. (Bitbake rev: 66f1b766997d53b4375fdd25719b1175f3828903) Signed-off-by: Richard Purdie

bitbake: data_smart: Allow numeric characters in overrides

2018-12-14T17:10:59+00:00

We're seeing problems due to the way x86-64 is handled (or not handled) as an override. Relax the containts on overrides from being lowercase to being lowercase or numeric. This fixes problem where MACHINE=qemux86 would work but MACHINE=qemux86-64 would fail the same tests. (Bitbake rev: 3a3be518536acc868c7eeb3c1111ad1b321480b7) Signed-off-by: Richard Purdie

bitbake: data_smart: Add missing regexp markup

2018-12-14T17:10:59+00:00

Fix some further python3 warnings about unescaped regexs. (Bitbake rev: 8667605d016e82add95638fcb15c2bbc1b489ecc) Signed-off-by: Richard Purdie

bitbake: data_smart: fix filename for compile()

2018-11-19T17:03:25+00:00

Fixed: Add the following two lines to conf/local.conf: FOO = "${@foo = 5}" HOSTTOOLS += "${FOO}" * Before the patch $ bitbake -p Check the first lines of bitbake bitbake-cookerdaemon.log [snip] File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 125, in python_sub codeobj = compile(code.strip(), self.varname or "", "eval") File "FOO", line 1 [snip] There isn't a file named 'FOO', but a variable name. * After the patch $ bitbake -p [snip] File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 129, in python_sub codeobj = compile(code.strip(), varname, "eval") File "Var ", line 1 foo = 5 (Bitbake rev: 540b546be55e0f5f5d91695956da3a7732b2f90a) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: data_smart: Add original traceback to ExpansionError

2018-11-19T17:03:25+00:00

This can make it print clearer errors, for exmaple: Add Runtime_error to 'def oe_import(d)" 16 def oe_import(d): 17 import sys 18 Runtime_error [snip] * Before the patch: $ bitbake -p ERROR: Unable to parse /buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py Traceback (most recent call last): File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 430, in DataSmart.expandWithRefs(s='${@oe_import(d)}', varname='OE_IMPORTED[:=]'): except Exception as exc: > raise ExpansionError(varname, s, exc) from exc bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception NameError: name 'Runtime_error' is not defined This error message has two problems: - "Unable to parse data_smart.py": This isn't the real cause. - It pionts to "raise ExpansionError(varname, s, exc) from exc" which isn't clear enough. * After the patch: $ bitbake -p ERROR: Unable to parse OE_IMPORTED[:=] Traceback (most recent call last): File "OE_IMPORTED[:=]", line 1, in File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 18, in oe_import(d=): import sys > Runtime_error bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception NameError: name 'Runtime_error' is not defined This one is more clearer than before. (Bitbake rev: c0fe524c1aeccb24ddd2e1f7bf235c00fdbf79a7) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: data_smart: Fix removal handling interaction issue with overrides

2018-10-20T13:27:03+00:00

If a variable has a _remove applied to it but that variable is in turn 'renamed' through OVERRIDES, the removal gets lost with the current code. TEST = "foo" TEST_someval = "bar" TEST_someval_remove = "bar" OVERRIDES = "someval" currently gives "bar" for TEST but should give "". This fixes the code to track the removal and adds a test case to ensure this doesn't regress again. (Bitbake rev: 8f55010c18057be040f073d8bcb4c5c2c311d809) Signed-off-by: Richard Purdie

bitbake: bitbake: data: Ensure task checksums account for remove data

2018-10-18T09:59:27+00:00

Currently remove operations are not being accounted for in the task checksums. This is a fairly serious oversight and needs to be fixed. To do so, we need internal data from getVarFlag combined with the expanded variable data so that only "active" remove operators are accounted for in the task checksum. We can get this from the new optional removes attribute in the returned parser object. The code can then use the data on active remove operators to account for the removals in task checksum but only when the removal is active. We have to be careful here not to reference any expanded data since this may for example contain build paths. This means we can only map back and reference the unsplit (and hence unexpanded) remove string which may expand to multiple removal values. [YOCTO #12913] (Bitbake rev: 57d2ee17ae83a139a37081eb082e6184fa883581) Signed-off-by: Richard Purdie

bitbake: data/data_smart: Allow getVarFlag to return the variable parser object

2018-10-18T09:59:27+00:00

(Bitbake rev: 136100dc932c9019737f927d826955425134010f) Signed-off-by: Richard Purdie

bitbake: data_smart: Fix expand_cache and _remove operator interaction issues

2018-10-18T09:59:27+00:00

The contents of the expand_cache is meant to match the return value of getVarFlag() but the implementation was mostly in expandWithRefs(). If an incorrect key was passed to expandWithRefs(), or a variable was only partially expanded with no remove processing, the cache could become corrupted. Move the code to getVarFlag making the data lifecycle very clear, meaning other calls to expandWithRefs() cannot corrupt the cache. The expand_cache reset code needs to be moved ahead of any remote data connectors too, since the expand_cache is now on the local side of the connection. (Bitbake rev: a039052f9b680eae53f3f12b7381b945f1d69253) Signed-off-by: Richard Purdie