linux/poky.git/bitbake/lib/bb/tests/data.py, branch 4.2_M1

bitbake: data_smart: allow python snippets to include a dictionary

2022-11-10T14:43:06+00:00

[YOCTO #14917] Attempting to use a dictionary in a python code snippet for variable assignment results in an error. For example attempting something such as IDX = "green" VAL = "${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]}" produces the error expansion of VAL threw ExpansionError: Failure expanding variable VAL, expression was ${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]} which triggered exception SyntaxError: '{' was never closed (Var , line 1) The existing __expand_python_regexp__, "\${@.+?}", will match the first close curly bracket encountered, resulting in incomplete and un-parsable code, and thus produce the error. We can correct this by allowing a single depth of nested curly brackets in __expand_python_regexp__ by using "\${@(?:{.*?}|.)+?}", which will match up to and including the matching close curly bracket to the open, '${@', curly bracket, even if there are one or more singly nested curly brackets present. This change allows the usecase described above to function. This change can't be made on its own though. The old regex would, in an obscure way, handle the case where a python snippet contained an unexpandable variable. Since the unexpandable variable is in curly brackets it would cause incomplete/un-parsable python code and thus remain unparsed. So something like VAL = "${@d.getVar('foo') + ${unsetvar}}" would remain unparsed as the close curly bracket in "${unsetvar}" would match and terminate the snippet prematurely. This quirk resulted in the proper handling of python snippets with unexpanded variables. With the change to __expand_python_regexp__ the full snippet will match and be parsed, but to match the old/correct behavior we would not want to parse it until ${unsetvar} can be expanded. To ensure the old/correct behavior for python snippets with unexpanded variables remains in place we add a check for unexpanded variables in the python snippets before running them. This handling of unparsed variables brings two benefits. The first we now have an explicit check visible to all for unexpanded variables instead of a somewhat hidden behavior. The second is that if there are multiple python snippets the old behavior would run the code for each but a single snippet with unexpanded variables would mean all snippets would remain unparsed, meaning more and repeated processing at a later time. For example: "${@2*2},${@d.getVar('foo') ${unsetvar}}" old behavior would give: "${@2*2},${@d.getVar('foo') ${unsetvar}}" new behavior will give: "4,${@d.getVar('foo') ${unsetvar}}" The old behavior would calculate '2*2' but toss the result when the second snippet would fail to parse resulting in future recalculations (or fetching from cache), while the new behavior avoids this. (Bitbake rev: 94e49b9b9e409c29eb04603b1305d96ebe661a4b) Signed-off-by: Mark Asselstine Signed-off-by: Richard Purdie

bitbake: data_smart: Fix inactive overide accidental variable value corruption

2021-08-04T09:38:43+00:00

Setting something like: BAR:append:unusedoverride should cause BAR to be None, not "" which was what the datastore was returning. This caused problems when mixing variables like: RDEPENDS:${PN}:inactiveoverride RDEPENDS:${BPN} since key expansion would report key overlap when there was none. This is a bug in the datastore. Fix it and add a test too. [YOCTO #14088] (Bitbake rev: 699e36c270d863258502d315ed00a1b940bfbf96) Signed-off-by: Richard Purdie

bitbake: doc/lib: Add fixes for issues missed by the automated conversion

2021-08-02T14:44:10+00:00

The examples and tests use non-standard override names, convert these to the new syntax by hand. (Bitbake rev: a6c40eca1146c0160da7e4e0bd7ac52fef2029e0) Signed-off-by: Richard Purdie

bitbake: doc/lib: Update to use new override syntax containing colons

2021-08-02T14:44:10+00:00

This runs the overrides conversion script in OE-Core over the bitbake code base including the docs. A handful of things were excluded in toaster and for the Changelog file. (Bitbake rev: 47f8d3b24fd52381bf3b41e2f55a53e57841344c) Signed-off-by: Richard Purdie

bitbake: lib: fix most undefined code picked up by pylint

2020-08-25T17:14:53+00:00

Correctly import, and inherit functions, and variables. Also fix some typos and remove some Python 2 code that isn't recognised. (Bitbake rev: b0c807be5c2170c9481c1a04d4c11972135d7dc5) Signed-off-by: Frazer Clews Signed-off-by: Richard Purdie

bitbake: data/siggen: Don't expand ignored variables

2020-04-06T15:45:34+00:00

If a variable is in the signature whitelist, we'd currently expand it, then later ignore the data. This is problemtic for code which has effects when expanded, recently source date epoch in OE-Core for example. We don't actually need to do this, if we pass the whitelist into the earlier function it can avoid the expansion. This also also give a small performance boost since we avoid running code in some cases. [YOCTO #13581] (Bitbake rev: f483ee4a869fb1dafbe4bdf2da228cdaa40b38bd) Signed-off-by: Richard Purdie

bitbake: tinfoil: Simplify remote datastore connections

2020-03-24T22:01:03+00:00

The current approach to remote datastores used in tinfoil is breaking. For example, adding a devupstream extension to a recipe with a git upstream, making it the preferred version and then running "devtool modify" on it causes get_srcrev() circular dependency issues. The problem is the override handling in the datastore is broken. This gets broken since remotedata:recieve_datastore() sets d.dict but doesn't update d.overridedata (or d.inchistory or d.varhistory). We could play whack-a-mole but the current implementation seems to be flawed to me. It also doesn't cover, or only partially covers some datastore operations and each needs new dedicated command API. Instead, step back and reimplement the way the datastore connector works. With this change, the datastore is either remote or local but the data is not spread on two sides of the connection. All the API is proxied over the connection by a single function for the datastore (and two to support variable history and include history). This code does not support using the datastore as a parameter to any data store functions. We did have one case of that but its just bad code and can be replaced. The result is something which is much simpler and less invasive to the datastore code itself, meaning its behaviour should be much more consistent. The existing tests for the remote data no longer make any sense and are removed. The one bug this code would have is if key/value pairs are returned over the IPC and those values contained a DataSmart object since we don't recurse into return values to find such things. Nothing appears to do that currently so lets worry about it if its ever an issue. This change should simplfy a ton of other issues and avoid a ton of other bugs so is a huge net gain. Tested with bitbake's and OE's selftests. (Bitbake rev: 85e03a64dd0a4ebe71009ec4bdf4192c04a9786e) Signed-off-by: Richard Purdie

bitbake: tests/data: Test combinations of _append together with override

2019-09-27T12:02:19+00:00

(Bitbake rev: f31f35e8527c60a95931a4a8311a4cd237770b42) Signed-off-by: Jacob Kroon Signed-off-by: Richard Purdie

bitbake: siggen: Clean up task reference formats

2019-08-06T10:21:32+00:00

Currently siggen uses the format "." for referencing tasks whilst runqueue uses ":". This converts to use ":" as the separator everywhere. This is an API breaking change since the cache is affected, as are siginfo files and any custom signature handlers such as those in OE-Core. Ultimately this will let us clean up and the accessor functions from runqueue, removing all the ".rsplit(".", 1)[0]" type code currently all over the place. Once a standard is used everwhere we can update the code over time to be more optimal. (Bitbake rev: 07e539e1c566ca3434901e1a00335cb76c69d496) Signed-off-by: Richard Purdie

bitbake: bitbake: Strip old editor directives from file headers

2019-05-04T09:44:10+00:00

There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. (Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0) Signed-off-by: Richard Purdie