linux/poky.git/bitbake/lib/bb/tests, branch 4.2_M1

bitbake: data_smart: allow python snippets to include a dictionary

2022-11-10T14:43:06+00:00

[YOCTO #14917] Attempting to use a dictionary in a python code snippet for variable assignment results in an error. For example attempting something such as IDX = "green" VAL = "${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]}" produces the error expansion of VAL threw ExpansionError: Failure expanding variable VAL, expression was ${@{ 'green': 1, 'blue': 2 }[d.getVar('IDX')]} which triggered exception SyntaxError: '{' was never closed (Var , line 1) The existing __expand_python_regexp__, "\${@.+?}", will match the first close curly bracket encountered, resulting in incomplete and un-parsable code, and thus produce the error. We can correct this by allowing a single depth of nested curly brackets in __expand_python_regexp__ by using "\${@(?:{.*?}|.)+?}", which will match up to and including the matching close curly bracket to the open, '${@', curly bracket, even if there are one or more singly nested curly brackets present. This change allows the usecase described above to function. This change can't be made on its own though. The old regex would, in an obscure way, handle the case where a python snippet contained an unexpandable variable. Since the unexpandable variable is in curly brackets it would cause incomplete/un-parsable python code and thus remain unparsed. So something like VAL = "${@d.getVar('foo') + ${unsetvar}}" would remain unparsed as the close curly bracket in "${unsetvar}" would match and terminate the snippet prematurely. This quirk resulted in the proper handling of python snippets with unexpanded variables. With the change to __expand_python_regexp__ the full snippet will match and be parsed, but to match the old/correct behavior we would not want to parse it until ${unsetvar} can be expanded. To ensure the old/correct behavior for python snippets with unexpanded variables remains in place we add a check for unexpanded variables in the python snippets before running them. This handling of unparsed variables brings two benefits. The first we now have an explicit check visible to all for unexpanded variables instead of a somewhat hidden behavior. The second is that if there are multiple python snippets the old behavior would run the code for each but a single snippet with unexpanded variables would mean all snippets would remain unparsed, meaning more and repeated processing at a later time. For example: "${@2*2},${@d.getVar('foo') ${unsetvar}}" old behavior would give: "${@2*2},${@d.getVar('foo') ${unsetvar}}" new behavior will give: "4,${@d.getVar('foo') ${unsetvar}}" The old behavior would calculate '2*2' but toss the result when the second snippet would fail to parse resulting in future recalculations (or fetching from cache), while the new behavior avoids this. (Bitbake rev: 94e49b9b9e409c29eb04603b1305d96ebe661a4b) Signed-off-by: Mark Asselstine Signed-off-by: Richard Purdie

bitbake: tests/fetch: Allow handling of a file:// url within a submodule

2022-10-26T22:02:13+00:00

CVE-2022-39253 in git meant file:// urls within submodules were disabled. Add a parameter to the commands in the tests to allow this to continue to work. (Bitbake rev: 209f7ba352b60722830157054e3fc56cb9c693eb) Signed-off-by: Richard Purdie

bitbake: tests: bb.tests.fetch.URLHandle: add 2 new tests

2022-10-26T11:32:08+00:00

Add a test for special characters in user and password to qualify decodeurl() inspired by a bug report describing that '=' signs in a password was problematic. Add a second test to qualify decodeurl() as related to the change in commit 628c4bf6c89b [fetch2/__init__: handle @ in package names]. Relates to [YOCTO #14476] (Bitbake rev: ee04cf09c7022168c035affa654773652a49793e) Signed-off-by: Mark Asselstine Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

bitbake: tests: Add test for possible gitsm deadlock

2022-09-01T09:07:00+00:00

(Bitbake rev: b0506480baa9bcf3ef645b0aed5a07ad9950245c) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: tests: Add Timeout class

2022-09-01T09:07:00+00:00

The class and exception aim to test rare cases there deadlocks are possible. Can be used in context managers: with Timeout(): do_deadlock() (Bitbake rev: c5fcdd804d422f959a189b270d72123a50e74da6) Signed-off-by: Pavel Zhukov Signed-off-by: Richard Purdie

bitbake: BBHandler/cooker: Implement recipe and global classes

2022-08-12T10:49:29+00:00

We have some confusion for users since some classes are meant to work in the configuration space (or "globally") and some are meant to be selected by recipes individually. The cleanest way I could find to clarify this is to create "classes-global" and "classes-recipe" directories which contain the approproate classes and have bitbake switch scope between them at the appropriate point during parsing. The existing "classes" directory is always searched as a fallback. Once a class is moved to a specific directory, it will no longer be found in the incorrect context. A good example from OE is that INHERIT += "testimage" will no longer work but IMAGE_CLASSES += "testimage" will, which makes the global scope cleaner by only including it where it is useful and intended to be used (images). (Bitbake rev: f33ce7e742f46635658c400b82558cf822690b5e) Signed-off-by: Richard Purdie

bitbake: bitbake: Add copyright headers where missing

2022-08-12T10:49:29+00:00

Where copyright headers were not present, add them to make things clear. (Bitbake rev: 1aa338a216350a2751fff52f866039343e9ac013) Signed-off-by: Richard Purdie

bitbake: fetch2: Drop DL_DIR fallback for local file fetcher

2022-07-15T11:22:08+00:00

A long time ago, we made DL_DIR a final fallback for the local fetcher. Since then we added checksum support and task hashes and the world has changed. There were warnings added some time ago if this fallback triggers and it is now time to drop it entirely. The original use case was for sstate however the sstate code now sets FILESPATH correctly so DL_DIR is no longer needed. There have been a few small bugs exposed by this change, missing mkdir calls and some minor test issues that needed tweaks. In general this simplifies and improves the fetcher code flow though. This completes a cleanup that ensures local files are correctly covered at parse time which ensures rebuilds and reparses happen at the right times. (Bitbake rev: 3e1444e536c71d3885ef6b9d428807163c309640) Signed-off-by: Richard Purdie

bitbake: fetch: bb.fatal when trying to checksum non-existing files

2022-07-15T11:22:08+00:00

If the local fetcher was not able to find the file anywhere but it was included in the SRC_URI for checksumming just make it a fatal error. Ensure a list of searched locations is included too to match the runtime error that would have resulted. (Bitbake rev: 5e3b2ad90d9cd0f248b1cb740637caa24442d101) Signed-off-by: Paulo Neves Signed-off-by: Richard Purdie

bitbake: ConfHandler/BBHandler: Improve comment error messages and add tests

2022-07-04T21:52:36+00:00

Currently if you trigger one of the comment errors, the newline characters are stripped and the line numbers are incorrect. In one case it prints the empty line which is also unhelpful. Rework the code around these errors so the line numbers are correct and the lines in question are more clearly displayed complete with newlines so the user can more clearly see the error. I also added a couple of simplistic test cases to ensure that errors are raised by the two known comment format errors. [YOCTO #11904] (Bitbake rev: 712da71b24445c814d79a206ce26188def8fce0a) Signed-off-by: Richard Purdie