linux/poky.git/bitbake/lib/bb/codeparser.py, branch kirkstone-4.0.24

bitbake: codeparser: Fix handling of string AST nodes with older Python versions

2024-11-15T14:05:32+00:00

Commits 4591011449212c8e494ea42348acb2d27a82a51b and 6c19b6cf105ac321ec89da1a876a317020c45ab7 unconditionally changed codeparser to rely on CPython 3.8 semantics. However, kirkstone continues to support CPython versions >= 3.6.0 and as such string AST nodes were no longer correctly identified. Fix this by continuing to use `ast.Str` for Python versions < 3.8.0 and only using the new code path for more recent versions. Detecting which version of the AST API to use seems to be non-trivial so the Python feature version is used instead. Instances of this issue can be identified when executing bitbake with debug logging: while parsing MACHINE_ARCH, in call of d.getVar, argument ''TUNE_PKGARCH'' is not a string literal As a consequence of these parsing issues, bitbake may assume that task inputs haven't changed and as such erroneously reuse sstate objects when it shouldn't. (Bitbake rev: fb73c495c45d1d4107cfd60b67a5b4f11a99647b) Signed-off-by: Philip Lorenz Signed-off-by: Steve Sakoman

bitbake: bitbake/codeparser.py: address ast module deprecations in py 3.12

2024-03-01T18:00:58+00:00

Specifically: /srv/work/alex/poky/bitbake/lib/bb/codeparser.py:279: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead if isinstance(node.args[0], ast.Str): /srv/work/alex/poky/bitbake/lib/bb/codeparser.py:280: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead value = node.args[0].s (Bitbake rev: 6c19b6cf105ac321ec89da1a876a317020c45ab7) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: codeparser: replace deprecated ast.Str and 's'

2024-03-01T18:00:58+00:00

These have been deprecated since 3.8 (Bitbake rev: 4591011449212c8e494ea42348acb2d27a82a51b) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: bitbake: Add copyright headers where missing

2022-10-11T20:57:28+00:00

Where copyright headers were not present, add them to make things clear. (Bitbake rev: e591325b2bd901c381003deb96a7b32a7148e93e) Signed-off-by: Richard Purdie (cherry picked from commit 1aa338a216350a2751fff52f866039343e9ac013) Signed-off-by: Steve Sakoman Signed-off-by: Richard Purdie

bitbake: codeparser: Avoid log bufer overhead in cache case

2022-03-18T13:17:10+00:00

Creating the new log instances triggers a lot of python logging overhead in a commonly called function (about 600,000 for parsing OE-Core). We only need the log functionality if we're parsing, not if we just hit from the cache. Therefore defer the log setup overhead until we know it is a cache miss. Whilst this complicates the code slightly, the performance gain is worth it as for parsing OE-Core we drop 60 million funciton calls (from 225 overall). (Bitbake rev: ac868167ad854f9bb32dcb2e63528870547805a7) Signed-off-by: Richard Purdie

bitbake: codeparser: Fix TypeError in bitbake debug mode

2021-03-15T21:47:42+00:00

Commit 75f87db413 fixed the confusion between bitbake and python logger but in codeparser still old method of setting debug level was used causing TypeError, because debug level value was incorrectly returned and assigned to event.msg. | File "./bitbake/lib/bb/ui/knotty.py", line 660, in main | event.msg = event.fn + ': ' + event.msg | TypeError: can only concatenate str (not "int") to str [YOCTO #14298] (Bitbake rev: bec9ea7ab28a8dfad1a6010ca5c6ec691754748d) Signed-off-by: Tomasz Dziendzielski Signed-off-by: Richard Purdie

bitbake: lib: remove unused imports

2020-01-19T13:31:05+00:00

removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews Signed-off-by: Richard Purdie

bitbake: bitbake: Add initial pass of SPDX license headers to source code

2019-05-04T09:44:04+00:00

This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie

bitbake: codeparser: Switch to sha256 from md5

2019-01-08T11:16:03+00:00

We've reports of hash collision with codeparser. Looking at the way collision problems occur with md5 and the way our function templating works, I can believe we may run into issues. This patch therefore switches to sha256. Performance wise, parse time could appear to rise by 4s in 374s Before: 384329 in 2.966s (md5) After: 349743 in 2.340s (sha256) 34723 in 1.245s (md5) since we still have md5 used elsewhere in the code, something we should look at next (using sha256 everywhere is around 5.3s in total) Unfortunately this does nearly double the size of the codeparser cache file due to the hash size change. (Bitbake rev: 4bed7a97198176503fe8c72d8dd7c36b28fc9dd1) Signed-off-by: Richard Purdie

bitbake: bitbake: pysh: Improve error handling for shell code

2018-12-09T11:07:48+00:00

The p_error() is used for printing errors when parse shell code, but it can't the EOF error correctly - Add the following lines to quilt.inc do_configure_prepend () { find ${s} -name "*.in" -exec sed -i -e "1s,^#\!.*@perl@ -w$,#\! @perl@\nuse warnings;," {} \; if [ hello ]; then } - Before the patch: $ rm -fr cache/ tmp/cache/; bitbake -p [snip] WARNING: /path/to/quilt/quilt-native_0.65.bb: Error during finalise of /path/to/quilt/quilt-native_0.65.bb [snip] bb.pysh.sherrors.ShellSyntaxError: None followed by: We can see that this isn't easy to debug, let p_error() check wheter it is EOF and print appropriate errors can improve the error message. And don't let codeparser.py except pyshlex.NeedMore (in fact, it never worked since p_error() only raise ShellSyntaxError), but make it print the last 5 lines which might be useful for debuging. - After the patch $ rm -fr cache/ tmp/cache/; bitbake -p [snip] ERROR: /path/to/quilt/quilt_0.65.bb: Error during parse shell code, the last 5 lines are: find /path/to/quilt/0.65-r0/quilt-0.65 -name "*.in" -exec sed -i -e "1s,^#\!.*@PERL@ -w$,#\! @PERL@\nuse warnings;," {} \; if [ hello ]; then autotools_do_configure sed -e 's,^COMPAT_SYMLINKS.*:=.*,COMPAT_SYMLINKS :=,' -i /path/to/quilt/0.65-r0/quilt-0.65/Makefile [snip] File "/path/to/bb/pysh/pyshyacc.py", line 649, in p_error(p=None): w('Unexpected EOF') > raise sherrors.ShellSyntaxError(''.join(msg)) bb.pysh.sherrors.ShellSyntaxError: Unexpected EOF (Bitbake rev: 44790597951638e32eb1672de2e40bd5a603326b) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie