linux/poky.git/bitbake/lib/bb/data_smart.py, branch yocto-5.0.12

bitbake: data_smart.py: clear expand_cache in _setvar_update_overridevars

2025-03-08T14:22:56+00:00

At the end of this function, self.overrides is cleared, which means we'll construct a new self.overrides after this call. And a new self.overrides will give out different expand_cache, so the old expand_cache should also be cleared to avoid any wrong value. Currently, there's a problem revealed by recent recipe specific virtual provider patch. If we enable multilib and set "OVERRIDES:prepend" in local.conf, things don't work. Here's the error message: ERROR: Nothing PROVIDES 'lib32-gcc-cross-x86_64' Below are reproduce steps: 1. Add in local.conf the following lines: MACHINE ?= "qemux86-64" require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE:virtclass-multilib-lib32 ?= "core2-32" OVERRIDES:prepend = "some-override:" (Note that using :append and :remove also reproduces the issue.) 2. bitbake -n lib32-sysstat (bitbake -n core-image-minimal also reproduces the issue) The expandWithRefs calls getVar, which fills expand_cache. So when setting OVERRIDES:prepend, this will fill the expand_cache. When overridevars are updated, if we don't clear expand_cache, we'll retrieve wrong values. Previously, things happened to work because there's a call to expand PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc, which in turn expands TARGET_VENDOR. Now what we expand is PREFERRED_PROVIDER_ virtual/cross-cc, so the problem is revealed. (Bitbake rev: 7375d32e8c1af20c51abec4eb3b072b4ca58b239) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: data_smart.py: simple clean up

2025-03-08T14:22:56+00:00

When var matches __setvar_regexp__, which has the value as below, __setvar_regexp__ = re.compile(r'(?P.*?)(?P:append|:prepend|:remove)(:(?P[^A-Z]*))?$') the keyword will not be __doc__ or __module__ and var will always contain ":". (Bitbake rev: 9cac695951d0fd587063b250bfc8ef0f87acad18) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: data_smart.py: remove unnecessary ? from __expand_var_regexp__

2025-03-08T14:22:56+00:00

The non-greedy modifier suffix ? is not necessary here because } is not in the character set [a-zA-Z0-9\-_+./~:]. (Bitbake rev: aae570b7f051fc5deee7a1712a02ed92498a4461) Signed-off-by: Chen Qi Signed-off-by: Richard Purdie Signed-off-by: Steve Sakoman

bitbake: data_smart: Improve performance for VariableHistory

2024-08-26T12:18:43+00:00

Fixed: - BBMULTICONFIG = "qemux86-64 qemuarm64" and more than 70 layers in BBLAYERS $ bitbake -p -P Check profile.log.processed, the record() cost more than 20 seconds, it is less than 1 second when multiconfig is not enabled, and there would be the following error when more muticonfigs are enabled: Timeout while waiting for a reply from the bitbake server Don't change the type of loginfo['detail'] or re-assign it can make record() back to less than 1 second, this won't affect COW since loginfo is a mutable type. The time mainly affected by two factors: 1) The number of enabled layers, nearly 1 second added per layer when the number is larger than 50. 2) The global var such as USER_CLASSES, about 1 ~ 2 seconds added per layer when the layers number is larger than 50. (Bitbake rev: d251668d9a7a8dd25bd8767efb30d6d9ff8b1ad3) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie (cherry picked from commit 0596aa0d5b0e4ed3db11b5bd560f1d3439963a41) Signed-off-by: Steve Sakoman

bitbake: data_smart: directly check for methodpool functions in context lookup

2023-08-04T10:37:56+00:00

We previously checked for the existence of the 'func' flag to determine if we should avoid looking up in the metadata. This was done to ensure the user gets the function for 'def' python functions rather than their string contents. We can sidestep the metadata lookup and check our function context directly, instead. (Bitbake rev: 6cac1eac51efa9a54e8457f60ea1ea0e604c50b7) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: data_smart: check for python builtins directly for context lookup

2023-08-04T10:37:55+00:00

This avoids the need to hardcode a list of python builtins. This also slightly changes behavior, in a case like `${@eval("3")}`, this will ensure we always call the builtin, even if the metadata has an 'eval' variable defined. (Bitbake rev: 9976ae50677b333d646ca0fd395468bd2301d03f) Signed-off-by: Christopher Larson Signed-off-by: Richard Purdie

bitbake: data_smart: Small optimisation to _findVar()

2023-01-24T21:59:44+00:00

Some users of _findVar don't need the override data and even getVarFlag doesn't need it in some common cases (parsing=True). Rearrange the code as the current overridedata approach doesn't need to be in the _findVar code anyway. This removes some search overhead from a critical path. (Bitbake rev: fcb64e1138a20eb19560af3fc5d1fa748cc9cf34) Signed-off-by: Richard Purdie

bitbake: data_smart: Small cache reuse optimization

2022-11-29T10:25:51+00:00

Currently the expand cache doesn't work for "parser" return types, which is the main type used by the build_dependencies() call that we spend most of the time in when parsing. Tweak the code to cache the unexpanded value in the expand cache and hence allow reuse of the parser in other fast path cases for small speed gains. (Bitbake rev: b4a8e5071dbcba2217b79e83e08b275ffcbc0eef) Signed-off-by: Richard Purdie

bitbake: data_smart: Use regex consistently for override matching

2022-11-29T10:25:51+00:00

One section of the code is enforcing lowercase overrides, the other is allowing numeric characters. We should be consistent with one or the other. (Bitbake rev: df5b3b841fd8d6a652d643e9ae2bba09d60043e0) Signed-off-by: 张忠山 Signed-off-by: Richard Purdie

bitbake: data_smart: Add debugging for overrides stability issue

2022-11-27T23:48:15+00:00

If someone is unfortunate enough to run into override recursion issues they're hard to debug with the existing message. We can at least show the values that OVERRIDES takes to show there is some problem and aid debugging. (Bitbake rev: 43035b75201616e7bfd680d3d15c5c0fc7c04eb6) Signed-off-by: Richard Purdie