linux/poky.git/bitbake/lib/bb/cookerdata.py, branch mickledore-4.2.3

bitbake: cookerdata: Drop dubious exception handling code

2023-02-19T07:48:28+00:00

This code appears to be dangerous, it swallows exceptions, turning them into "handled" versions which then show no errors to the user. This is a pretty poor user experience and I can't see why this code should be swallowing such things. Drop the worst bits of code. (Bitbake rev: 13279044f16f2cf2502ebf39d277415f99bb6c18) Signed-off-by: Richard Purdie

bitbake: cookerdata: Improve early exception handling

2023-02-19T07:48:28+00:00

Martin Jansa reported that if you put a syntax error into an imported module such as qa.py in OE, no error is shown. Part of the issue appears to be that the catch_parse_error() decorator only catches certain exceptions and SyntaxError isn't one of them. As far as I can tell we should remove all the special cases and use the more advanced code in all cases, not just expansion errors. I confirmed this now prints a proper error message for a qa.py syntax error. (Bitbake rev: 2365d891847f8e73d1c4661ddfdab8818ff619dc) Signed-off-by: Richard Purdie

bitbake: cookerdata: Remove incorrect SystemExit usage

2023-02-17T15:10:02+00:00

Calling SystemExit doesn't work well with server/client usage since the string isn't printed to the right place. Use bb.fatal() instead which prints the right log output and raises and handled exception which then shows correctly on the UI. (Bitbake rev: aaefb43b41a0d9b16a59643136268eb6e5d48cd2) Signed-off-by: Richard Purdie

bitbake: cache/codeparser: Switch to a new BB_CACHEDIR variable for cache location

2023-01-26T21:50:31+00:00

Currently the codeparser cache is set from CACHE, which is typically in bitbake.conf which means we can't read/write any cache until it is found/read. We may well have python expressions to parse before that happens. The net result is suboptimal functioning of the codeparser cache since it will often be invalidated by data that is never written. This patch changes the codeparser and filechecksum caches to use BB_CACHE as their setting and defaults it to ${TOPDIR}/cache. The patch doesn't change where the "persistent" data such as prserver and hash-equiavalance resides (PERSISTENT_DIR) or where the metadata parsing cache resists (still currently CACHE). I've left those for a later patch. The patch does ensure data parsed by the core datastore parsing calls is written back since this is now much more useful after this change. (Bitbake rev: ee89ade5b5a4cf9c53f336d8b800e06fbe436628) Signed-off-by: Richard Purdie

bitbake: cookerdata: Fix previous commit to use a string, not a generator

2023-01-02T00:29:09+00:00

The previous commit was injecting a generator object instead of a string, which happened to fix the issue but we should fix it properly! Thanks to Christopher Larson for spotting. (Bitbake rev: 36589f13c5babb3b2af82666c8a96338e6ea4eb9) Signed-off-by: Richard Purdie

bitbake: cookerdata: Fix cache/reparsing issue

2023-01-01T14:25:20+00:00

When setting the LAYERSERIES_COMPAT and LAYERSERIES_CORENAMES variables, we need to be deterministic. The random ordering from the sets was causing unexpected reparses. (Bitbake rev: 8c405c97430ac830837e25438e8795f6f7abbdaa) Signed-off-by: Richard Purdie

bitbake: cooker/cookerdata: Rework the way the datastores are reset

2022-12-31T17:05:17+00:00

As far as I could tell, the current code could result in some strange situations where some data was set back to the original data store copy but the multiconfig data was not restored. There are also some changes made to the datastore which did not persist. The data store was also being reset at every client reset, which seems a little excessive if we can reset it to the original condition properly. Move the __depends -> __base_depends rename into databuilder along with the __bbclasstype change so these are saved in the original data. Tweak the databuilder code to be clearer and easier to follow on which copies are where, then save copies of all the mc datastores. Finally, drop the cache invalidation upon reset for the base config as we shouldn't need that now (oe-selftest -r tinfoil works with memory resident bitbake which was the original reproducer requiring that change). (Bitbake rev: 5f8b9b9c35b4ec0c8c539bf54ca85f068f4a5094) Signed-off-by: Richard Purdie

bitbake: codeparser/data: Add vardepsexclude support to module dependency code

2022-12-17T08:52:28+00:00

We need to be able to exclude dependencies from the python module dependency code. Add support for the vardepexclude flag for these. It only works from the configuration namespace rather than per recipe for efficiency. (Bitbake rev: 1aa672b01037fda4ca82f2c7e394783287c09ecd) Signed-off-by: Richard Purdie

bitbake: cookerdata: Ensure layers use LAYERSERIES_COMPAT fairly

2022-12-08T11:15:47+00:00

Some layers think they're going to be 'clever' and copy the values from another layer, e.g. using ${LAYERSERIES_COMPAT_core}. The whole point of this mechanism is to make it clear which releases a layer supports and show when a layer master branch is bitrotting and is unmaintained. Therefore add some code to avoid people doing this. I wish we didn't have to but... (Bitbake rev: 6709aedccbb2e7ddbb1b2e7e4893481a7b536436) Signed-off-by: Richard Purdie

bitbake: parse: Add support for addpylib conf file directive and BB_GLOBAL_PYMODULES

2022-12-08T11:15:47+00:00

For many years OE-Core has injected it's own python modules into the python namespace using an immediate expansion of a variable in base.bbclass. It also added all entries from BBPATH to sys.path. We really need this to become a first class citizen of the langauge, this new addpylib directive allows that. Usage is of the form: addpylib The namespace is imported and if there is an attribute BBIMPORT, that list of names is iterated and imported too. This mirrors what OE-Core has done for a long time with one difference in implmentation, sys.path is only appended to. This means later imported namespaces can't overwrite an earlier one and can't overwrite the main python module space. In practice we've never done that and it isn't something we should encourage or support. The new directive is only applied for .conf files and not other filetypes as it only makes sense in that context. It is also only allowed in the "base configuration" context of cookerdata since adding it at the recipe level wouldn't work properly due to the way it changes the global namespace. At the same time, move the list of modules to place in the global namespace into a BB_GLOBAL_PYMODULES variable. It is intended that only the core layer should touch this and it is meant to be a very small list, usually os and sys. BB_GLOBAL_PYMODULES is expected to be set before the first addpylib directive. Layers adding a lib directory will now need to use this directive as BBPATH is not going to be added automatically by OE-Core in future. The directives are immediate operations so it does make modules available sooner than the current OE-Core approach. The new code appends to sys.path rather than prepends as core did, as overwriting python standard library modules would be a bad idea and naturally encouraging people to collaborate around our own core modules is desireable. (Bitbake rev: afb8478d3853f6edf3669b93588314627d617d6b) Signed-off-by: Richard Purdie