linux/poky.git/bitbake/lib/bb/cache.py, branch 4.2_M3

bitbake: cache/codeparser: Switch to a new BB_CACHEDIR variable for cache location

2023-01-26T21:50:31+00:00

Currently the codeparser cache is set from CACHE, which is typically in bitbake.conf which means we can't read/write any cache until it is found/read. We may well have python expressions to parse before that happens. The net result is suboptimal functioning of the codeparser cache since it will often be invalidated by data that is never written. This patch changes the codeparser and filechecksum caches to use BB_CACHE as their setting and defaults it to ${TOPDIR}/cache. The patch doesn't change where the "persistent" data such as prserver and hash-equiavalance resides (PERSISTENT_DIR) or where the metadata parsing cache resists (still currently CACHE). I've left those for a later patch. The patch does ensure data parsed by the core datastore parsing calls is written back since this is now much more useful after this change. (Bitbake rev: ee89ade5b5a4cf9c53f336d8b800e06fbe436628) Signed-off-by: Richard Purdie

bitbake: cache: Only write files if we have data

2023-01-24T21:59:44+00:00

By writing the cache files only if there is data to write, we can save a bit of time. (Bitbake rev: abeff1f80bb1c690b92d535d472dff9df7a56067) Signed-off-by: Richard Purdie

bitbake: cache: Drop reciever side counting for SiggenRecipeInfo

2022-12-30T15:07:19+00:00

Joshua Watt pointed out maintaining the counting on both sides of the connection isn't needed. Remove the receiver side counting and simplify the code, also allowing errors if the counts do go out of sync. (Bitbake rev: aeacfd391903fe68ae600470fc2bbad0502d401e) Signed-off-by: Richard Purdie

bitbake: cache/siggen: Fix cache issues with signature handling

2022-12-29T00:05:45+00:00

There is a bug in the current cache code where the restored data structures were "inverted" leading to some very weird issues, not sure how anything worked like that, this patch fixes it. Also fix some issues with multiconfig cache ordering problems by resetting the stream counters when appropriate. (Bitbake rev: cd06beb948eff5eaf2d474f5b127d51a61b0b2ef) Signed-off-by: Richard Purdie

bitbake: cache: Allow compression of the data in SiggenRecipeInfo

2022-12-21T14:15:26+00:00

The data in SiggenRecipeInfo is large and has a lot of duplication. The size causes a few problems, impacting: - bitbake's overall memory usage - the amount of data sent over IPC between parsing processes and the server - the size of the cache files on disk - the size of "sigdata" hash information files on disk The data consists of strings (some large) or frozenset lists of variables. To reduce the impact we can: a) deplicate the data b) pass references to the object on the second usage (e.g. over IPC or saving into pickle). This patch does this for SiggenRecipeInfo mostly behind the scenes but we do need a couple of reset points so that streamed data is written correctly on the second usage. (Bitbake rev: 9a2b13af483c20763d6559a823310954884f6ab1) Signed-off-by: Richard Purdie

bitbake: siggen/cache: Optionally allow adding siggen hash data to the bitbake cache

2022-12-08T10:49:53+00:00

Being able to track siggen hash construction data can be useful for cache debugging. For now, add an extra cache class which contains this information. It can be enabled in the same way as the hob data cache through a feature flag to cooker. This allows us to experiment with the data without carrying larger patches around and ultimately may allow use to have a hash mismatch debugging mode that is more easily enabled. (Bitbake rev: 0736a8a03da8b774fafbd28f746bef4705378049) Signed-off-by: Richard Purdie

bitbake: cache/siggen: Simplify passing basehash data into the cache

2022-12-08T10:49:53+00:00

The basehash data is really internal bitbake data and passing an object directly is more efficient than creating and then extracting variables. This will match the format of other data we may optionally wish to store in the cache so more to the more efficient method. Nothing I can see is using this data today (and nothing should be). (Bitbake rev: e621093a1bf37cd75ede3fb77ab6845556870fc7) Signed-off-by: Richard Purdie

bitbake: cache: Drop support for not saving the cache file

2022-11-20T08:31:28+00:00

We don't test not using the cache and I'm not aware of anyone using this, it would be hard to with modern bitbake. Drop the conditional code and simply error if CACHE isn't set. (Bitbake rev: 063ffe699bc5fc23174643dfedb66864cacfcff8) Signed-off-by: Richard Purdie

bitbake: cache: Drop unused function

2022-11-20T08:31:28+00:00

At this point users appear to all call add_info directly. Failing to do that means the file dependency tracking code isn't active so would cause problems. Therefore drop the unused function. (Bitbake rev: 6b24efc0f4d19738d96754280e70bc493005167d) Signed-off-by: Richard Purdie

bitbake: cache: Drop broken/unused code

2022-11-20T08:31:28+00:00

Some parts of functions in Cache() were broken and unused, there was also a totally unused function. This was historical as a result of the cooker parsing process needing to handle cached entries in the main thread but parsing actions in seperate processes. Document the way it works, update the function name to be clear about what it now does and drop the old code which was unused. (Bitbake rev: af83ee32df85c8e4144f022a1f58493eb72cb18e) Signed-off-by: Richard Purdie