linux/poky.git/bitbake/lib/bb, branch yocto-3.0.2

bitbake: cooker/siggen: Empty siggen cache during parsing

2020-02-18T13:59:06+00:00

When parsing recipes its apparent the memory usage of bitbake rises linearly with number of recipes parsed. It shouldn't. Using tracemalloc (thanks for the tip Joshua Lock) it was clear that the dependency information left behind in siggen was the culprit. Add a new method to allow us to drop this information. We don't need it after the recipe has been parsed and hashes calculated (at runtime its different but only the currently executing task would be in memory). This should give signficant memory usage improvements for bitbake and that in turn should help speed on more constrained systems, as well as when used in multiconfig environments. (Bitbake rev: ef29309d0b512b64d024e383e7baff22c727711c) Signed-off-by: Richard Purdie

bitbake: siggen: Avoid cache mismatch issues with locked sigs

2020-02-11T23:02:14+00:00

If locked sigs are in use this function makes little sense, need to avoid generating mismatch warnings. (Bitbake rev: 95687be83e716220eb3893b67428f97fd59fc2c5) Signed-off-by: Richard Purdie (cherry picked from commit 27ad9c1d468fba858a4adeb56b605227b415ae0f) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: siggen: Cache unihash values to avoid cache lookup

2020-02-11T23:02:14+00:00

Add unihash cache of values to speed up cache lookup. This avoids the overhead of the disk based check functions. (Bitbake rev: 1f5a662176fd174c4793e3e28d1b4e95616b1715) Signed-off-by: Richard Purdie (cherry picked from commit 5c9cc45b60904a1c355db9bf9c4495f1b25aca37) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: siggen: Optimise get_unihash disk based cache handling

2020-02-11T23:02:14+00:00

Currently the cache can grow huge since any previously used hash is retained in the cache. This change moves to use one hash per task which improves the speed of the functions considerably. Currently performance is an issue, as are very large cache files and cache load time. By moving to a single hash per task, the shorted filename as a key is no longer usable as the same recipe has multiple variants for the same filename so this has to change. (Bitbake rev: 8f4f6c2f9acab23bc795ffe389c4cd74711d10ff) Signed-off-by: Richard Purdie (cherry picked from commit ed764e7fcf04b6d0ba6b4cac7415b1ee8f492865) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: cooker/toaster: replaced deprecated method warn() with warning()

2020-02-11T23:02:13+00:00

Removed the deprecated methods as it will only cause problems later on, and since warn() just calls warning(), it shouldn't change anything (Bitbake rev: c131015f1ac152f1fea4b83a3d451c3e4d05ebec) Signed-off-by: Frazer Clews Signed-off-by: Richard Purdie (cherry picked from commit a194f275235f22411cb2368f06a44f61ceb6a0f3) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: persist_data.py: Immediately get exclusive lock in setitem

2020-02-11T23:02:13+00:00

To avoid races, SQLTable::__setitem__ needs an exclusive lock for the entire transaction, not just the INSERT/UPDATE part. (Bitbake rev: 2ba2f224b50956313f5c2ba01942143b4d6098f2) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie (cherry picked from commit feb43e7c30f5bfab75d718896c45df621810d06f) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: fetch2/git: _revision_key: collapse adjacent slashes

2020-02-11T23:02:13+00:00

>From a SRCREV caching point of view, there is no reason to treat the following upstreams as different: SRC_URI = "git://github.com/file/file.git" SRC_URI = "git://github.com//file/file.git" (Bitbake rev: 716cdf737bc536f84ed1254d464c9f286e0d5a9a) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie (cherry picked from commit 425e21c14955dd38868c6e97637df3bbe0f89fac) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: fetch2: do not suffix srcrev cache key with PN

2020-02-11T23:02:13+00:00

Prior to this change, two different recipes pulling from the same exact repo could get a different SRCREV during a single parse session. This was originally observed using git. For git at least, it still allows recipes to pull from the same repo, but with different branches or tags, since the form of the srcrev cache key for git is: "git:" + ud.host + ud.path.replace('/', '.') + ud.unresolvedrev[name] Where the 'unresolvedrev' part is the branch or tag name. (Bitbake rev: 63556259f2e34003ad0618a8fe19e1cfe8aef5c9) Signed-off-by: Chris Laplante Signed-off-by: Richard Purdie (cherry picked from commit 6c938e6fd29beebe09b32be839dae008fe6491d2) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: runqueue: Use a set for the setscene tasks list

2020-01-28T11:51:03+00:00

This should give performance improvements to functions using this list of tasks (sets are used for most of the other code for this reason, not sure why this wasn't a set in the first place). (Bitbake rev: 1a026d4aca47ed1b0b74a8a818635e7520e8f9c8) Signed-off-by: Richard Purdie (cherry picked from commit f5daef68703481a3c243dfecc7de404e6ebfdbb6) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: knotty: Be consistent when creating/updating progress bars

2020-01-28T11:51:03+00:00

When creating a new progress bar (using BBProgress), a colon was appended to the supplied message. However, when updating the message, no colon was appended. Change this so that the colon is instead part of the widgets that make up the progress bar so that it does not matter when and how the message is updated, it always displays the same. (Bitbake rev: 1115a14c93dc44124c3ab95053142da78032a004) Signed-off-by: Peter Kjellerstedt Signed-off-by: Richard Purdie (cherry picked from commit 08f35c04f6e1ce4c4ca5c2bef4cd8a192e12e682) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

linux/poky.git/bitbake/lib/bb, branch yocto-3.0.2

bitbake: cooker/siggen: Empty siggen cache during parsing

bitbake: siggen: Avoid cache mismatch issues with locked sigs

bitbake: siggen: Cache unihash values to avoid cache lookup

bitbake: siggen: Optimise get_unihash disk based cache handling

bitbake: cooker/toaster: replaced deprecated method warn() with warning()

bitbake: persist_data.py: Immediately get exclusive lock in __setitem__

bitbake: fetch2/git: _revision_key: collapse adjacent slashes

bitbake: fetch2: do not suffix srcrev cache key with PN

bitbake: runqueue: Use a set for the setscene tasks list

bitbake: knotty: Be consistent when creating/updating progress bars

bitbake: persist_data.py: Immediately get exclusive lock in setitem