summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/siggen.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: siggen: Fix sorting in diff outputRichard Purdie2021-09-261-7/+7
| | | | | | | | | The diff output isn't deterministic at the moment as the sets can have differing ordering. Sort the output so it is consistent. (Bitbake rev: 117830c1d7ef3e53052fa326e1ca62c5c3946c45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Switch to using new override syntaxRichard Purdie2021-08-021-2/+2
| | | | | | | | | | | | | | | | | This change updates the datastore to use the new override syntax using colons instead of underscores exclusively. It is expected layers would have to be converted to work with bitbake after this change. Supporting mixed syntax isn't possible, it is only feasible to have one internal representation of overrides. Whilst we can't warn for every possible override that may be set in the old format, show errors for _append/_prepend/_remove since those should never be present. (Bitbake rev: 7dcf317cc141dc980634f8c18bfa84f83e57206a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add bb.utils.rename() helper function and use for renamingDevendra Tewari2021-05-061-1/+1
| | | | | | | | | | | | | | | | | | | | os.rename can fail for example an incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than trying to fix every call site, add a wrapper in bb.utils for renames. We can then handle cross device failures and fall back to shutil.move. The reason os.rename is still used is because shutil.move is too slow for speed sensitive sections of code. [YOCTO #14301] (Bitbake rev: c5c4e49574ab2a65e06298a0a77bb98b041cf56b) Signed-off-by: Devendra Tewari <devendra.tewari@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Use generic ConnectionErrorPaul Barker2021-04-271-3/+3
| | | | | | | | | | The Python built-in ConnectionError type can be used instead of a custom HashConnectionError type. This will make code refactoring simpler. (Bitbake rev: 8a796c3d6d99cfa8ef7aff0ae55bb0f23bbbeae1) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: logging: Make bitbake logger compatible with python loggerJoshua Watt2021-02-101-3/+3
| | | | | | | | | | | | | | | | | The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Don't treat mc recipe (Midnight Commander) as a multiconfig ↵Tomasz Dziendzielski2021-02-061-1/+1
| | | | | | | | | | | | | | | | target When we run `devtool build mc` recipe's task dependencies are expanded to "mc:do_populate_sysroot" where "mc" name is treated as multiconfig and "do_package_sysroot" as multiconfigname. | ERROR: Multiconfig dependency mc:do_populate_sysroot depends on | nonexistent multiconfig configuration named do_populate_sysroot (Bitbake rev: 3ce4b2caccfe608a54dff159459f3687ea610597) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Remove broken optimisationRichard Purdie2020-11-031-7/+1
| | | | | | | | | | | When a single signature is locked, dependent task checksum calculations fail. This in turn is because get_unihash cannot be cached correctly by this function. Remove that has turned out to be a poor optimisation to avoid that bug. (Bitbake rev: 7ae2b73d131491ab5bf09fd6055e1fdb67a560c6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: use correct umask when writing siginfoRoss Burton2020-09-301-1/+2
| | | | | | | | | | | | | | | We try to write sstate with group-write permissions so that sstate-cache can be shared between multiple users. However the siginfo files are created in various tasks which may set their own umask (such as do_populate_sysroot, 0022). This results in no group write permission on the intermediate directories, which is fatal when sharing a cache. Fix this by wrapping the siginfo mkdir in a umask change to 0002. (Bitbake rev: 75d9ef04a908e366633b255d23ab3275f6860d3a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: siggen: clean_basepath: improve perfo and readabilityJean-Francois Dagenais2020-09-231-10/+20
| | | | | | | | | | | | | | | This change improves performance by reducing runtime about 33% for typical inputs. (using test_clean_basepath_performance) It is also easier to read, and slightly more resilient to future changes since it doesn't mention 'virtual' anymore. (Bitbake rev: 27b53186fa67d281d29b2f8e15bcff8dc2557b8a) Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com> Co-Developed-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com> Signed-off-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz ↵Jean-Francois Dagenais2020-08-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | present Before this fix, this example basepath (a): virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb:do_compile would get incorrectly "cleaned" into: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb When searching backwards in `a` trying to isolate the 'virtual:xyz' to add it to the end of the string, we need to consider `a` still has the recipe path and taskname. So stoping the rsplit after only 1 split is not enough. We want to reach the second ':' from the end. This way, we obtain: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native reviewed-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com> (Bitbake rev: d193d93422a0ad62aa35b5d4ca5da8d422f72180) Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix error when hash equivalence has an exceptionJoshua Watt2020-06-281-0/+1
| | | | | | | | | | The code that handled exceptions from the hash equivalence client was raising an exception itself because hashserv.client wasn't imported (Bitbake rev: a76290dfc6f34ff9f6efdb13a6db74b6b4759daf) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: siggen: Pass all data caches to hash functionsJoshua Watt2020-06-121-16/+62
| | | | | | | | | | | | | | | | | | | Passing all the data caches to the task hashing functions allows them to correctly account for mcdepends in task signatures. This allows tasks to be correctly re-run when a mcdepends changes. By default, the legacy behavior is maintained for derived signature generators by passing a special proxy object that can either be used to access all multiconfigs or the legacy behavior. If a derived signature generator is updated, it can set the supports_multiconfig_datacaces property to instruct bitbake it deals with multiconfigs properly. [YOCTO #13724] (Bitbake rev: 8ff9203de4fce9c104c2987d86980c9f34036b97) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/siggen: Don't expand ignored variablesRichard Purdie2020-04-061-1/+1
| | | | | | | | | | | | | | | | | | If a variable is in the signature whitelist, we'd currently expand it, then later ignore the data. This is problemtic for code which has effects when expanded, recently source date epoch in OE-Core for example. We don't actually need to do this, if we pass the whitelist into the earlier function it can avoid the expansion. This also also give a small performance boost since we avoid running code in some cases. [YOCTO #13581] (Bitbake rev: f483ee4a869fb1dafbe4bdf2da228cdaa40b38bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/siggen: Lower hash equivalence loggingJoshua Watt2020-03-131-4/+4
| | | | | | | | | | | | | | Lowers the level at which hash equivalence messages are logged so as to not annoy the majority of users. The autobuilder can use a custom logging configuration to log these to a file for debugging (see contrib/autobuilderlog.json) [YOCTO #13813] (Bitbake rev: 2ddb649ea31afe052f08e3969e36abf6fb515bc2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/siggen: Log hash equivalence with a different loggerJoshua Watt2020-03-131-8/+9
| | | | | | | | | | Switches the hash equivalence logging to use a different logger so that it can be easily filtered out with python's structured logging. (Bitbake rev: 20bb29ef973e9c5483eb50a74550ea207637367b) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/siggen: Empty siggen cache during parsingRichard Purdie2020-02-171-0/+11
| | | | | | | | | | | | | | | | | | | When parsing recipes its apparent the memory usage of bitbake rises linearly with number of recipes parsed. It shouldn't. Using tracemalloc (thanks for the tip Joshua Lock) it was clear that the dependency information left behind in siggen was the culprit. Add a new method to allow us to drop this information. We don't need it after the recipe has been parsed and hashes calculated (at runtime its different but only the currently executing task would be in memory). This should give signficant memory usage improvements for bitbake and that in turn should help speed on more constrained systems, as well as when used in multiconfig environments. (Bitbake rev: 5d98d8e39bba42f458532b1eef3619f2321d8a2b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Avoid cache mismatch issues with locked sigsRichard Purdie2020-02-081-0/+4
| | | | | | | | | If locked sigs are in use this function makes little sense, need to avoid generating mismatch warnings. (Bitbake rev: 27ad9c1d468fba858a4adeb56b605227b415ae0f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Cache unihash values to avoid cache lookupRichard Purdie2020-02-081-5/+17
| | | | | | | | | | Add unihash cache of values to speed up cache lookup. This avoids the overhead of the disk based check functions. (Bitbake rev: 5c9cc45b60904a1c355db9bf9c4495f1b25aca37) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Optimise get_unihash disk based cache handlingRichard Purdie2020-02-081-18/+32
| | | | | | | | | | | | | | | | Currently the cache can grow huge since any previously used hash is retained in the cache. This change moves to use one hash per task which improves the speed of the functions considerably. Currently performance is an issue, as are very large cache files and cache load time. By moving to a single hash per task, the shorted filename as a key is no longer usable as the same recipe has multiple variants for the same filename so this has to change. (Bitbake rev: ed764e7fcf04b6d0ba6b4cac7415b1ee8f492865) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Test extra cross/native hashserv methodRichard Purdie2020-01-021-6/+21
| | | | | | | | | | | | | | | | Hack the hashserv to allow extra data to be injected into the hashserv method. This allows OE-Core to handle cases where there are multiple sstate objects for the same taskhash, e.g. native/cross objects based upon BUILD_ARCH or the host distro (when uninative isn't used). This has been tested and proven to be very effective. We will likely rework the code to improve how this is handled but for now this improves automated builds until we can get to that refactoring and more invasive changes. (Bitbake rev: 0a09b0fa03d1afc08037964dc63a18ef7cff9c78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add BB_SIGNATURE_LOCAL_DIRS_EXCLUDE to speed-up taskhash on ↵Aníbal Limón2019-12-301-2/+3
| | | | | | | | | | | | | | | directories The new BB_SIGNATURE_LOCAL_DIRS_EXCLUDE allows you to specify a list of directories to exclude when making taskhash, our specific case is using SRC_URI that points local VCS directory. Use bb.fetch.module to set default to: "CVS .bzr .git .hg .osc .p4 .repo .svn" (Bitbake rev: 923aff060d8aba8456979c35b16d300ba7c13ff9) Signed-off-by: Aníbal Limón <anibal.limon@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Split get_tashhash for performanceRichard Purdie2019-12-161-9/+24
| | | | | | | | | | | | | | | There are two operations happening in get_taskhash, the building of the underlying data and the calculation of the hash. Split these into two funtions since the preparation part doesn't need to rerun when unihash changes, only the calculation does. This split allows sigificant performance improvements for hashequiv in builds where many hashes are equivalent and many hashes are changing. (Bitbake rev: 6a32af2808d748819f4af55c443578c8a63062b3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix performance issue in get_unihashRichard Purdie2019-12-151-3/+3
| | | | | | | | | | There is a significant performance issue in get_unihash(). The issue turns out to be the lookups of setscene tasks. We can fix this by using a set() instead of the current list. (Bitbake rev: 1e561672d039ebfb8cd0e0654a44dcf48513317c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "siggen: Fix hashequiv performance issues"Richard Purdie2019-12-121-13/+14
| | | | | | | | This reverts commit c4b8440f730c33eaf9f818b856ae81b2f1017fec. The logic in this change is flawed and needs to be re-thought. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix hashequiv performance issuesRichard Purdie2019-12-121-14/+13
| | | | | | | | | | | | | | | | | | | We're seeing huge slowdowns on large builds on the autobuilder. A qemux86 world build was running really slowly, a primary feature was lots of rehashing going on due to an sstate change which caused a rebuild when all output should be identical. This was traced to the hundreds of thousands of calls to get_unihash() from get_taskash(). If we simplify the unitaskhashes data structure, we can bypass the function call and access the data directly. In local profile charts, this significanly sped up process_possible_migrations(), both on profiles and visually. Whilst this change doesn't aid readability, it does solve an otherwise huge performance issue. (Bitbake rev: c4b8440f730c33eaf9f818b856ae81b2f1017fec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure new unihash propagates through the systemRichard Purdie2019-12-091-0/+1
| | | | | | | | | | Its possible the new unihash may not exist in sstate. Currently the code would create an sstate object with the old hash however this updates it to create the object with the new unihash. (Bitbake rev: abcaa1398031fa5338a43859c661e6d4a9ce863d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Avoid taskhash mismatch errors for nostamp tasks when ↵Richard Purdie2019-12-061-4/+8
| | | | | | | | | | | | | | | | | | | dependencies rehash An example: NOTE: recipe binutils-cross-testsuite-2.32.0-r0: task do_check: Started ERROR: Taskhash mismatch b074da4334aff8aa06572e7a8725c941fa6b08de4ce714a65a90c0c0b680abea versus 17375278daed609a7129769b74a1336a37bdef14b534ae85189ccc033a9f2db4 for /home/pokybuild/yocto-worker/qemux86-64/build/meta/recipes-devtools/binutils/binutils-cross-testsuite_2.32.bb:do_check NOTE: recipe binutils-cross-testsuite-2.32.0-r0: task do_check: Succeeded Is caused by a rehash in a dependency happening somewhere earlier in the build and the taint being reset. Change the code so that nostamp taints are preserved to avoid the issue. (Bitbake rev: 61624a3fc38e8546e01356d5ce7a09f21e7094ab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/siggen: Allow handling of equivalent hashesRichard Purdie2019-12-041-0/+26
| | | | | | | | | | | | | Based on the hashserv's new ability to accept hash mappings, update runqueue to use this through a helper function in siggen. This addresses problems with meta-extsdk-toolchain and its dependency on gdb-cross which caused errors when building eSDK. See the previous commit for more details. (Bitbake rev: 39098b4ba2133f4d9229a0aa4fcf4c3e1291286a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix hashequiv bug where new hash wasn't referenced correctlyRichard Purdie2019-11-251-0/+1
| | | | | | | | | | | If a hash is reported to the hash server, the stamp written out by the current task didn't account for any new hash the server may have provided. Fix this so the correct stamp is written. This means "bitbake X; bitbake X" no longer rebuilds lots of things when hashequiv is active. (Bitbake rev: 9dc5178f56ab8ae37e1a646c09117c503e48d072) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Remove full path from unitaskhashes keysRichard Purdie2019-09-301-2/+2
| | | | | | | | | | The full paths make the cache useless in the sdk. They also bloat the cache size. They're for human debugging benefit only so compromise and reduce this to the filename. (Bitbake rev: 3b275c4083eae1d3781f0862919af9de83932b0f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Avoid writing misleading sigdata filesRichard Purdie2019-09-301-1/+2
| | | | | | | | | | Use the unihash in the output filename of sigdata files else the contents of stamp directories is misleading. Write the unihash into the singature to make it clear what happened. (Bitbake rev: feb01ee54d3706fe93768f332054c7532f7209e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen/runqueue: Fix signature mismatch issuesRichard Purdie2019-09-301-1/+5
| | | | | | | | | | | | We need to set the setscene tasklist before we call into the taskhash/unihash code else the behaviour is inconsistent. Avoid reporting hashes for non setscene tasks since we'd never query that. (Bitbake rev: 419a7840b8627278db694029c25df00214d01d96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure setscenetasks list is available to worker contextRichard Purdie2019-09-301-2/+3
| | | | | | | | | | | The setscenetasks list needs to be available in the worker contexts else the signature behaviour there mismatches what the server does. Add the data to get/set_taskdata to ensure this happens. (Bitbake rev: 632980ef90fe126b7ba3d138f4d574ae05914779) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix attribute error when hashserver failsJoshua Watt2019-09-271-2/+2
| | | | | | | | | | | | The HashConnectionError class was moved to the client module and needs to be updated. [YOCTO #13537] (Bitbake rev: 9fb862685e5e5a2aa534bc25cab1e4158d708b40) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Rework hash equivalenceJoshua Watt2019-09-181-48/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reworks the hash equivalence server to address performance issues that were encountered with the REST mechanism used previously, particularly during the heavy request load encountered during signature generation. Notable changes are: 1) The server protocol is no longer HTTP based. Instead, it uses a simpler JSON over a streaming protocol link. This protocol has much lower overhead than HTTP since it eliminates the HTTP headers. 2) The hash equivalence server can either bind to a TCP port, or a Unix domain socket. Unix domain sockets are more efficient for local communication, and so are preferred if the user enables hash equivalence only for the local build. The arguments to the 'bitbake-hashserve' command have been updated accordingly. 3) The value to which BB_HASHSERVE should be set to enable a local hash equivalence server is changed to "auto" instead of "localhost:0". The latter didn't make sense when the local server was using a Unix domain socket. 4) Clients are expected to keep a persistent connection to the server instead of creating a new connection each time a request is made for optimal performance. 5) Most of the client logic has been moved to the hashserve module in bitbake. This makes it easier to share the client code. 6) A new bitbake command has been added called 'bitbake-hashclient'. This command can be used to query a hash equivalence server, including fetching the statistics and running a performance stress test. 7) The table indexes in the SQLite database have been updated to optimize hash lookups. This change is backward compatible, as the database will delete the old indexes first if they exist. 8) The server has been reworked to use python async to maximize performance with persistently connected clients. This requires Python 3.5 or later. (Bitbake rev: 2124eec3a5830afe8e07ffb6f2a0df6a417ac973) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/siggen: Optimise hash equiv queriesRichard Purdie2019-09-161-0/+10
| | | | | | | | | We only have hash equivalence for setscene tasks so only query the server for those, reducing the number of connections needed. (Bitbake rev: 22082c7b3ca0cffcedb7d1d8c6681d35286376db) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/runqueue: Add hashserv+runqueue testRichard Purdie2019-08-061-4/+19
| | | | | | | | Add a test which tests the runqueue adaptations for hash equivalency. (Bitbake rev: 477321d0780df177c1582db119c2bb6795912fc6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Clean up task reference formatsRichard Purdie2019-08-061-81/+71
| | | | | | | | | | | | | | | | | | Currently siggen uses the format "<filename>.<taskname>" for referencing tasks whilst runqueue uses "<filename>:<taskname>". This converts to use ":" as the separator everywhere. This is an API breaking change since the cache is affected, as are siginfo files and any custom signature handlers such as those in OE-Core. Ultimately this will let us clean up and the accessor functions from runqueue, removing all the ".rsplit(".", 1)[0]" type code currently all over the place. Once a standard is used everwhere we can update the code over time to be more optimal. (Bitbake rev: 07e539e1c566ca3434901e1a00335cb76c69d496) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Convert to use self.unitaskhashesRichard Purdie2019-08-061-4/+8
| | | | | | | | | Rather than metadata driven sqlite databases for communication, use bitbake's unitaskhashes variable instead. (Bitbake rev: a0d941c787cf3ef030d190903279d311bc05d752) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add new unitaskhashes data variable which is cachedRichard Purdie2019-08-061-2/+18
| | | | | | | | | | | We need to preserve unihash task hashes between runs. Use the new SimpleCache class to create such a class within the signature generator which is loaded at init time and saved when builds complete. The default is unpopulated but metadata sig handlers can populate this cache. (Bitbake rev: 1f326f2c29c2664a5daaeeb0c1fd332630efbdba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Import unihash code from OE-CoreRichard Purdie2019-08-061-0/+163
| | | | | | | | | | | | This code is closely tied with the hash server in bitbake and also means we can't relibably test the hashserv runqueue functionality without OE metadata. Moving this to bitbake as a MixIn class makes most sense and encourages code collaboration and reuse as well as enabling easier and more accurate testing of the APIs. (Bitbake rev: 7bb79099a6c1b463d6ae9226c4cab5e76a965675) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix handling of tainted sig filesRichard Purdie2019-07-191-2/+2
| | | | | | | | | | The addition of some debugging code meant that comparisions between sig files with a taint and without a taint weren't working. Tweak the logic to avoid tracebacks if one side doesn't have a taint. (Bitbake rev: f5ea06fc2b6713c9f8e85ecf7cb981ae9a84d896) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Use unique hashes for tasksRichard Purdie2019-07-151-2/+3
| | | | | | | | | | | | Now that runqueue optimises based on task hash, we need to ensure tasks have unique hashes even in the simplest siggen mode. Use the task name to calculate a unique hash. This fixes runqueue tests when hash optimisations are added. (Bitbake rev: 8ede873ef4ef492fbaf01474685c1ca8b34d80d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix default handlerRichard Purdie2019-07-151-1/+2
| | | | | | | | | After the unihash changes the default signature handler didn't work. Tweak it to adapt to those changes (allowing the runqueue tests to work). (Bitbake rev: 7d486d3fb7176a3486f3f2484457724d7185df58) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: multiconfig: Switch from 'multiconfig' -> 'mc'Richard Purdie2019-06-101-4/+4
| | | | | | | | | | | | | | After real world use its clear the "multiconfig:" prefix to multiconfig tasks, whilst clear, is also clumbersome. Switch to use the short version instead. mcdepends will continue to work with "multiconfig:" for now as well. The commandline will only accept mc: going forward. [YOCTO #11168] (Bitbake rev: 821daf093b76504067a8b77dfa4b181af6ec92b4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-041-0/+4
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb: siggen: Print more info when basehash are mis-matchedRobert Yang2019-04-231-0/+6
| | | | | | | | | This is useful for debugging. (Bitbake rev: 07b6054b32b37375e2de4b1276bf6fe3f9ad0b04) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb: siggen: Make dump_sigfile and compare_sigfiles print uuid4Robert Yang2019-04-231-1/+9
| | | | | | | | | | | | | | | | | | | | | This can make people easier to understand bitbake-diffsigs/dumpsig's output, otherwise, it's hard to know it is a random uuid unless look into the code. E.g.: $ bitbake bc-native -ccleansstate -Snone $ bitbake bc-native -ccleansstate -Snone $ bitbake-diffsigs tmp/stamps/x86_64-linux/bc-native/1.07.1-r0.do_cleansstate.sigdata.* * Before: Taint (by forced/invalidated task) changed from nostamp:fe79d162-c4a8-4174-8007-f6d4aa09abdc to nostamp:28192187-5021-40c1-9e21-45483b62c910 * Now: Taint (by forced/invalidated task) changed from nostamp(uuid4):fe79d162-c4a8-4174-8007-f6d4aa09abdc to nostamp(uuid4):28192187-5021-40c1-9e21-45483b62c910 (Bitbake rev: 724b4a5cec8c611d53350f3e5a3988ec3222684b) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix multiconfig corner caseRichard Purdie2019-02-251-0/+2
| | | | | | | | | | | | | There was already a fix to ignore some multiconfig dependencies but its 'opposite' case wasn't covered. Cover that combination to so as to avoid tracebacks in multiconfig builds. [YOCTO #13090] [YOCTO #13130] (Bitbake rev: c883dfe378af9dfc192a8e392e84325d68648806) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-1/+1
| | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. (Bitbake rev: c1fcc46e2498ddd41425d8756754f814d682aba3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-15 16:46:22 +0000