summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/siggen.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: siggen: Ensure taskdata default functions exist in base classRichard Purdie2015-01-231-6/+7
| | | | | | | | | | | | The get/set_taskdata functions are now part of the API of the class, ensure they exist in the base class definition so the noop handler works. [YOCTO #7233] (Bitbake rev: 9b5b1bd7d77e3f5886f6c557d3b750de1f6d6025) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache/fetch2/siggen: Ensure we track include history for file checksumsRichard Purdie2015-01-081-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if you reference a file url, its checksum is included in the task hash, however if you change to a different file at a different location, perhaps taking advantage of the FILESPATH functionality, the system will not reparse the file in question and change its checksum to match the new file. To correctly handle this, the system not only needs to know if the existing file still exists or not, but also check the existance of every file it would have looked at when computing the original file. We already do this in the bitbake parsing code for class inclusion. This change uses the same technique to log the file list we looked at and if files in these locations exist when they previously did not, to invalidate and reparse the file. Since data stored in the cache is flattened text, we have to use a string form of the data and split on the ":" character which is ugly, but is an internal detail we can improve later if a better method is found. The cache version changes to trigger a reparse since the previous cache data is now incompatible. [YOCTO #7019] (Bitbake rev: 6c0706a28d72c591f1b75b6e3f3b645859387c7e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: ensure nostamp tasks force dependent tasks to re-executePaul Eggleton2014-12-231-0/+6
| | | | | | | | | | | | | If a nostamp task is depended on by a non-nostamp task, then we want the signature of that task to change such that it re-executes afterwards. This is an unusual situation, but we want this to work in OE in externalsrc.bbclass so that compilation happens every time it is requested. (Bitbake rev: 73498afc3d45beede5b8f24a9acd523a1663b793) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix shared work checksum mismatch/rebuild issuesRichard Purdie2014-11-081-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Similar to the last shared work task signature bug, we've found another one. Looking at the improved output of diffsigs in this case: runtaskdeps changed from [ 'autoconf_2.69.bb.do_populate_sysroot:virtual:native', 'gnu-config_20120814.bb.do_populate_sysroot:virtual:native', 'libgcc-initial_4.9.bb.do_patch:virtual:nativesdk' ] to [ 'autoconf_2.69.bb.do_populate_sysroot:virtual:native', 'gcc-crosssdk-initial_4.9.bb.do_patch', 'gnu-config_20120814.bb.do_populate_sysroot:virtual:native' ] so we can get a different task hash since libgcc sorts before gnu-config and gcc sorts after it. We could do with a way of fixing this, the best I can come up with is to include a single parent directory. Since recipes are never at the top of any metadata trees I've seen, this should suffice for now. I'm planning to burn the concept of shared work within bitbake and do something at the metadata level in the 1.8 timeframe as its just too fragile as things stand and hard to fix well. (Bitbake rev: d753644c67d163f338f2bdc3d600203e8b1a5734) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add computed hash code to verify hash computation to dumpsigRichard Purdie2014-11-081-0/+13
| | | | | | | | | This is useful code to double check the computed checksum value if nothing else. Might as well have it in tree. (Bitbake rev: 54ecf96c6f031927ee2410f6efde4e16f19bbf66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure we output if the ordering of runtaskdeps changesRichard Purdie2014-11-081-0/+17
| | | | | | | | | | | | | Order of runtaskdeps is important. If the hashes differ we should print output. This is complicated by shared work where the filenames themselves can differ, but the checksum should not. This fixes a case where two different checksums could show no output with bitbake-diffsigs. (Bitbake rev: 40c95cb9def282dc88234cd72ff462d7a01e47c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen/runqueue/bitbake-worker: Improve siggen data transfer interfaceRichard Purdie2014-09-111-4/+5
| | | | | | | | | | | | | We need to transfer some of the siggen data from the core/cooker into the worker instances. There was a partial API created for this but its ugly and its not possible to extend it from the siggen class. This patch completes the interface/abstraction for the data and means the class can extend/customise it in any siggen class. (Bitbake rev: cf2d642052979d236185c5b8ca2c5478c06e62ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix a subtle bug in hash calculation for shared work tasksRichard Purdie2014-08-231-1/+1
| | | | | | | | | | | | | | | | | | | With a shared work task like gcc, the task can be run from a variety of different recipes which may have different virtual extensions in place. Depending on whether gcc-runtime or nativesdk-gcc-runtime's do_preconfigure task is called for example will change the sorting of the task hashes due to the way clean_basename currently works. The correct thing to do here is sort on the base filename first, then any extension when ordering the hashes. This means we do account for things like recipes with both a native and non-native dependency but we also fix the shared work case where we don't care whether it was a virtual version or not. (Bitbake rev: 2e80b5d10a5037ed6f0bc227a1f9b42529c87086) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Print warning about tainted tasksRichard Purdie2014-06-011-0/+1
| | | | | | | | | | The big warning printed when people use -f is easily ignored/forgotten. To raise user awareness, print a warning any time we include a tainted stamp file into a build instead. (Bitbake rev: 18f9bcbad059608e22fca20309314e1c399acec7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/siggen: Pass in commandline options to dump_sigs()Richard Purdie2014-03-271-2/+2
| | | | | | | | | This allows the commandline options to be processed in the dump signature code. (Bitbake rev: ef8537a2e9b48f4fe065a165c102935aee2c9029) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen.py: fix the SignatureGenerator()Robert Yang2014-01-281-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | If we set: BB_SIGNATURE_HANDLER = "noop" Then we would get the following errors: [snip] File "runqueue.py", line 876, in RunQueue._start_worker(fakeroot=False, rqexec=None): "fakerootnoenv" : self.rqdata.dataCache.fakerootnoenv, > "hashes" : bb.parse.siggen.taskhash, "hash_deps" : bb.parse.siggen.runtaskdeps, AttributeError: 'SignatureGenerator' object has no attribute 'taskhash' [snip] This patch fixes the problem. [YOCTO #5741] (Bitbake rev: 2bfcb751891cf3b4050e996b3c8e28678c3a8bf4) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: SignatureGenerator: Add empty implementation for dump_sigsMartin Jansa2014-01-181-0/+2
| | | | | | | | | | | * bitbake -S throws exception when 'noop' signature handler is used [YOCTO #5738] (Bitbake rev: 53352e8d388b7fc4da73f95b93dcc087e76d0426) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Remove fsync that is causing performance issuesRichard Purdie2014-01-181-1/+0
| | | | | | | | | | | | | | | | | | | | | | This fsync was added for belt and braces protection for things like sstate on NFS. To be honest, it probably doesn't buy much, if the rename isn't atomic, all bets are off anyway and there are bigger issues to worry about. The issue is that at the end of every task, the dump_sig() code is triggered to save out information about the task and this was triggering an fsync(fd). Whilst it may select the file descriptor, on file systems like ext4, it will require large parts of the journal to be written out so it can have significant impact. latencytop showed an average fsync() call overhead of about 2s and if that happens for 5000 tasks, the time mounts up. This blocks the next task execution by that time. We therefore drop the fsync since in reality its causing problems and is unlikely to buy much. (Bitbake rev: 46fd841319479f6079d850b3813e64bd8c2680a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix reversed difference outputRichard Purdie2013-12-201-6/+6
| | | | | | | | | | The output when comparing siginfo files for dict_diff is reversed and shows additions when things were removed and vice versa. This patch reverses the operation so the changes are shown correctly and makes the output less confusing. (Bitbake rev: 9b4142df36619099670740a5d3bc94e404ab2b56) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: When printing signatures recursively, limit the outputRichard Purdie2013-12-201-1/+3
| | | | | | | | | | | | | | | | Currently the code prints all differences. If the task dependencies have changed hash, we recurse into those and print those differences as well. This leads to a lot of output. The reality is if the parents changed signature, we might as well just say that and recurse with no other output since we're much more interested in how the parents changed in nearly all cases. The changes in the parent are probably the same ones we'd have printed at each level anyway. By doing this we focus the output more carefully on the thing the user wants/needs to see. (Bitbake rev: 7a17fd6e51a76d3582c357b79f5ef86e1969650c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: handle recipe path changes in siginfo filesPaul Eggleton2013-10-141-4/+37
| | | | | | | | | | | | | | | | | | | | | Avoid storing paths to files in SRC_URI when writing out the the file checksums to siginfo files. This prevents a move of the source directory being reported by bitbake-diffsigs as files being removed and then added (the signature itself is not affected since the file paths have never been included in the signature). This has required the format of the file checksums in the siginfo file to be changed from a dict to a list of tuples (in order to handle multiple files with the same name under different paths, which is uncommon but possible); the code remains backwards-compatible with older siginfo files that use a dict however. Fixes [YOCTO #5245]. (Bitbake rev: e4d3077c5b0cc57964640512f3646c2d73c1d855) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Use lookup cache exclusivelyRichard Purdie2013-09-171-12/+3
| | | | | | | | | | All the values we need are already guaranteed to be in the lookupcache so rather than fetch variables again, just use the cache. This gives a small performance improvement and simplifies the code. (Bitbake rev: 8ffaba61da7f195d7c3b64dce35b6a56272aecae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Split runqueue to use bitbake-workerRichard Purdie2013-06-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a pretty fundamental change to the way bitbake operates. It splits out the task execution part of runqueue into a completely separately exec'd process called bitbake-worker. This means that the separate process has to build its own datastore and that configuration needs to be passed from the cooker over to the bitbake worker process. Known issues: * Hob is broken with this patch since it writes to the configuration and that configuration isn't preserved in bitbake-worker. * We create a worker for setscene, then a new worker for the main task execution. This is wasteful but shouldn't be hard to fix. * We probably send too much data over to bitbake-worker, need to see if we can streamline it. These are issues which will be followed up in subsequent patches. This patch sets the groundwork for the removal of the double bitbake execution for psuedo which will be in a follow on patch. (Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: Use modern expcetion syntaxRichard Purdie2013-05-091-1/+1
| | | | | | (Bitbake rev: a4a37b6a83faa62f61433122c4583e93e64f7372) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add missing error message parameterRichard Purdie2013-05-091-1/+1
| | | | | | | | There is clearly a missing parameter from this error message, add it. (Bitbake rev: d98be5ea69b99fb7934fc3093422f139bc37a1cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: add quotes around variable values to see whitespaceMartin Jansa2013-03-041-4/+4
| | | | | | | | | | | | | | | | | | * without this change it's hard to see what's changed: bitbake-diffsigs sstate-cache/*/sstate-depmodwrapper-cross-cortexa8-vfp-neon-oe-linux-gnueabi-1.0-r0-cortexa8-vfp-neon-*_populate-sysroot.tgz.siginfo basehash changed from 62f28c13172d38af917716d6d4b15ac5 to 9cf38842bcc79e0a4356856a64141548 Variable TUNE_CCARGS value changed from -march=armv7-a -mthumb -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 to -march=armv7-a -mthumb -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 Hash for dependent task depmodwrapper-cross_1.0.bb.do_install changed from 4494cb6c117309a9b4c541ad86c0aac5 to 1e3bcf2d14e5bd30a38350b41bb98db3 * with this change, we can see extra trailing space in first version: basehash changed from 62f28c13172d38af917716d6d4b15ac5 to 9cf38842bcc79e0a4356856a64141548 Variable TUNE_CCARGS value changed from ' -march=armv7-a -mthumb -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 ' to ' -march=armv7-a -mthumb -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8' Hash for dependent task depmodwrapper-cross_1.0.bb.do_install changed from 4494cb6c117309a9b4c541ad86c0aac5 to 1e3bcf2d14e5bd30a38350b41bb98db3 (Bitbake rev: 7d85b1a383feaa072503d71d3baeb3bdcd1012a8) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Remove whitelisted vars from non-task depsChristopher Larson2013-02-141-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Though the value of variables in the BB_BASEHASH_WHITELIST is kept out of the checksums, dependency on them is not, at least for variables and non-task functions. In the code, the whitelist is removed from the overall task dep list, but not the individual variable deps. The result of this is that functions like sysroot_stage_all and oe_runmake end up with whitelisted variables like TERM listed in their dependencies, which means that doing a 'unset TERM' before building will result in all checksums for tasks that depend on those changing, and shared state reuse not behaving correctly. This is only really a potential issue for variables from the environment, as it's the existance/removal of the variable that's an issue, not its value, and the other whitelisted variables are set in our metadata. This which means in practical terms the only cases where this is likely to be an issue are in environments where one of the following are unset: TERM, LOGNAME, HOME, USER, PWD, SHELL. This may seem like an unlikely circumstance, but is in fact a real issue for those of us using autobuilders. Jenkins does not set TERM when executing shell, which means shared state archives produced by your jenkins server would not be fully reused by an actual user. Fixed by removing the whitelisted elements from the individual variable deps, not just the accumulated result. (Bitbake rev: dac12560ac8431ee24609f8de25cb1645572d350) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/siggen.py: Avoid removing too many stamps when cleaningRichard Purdie2012-11-281-2/+1
| | | | | | | | | | | | | | | | The "*" part of the mask is to ensure we clean both any stamp, and any setscene varient. It turns out we would also trample other tasks, e.g. do_package_write could trample do_package_write_rpm. do_package also tramples do_package_write_* but this is less of an issue since the other tasks depend on it. Rather than use the wildcard, we can just use a list instead. [YOCTO #3484] (Bitbake rev: c14d831ea3f625e9a47266a0c4e6deefc924ca5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/siggen: Add support for stamp 'clean' masksRichard Purdie2012-09-201-2/+11
| | | | | | | | | | | | | | | | | | | | | Currently when we execute a task, we don't remove other potentially stale stamps. This can mean if you switch between two different versions of a recipe without a clean, the build can get very confused. This patch adds in functionality to allow a wildcard expression of stamp files to be removed when creating a new stamp file. This patch adds in the core of the code to enable this but it also requires metadata support to enable it. When writing this improvement I went through several different options but this was the only way I could find to allow things like noexec tasks to function correctly (where stamps need to be created without the data store). [YOCTO #2961] (Bitbake rev: e026469b307522e5b6a680e0ae5587749d33dcae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-diffsigs: allow specifying task & follow deps recursivelyPaul Eggleton2012-09-071-7/+10
| | | | | | | | | | | | | | | Add the ability to compare the two most recent runs of a specified task, and follow dependent hash changes recursively. This enables you to trace back and find exactly why a task was re-run after the fact. Note that this relies on the metadata providing a function, hooked in as bb.siggen.find_siginfo, which allows searching in the appropriate places to find signature data files. (Bitbake rev: cc70181659c07e04c205e17832846acf1ff31d28) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen.py: make signature dump/compare functions return a listPaul Eggleton2012-09-071-30/+38
| | | | | | | | | | These functions become a little bit more reusable if they return a list containing the output rather than just printing it. (Bitbake rev: a0ad2a947b71abcc0a1244cf139b9e9dfd8ee049) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen.py: insert a colon between class and recipe namePaul Eggleton2012-09-071-1/+1
| | | | | | | | | | | | | | | | | | | before: virtual:nativeautomake_1.12.1.bb.do_compile after: virtual:native:automake_1.12.1.bb.do_compile This separation ensures that the key is readable, and if necessary, parsable. Unfortunately this invalidates previous native sstate signatures with OE-Core - not much that can be done about that; however that occurs frequently during the development cycle so it's par for the course. (Bitbake rev: 5b96c32dad256090e9bda5af0f80c7dbcc90bde8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: replace tabs with spacesPaul Eggleton2012-09-071-17/+17
| | | | | | | | | | We had one section of the code mixing tabs with spaces, which is particularly undesirable with python code. (Bitbake rev: 8eaa093b179e03a6003a47220540b1bc73afca17) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen.py: Insure .siginfo files writes into shared sstate cache ↵Jeffrey C Honig2012-07-171-4/+15
| | | | | | | | | | | | | are atomic Use tempfile.mkstemp to create a temporary file in the sstate dir and move it into place after closing. The previous code would fail in the chmod() if two users were running jobs that touched the same signature file. (Bitbake rev: ff11e9ac5eba2d957917664a7b91b1277d8ad548) Signed-off-by: Jeffrey C Honig <jeffrey.honig@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen.py: log when tainting the signature of a taskPaul Eggleton2012-06-251-0/+1
| | | | | | | | | | | Log a note when applying a taint to a task signature (e.g. when using the -f or -C command line options) so that the user knows this has been done. (Bitbake rev: 0fd960fdea83874eedb541cbc2920257e0f3fb81) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: ensure -f causes dependent tasks to be re-runPaul Eggleton2012-06-211-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If -f is specified, force dependent tasks to be re-run next time. This works by changing the force behaviour so that instead of deleting the task's stamp, we write a "taint" file into the stamps directory, which will alter the taskhash randomly and thus trigger the task to re-run next time we evaluate whether or not that should be done as well as influencing the taskhashes of any dependent tasks so that they are similarly re-triggered. As a bonus because we write this file as <stamp file name>.taskname.taint, the existing code which deletes the stamp files in OE's do_clean will already handle removing it. This means you can now do the following: bitbake somepackage [ change the source code in the package's WORKDIR ] bitbake -c compile -f somepackage bitbake somepackage and the result will be that all of the tasks that depend on do_compile (do_install, do_package, etc.) will be re-run in the last step. Note that to operate in the manner described above you need full hashing enabled (i.e. BB_SIGNATURE_HANDLER must be set to a signature handler that inherits from BasicHash). If this is not the case, -f will just delete the stamp for the specified task as it did before. This fix is required for [YOCTO #2615] and [YOCTO #2256]. (Bitbake rev: f7b55a94226f9acd985f87946e26d01bd86a35bb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: include varflags in checksumsPaul Eggleton2012-05-301-0/+4
| | | | | | | | | | | | | | | | | | | | | | Add a dependency to each variable on a filtered list of its varflags. This is intended to catch things such as SRC_URI checksums, varflags controlling extra functionality from classes (e.g. the recently updated update-alternatives class in OE-Core), etc. and ensure their values influence the sstate checksums. There is an exclusion list which needs to be set via bitbake.conf (BB_SIGNATURE_EXCLUDE_FLAGS), if this is not set then the functionality is disabled. The existing vardepsexclude mechanism can also be used to exclude undesired varflags, but they must be fully specified, e.g.: do_patch[vardepsexclude] += "do_patch[someflag]" Implements [YOCTO #2517]. (Bitbake rev: 56c1ab18fcaf4ac245dcb412ed55e8e0af07883b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: implement checksums for local files in SRC_URIPaul Eggleton2012-05-231-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Gathers a list of paths to have checksums calculated at parse time, and processes these when calculating task hashes. Checksums are cached with the file's current mtime. Thus, changing any local file in SRC_URI will now cause the do_fetch taskhash to change, thus forcing a rebuild. This change adds very roughly about an 8% increase in parse time (a few seconds) and maybe a few seconds during runqueue generation, so a fairly moderate performance hit. Note that since paths are resolved at parse time, this will not force a rebuild when files are introduced which would cause that resolved path to be different - for example, where a machine-specific version of a file was added without otherwise changing the recipe. This will need to be handled in a future update. Code to hook this into the signature generator was courtesy of Richard Purdie <richard.purdie@linuxfoundation.org>. Implements [YOCTO #2044]. (Bitbake rev: c993b7c457f8b7776e8a5dff253bfa0724bc2cae) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen.py: ensure dumped signatures have group R/WJoshua Lock2012-05-131-2/+4
| | | | | | | | | | We anticipate signatures being shared amongst peers in a group so ensure they have rw-rw-r-- permissions to facilitate this. (Bitbake rev: e4716f316152cafabebcefec7b387edeb02ad3eb) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen: use open() rather than the file constructorJoshua Lock2012-05-131-4/+4
| | | | | | | | | | | | | | | | The Python documentation states: "When opening a file, it’s preferable to use open() instead of invoking the file constructor directly." [1] Further in Python 3 direct use of the file constructor is no longer possible. 1. http://docs.python.org/library/functions.html#open (Bitbake rev: 759f953e29a7131614e5b1f0312edf2b17523675) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Use basehash if no taskhashRobert Yang2012-03-121-1/+5
| | | | | | | | | | | | | | The bitbake-runtask doesn't work since there is no taskhash (it worked in the past, but seems recent changes made it doesn't work again), use the basehash when there is no taskhash, just like what dump_sigtask does. [YOCTO #1229] (Bitbake rev: b36f2e3803dfc494d5b171759f55faf33708e6f1) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen: output filename on error in finalisePaul Eggleton2012-03-121-1/+5
| | | | | | | | | | | | | If an error occurs during finalise() (when for example we might get an expansion error in SRCPV), log a note which mentions which recipe was being finalised. Fixes [YOCTO #1782] (Bitbake rev: 56f9f8ffd92b00b916dbc2386150c83689d5beed) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Pass the datacache to signature generator classesRichard Purdie2012-02-231-2/+2
| | | | | | | | | | | | | | | | The dataCache exposes much useful information to the singature generation code for example its possible to figure out whether kernel dependencies are important or not based on the RRECOMMENDS values. We therefore pass this information to the handler and trust it to handle any API changes in the dataCache structure. For backwards compatibility, OE-Core is already updated to handle the extra argument when it appears. (Bitbake rev: 4ebc25155d5f96a7f63de22b52b18d045eb4dec7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: compare runtaskdeps dictionary even when they have the same sizeMartin Jansa2012-02-221-21/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | * otherwise it reports lots of changed checksums just because it compares different tasks notice linux-libc-headers_3.1.bb.do_package_write: Task dependency hash changed from 42acced29debf54d598802474c5e9cbb to f438a54f995df77620d0727d6f4b4ce5 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy) Task dependency hash changed from 61f8babe1d10c6e7fb1423112bb04e1e to 1b3e21ff106ecfcb7ddf76a1e29537bb (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write) Task dependency hash changed from 512f9d6686d760b318d8b11c8b589226 to 42acced29debf54d598802474c5e9cbb (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write) Task dependency hash changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write) Task dependency hash changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write) with this patch: Dependency on task linux-nokia900-meego_git.bb.do_package_write was added Dependency on task linux-nokia900-meego_git.bb.do_deploy was added Dependency on task linux-gta04_git.bb.do_deploy was removed Dependency on task linux-gta04_git.bb.do_package_write was removed Hash for dependent task systemd_git.bb.do_package_write changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17 Hash for dependent task systemd-serialgetty.bb.do_package_write changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5 * added test if there is different task with same hash then we don't show it as added/removed dependency, because bitbake doesn't care (Bitbake rev: ca52bf32b479811bd7fed41648bedcc06b00430b) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Handle cases where the input to symmetric_difference() may be NoneRichard Purdie2012-02-101-3/+6
| | | | | | | | [YOCTO #1967] (Bitbake rev: f6cee29bf853874876923a7d6ae8574e3801ae8d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Abstract the runtime task dependency handling code in the generatorsRichard Purdie2012-01-201-7/+18
| | | | | | | | | | This means that custom signature handlers can override specific parts of the code without having to reimplement whole functions allowing them more flexibility. (Bitbake rev: 164195c068a656733cfe7aa07369c5ed6ea62ca5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: Ensure correct runtask dependency output is shown for diffsigsRichard Purdie2012-01-201-15/+21
| | | | | | | | | | | | | The actual task names are discounted for comparison of dependent tasks, only the actual hashes are used. This updates the comparison code to account for this change, attempting heuristic matching for more user friendly output but falling back to showing the changed hashes directly. This avoids some confusing output to users where it looked like tasks had changed when they had not and actually had the same hash but a different taskname. (Bitbake rev: 7da7dff83ed765c9cde1d7f91ee1b65e49520481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: If both sigs have a variable in it's whitelist then don't say ↵Matthew McClintock2011-12-051-3/+3
| | | | | | | | | | | | | it's changed Some BB_HASHBASE_WHITELIST variables are in the lists of variable dependencies for signatures. Ignore those differences in lists since this difference does not matter (Bitbake rev: 71b53a3f0766ca464560a1f6a449f9424fbdf7ae) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen.py: Don't backtrace if the taskhash data isn't presentRichard Purdie2011-11-251-2/+2
| | | | | | | | | | This allows the code to safely fall back to dumping the basehash data if the taskhash data isn't present for some reason. We could effecitvely obsolete the runtime option and use this approach instead exclusively. (Bitbake rev: 5ace320ccc01f4e326f90b7ba060dcbff3380dca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data/siggen: Add vardepvalue mechanism to allow the variable dependency code ↵Richard Purdie2011-11-241-2/+1
| | | | | | | | | | | | | | | | | | | | | | | to be forced to specific values We have a problem if we want to inject specific information into the variable dependency code. There are cases for example where we want a dependency on the value of X but it doesn't matter how X was constructed or what dependencies it might have had, we only care about the absolute value. With the current code, its near enough impossible to do this. This patch adds such a mechanism so the user can trigger this with code like: baselib[vardepvalue] = "${baselib}" It also refactors some of the code so we do variable lookups once instead of doing this in two different functions. [YOCTO #1583] (Bitbake rev: 6c879b44ccf42dc73fe4467076e114700d7ba81b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: sort task hash depedencies with basepathMatthew McClintock2011-11-231-11/+14
| | | | | | | | | | | | | | | Without this patch the tash hash dependencies can be in a order that is dependent upon directory/filesystem layout. With this change the data is sorted the same regardless. Without this the dependent hashes could be in different orders on different systems and consequently final md5 hash would differ as well even though nothing else changed. (Bitbake rev: 9a2029899c946ce9aa8adbc85f2cfe7a85b92182) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: print out symmetric difference when comparing sigsMatthew McClintock2011-11-231-0/+4
| | | | | | | | | | This is useful for really longs lists to pinpoint what has actually changed (Bitbake rev: f1eb6d3dcc10c42bb09383a87bde3afa69bc6ed9) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Include list of variables in hashesRichard Purdie2011-11-181-0/+1
| | | | | | | | | | | | | Ensure that the list of dependencies is included in the hash as well as their contents Prior to this, adding or removing dependencies with values of "None" would not change the hash, despite diffsigs reporting this difference. (Bitbake rev: 727ca945177ce9bd44515cf611e3e95a09466d98) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Fix diffsigs output for filename comparisionsRichard Purdie2011-11-181-3/+15
| | | | | | | | | | | | | | | When comparing sig files, if the recipe locations had changed, the dependent tasks list would show as changed even if the actual hash had not changed. This updates the code to only compare the base part of the pathnames. It also tweaks some of the output to add newlines to aid comparing two lists of variables as it makes the location of the difference clearer. (Bitbake rev: 165a22ddcc647b945707fb5c483146bb336d5f66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 02:59:22 +0000