summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/siggen.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: siggen.py: Insure .siginfo files writes into shared sstate cache ↵Jeffrey C Honig2012-07-171-4/+15
| | | | | | | | | | | | | are atomic Use tempfile.mkstemp to create a temporary file in the sstate dir and move it into place after closing. The previous code would fail in the chmod() if two users were running jobs that touched the same signature file. (Bitbake rev: ff11e9ac5eba2d957917664a7b91b1277d8ad548) Signed-off-by: Jeffrey C Honig <jeffrey.honig@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen.py: log when tainting the signature of a taskPaul Eggleton2012-06-251-0/+1
| | | | | | | | | | | Log a note when applying a taint to a task signature (e.g. when using the -f or -C command line options) so that the user knows this has been done. (Bitbake rev: 0fd960fdea83874eedb541cbc2920257e0f3fb81) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: ensure -f causes dependent tasks to be re-runPaul Eggleton2012-06-211-0/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If -f is specified, force dependent tasks to be re-run next time. This works by changing the force behaviour so that instead of deleting the task's stamp, we write a "taint" file into the stamps directory, which will alter the taskhash randomly and thus trigger the task to re-run next time we evaluate whether or not that should be done as well as influencing the taskhashes of any dependent tasks so that they are similarly re-triggered. As a bonus because we write this file as <stamp file name>.taskname.taint, the existing code which deletes the stamp files in OE's do_clean will already handle removing it. This means you can now do the following: bitbake somepackage [ change the source code in the package's WORKDIR ] bitbake -c compile -f somepackage bitbake somepackage and the result will be that all of the tasks that depend on do_compile (do_install, do_package, etc.) will be re-run in the last step. Note that to operate in the manner described above you need full hashing enabled (i.e. BB_SIGNATURE_HANDLER must be set to a signature handler that inherits from BasicHash). If this is not the case, -f will just delete the stamp for the specified task as it did before. This fix is required for [YOCTO #2615] and [YOCTO #2256]. (Bitbake rev: f7b55a94226f9acd985f87946e26d01bd86a35bb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: include varflags in checksumsPaul Eggleton2012-05-301-0/+4
| | | | | | | | | | | | | | | | | | | | | | Add a dependency to each variable on a filtered list of its varflags. This is intended to catch things such as SRC_URI checksums, varflags controlling extra functionality from classes (e.g. the recently updated update-alternatives class in OE-Core), etc. and ensure their values influence the sstate checksums. There is an exclusion list which needs to be set via bitbake.conf (BB_SIGNATURE_EXCLUDE_FLAGS), if this is not set then the functionality is disabled. The existing vardepsexclude mechanism can also be used to exclude undesired varflags, but they must be fully specified, e.g.: do_patch[vardepsexclude] += "do_patch[someflag]" Implements [YOCTO #2517]. (Bitbake rev: 56c1ab18fcaf4ac245dcb412ed55e8e0af07883b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: implement checksums for local files in SRC_URIPaul Eggleton2012-05-231-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | Gathers a list of paths to have checksums calculated at parse time, and processes these when calculating task hashes. Checksums are cached with the file's current mtime. Thus, changing any local file in SRC_URI will now cause the do_fetch taskhash to change, thus forcing a rebuild. This change adds very roughly about an 8% increase in parse time (a few seconds) and maybe a few seconds during runqueue generation, so a fairly moderate performance hit. Note that since paths are resolved at parse time, this will not force a rebuild when files are introduced which would cause that resolved path to be different - for example, where a machine-specific version of a file was added without otherwise changing the recipe. This will need to be handled in a future update. Code to hook this into the signature generator was courtesy of Richard Purdie <richard.purdie@linuxfoundation.org>. Implements [YOCTO #2044]. (Bitbake rev: c993b7c457f8b7776e8a5dff253bfa0724bc2cae) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen.py: ensure dumped signatures have group R/WJoshua Lock2012-05-131-2/+4
| | | | | | | | | | We anticipate signatures being shared amongst peers in a group so ensure they have rw-rw-r-- permissions to facilitate this. (Bitbake rev: e4716f316152cafabebcefec7b387edeb02ad3eb) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen: use open() rather than the file constructorJoshua Lock2012-05-131-4/+4
| | | | | | | | | | | | | | | | The Python documentation states: "When opening a file, it’s preferable to use open() instead of invoking the file constructor directly." [1] Further in Python 3 direct use of the file constructor is no longer possible. 1. http://docs.python.org/library/functions.html#open (Bitbake rev: 759f953e29a7131614e5b1f0312edf2b17523675) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Use basehash if no taskhashRobert Yang2012-03-121-1/+5
| | | | | | | | | | | | | | The bitbake-runtask doesn't work since there is no taskhash (it worked in the past, but seems recent changes made it doesn't work again), use the basehash when there is no taskhash, just like what dump_sigtask does. [YOCTO #1229] (Bitbake rev: b36f2e3803dfc494d5b171759f55faf33708e6f1) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen: output filename on error in finalisePaul Eggleton2012-03-121-1/+5
| | | | | | | | | | | | | If an error occurs during finalise() (when for example we might get an expansion error in SRCPV), log a note which mentions which recipe was being finalised. Fixes [YOCTO #1782] (Bitbake rev: 56f9f8ffd92b00b916dbc2386150c83689d5beed) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Pass the datacache to signature generator classesRichard Purdie2012-02-231-2/+2
| | | | | | | | | | | | | | | | The dataCache exposes much useful information to the singature generation code for example its possible to figure out whether kernel dependencies are important or not based on the RRECOMMENDS values. We therefore pass this information to the handler and trust it to handle any API changes in the dataCache structure. For backwards compatibility, OE-Core is already updated to handle the extra argument when it appears. (Bitbake rev: 4ebc25155d5f96a7f63de22b52b18d045eb4dec7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: compare runtaskdeps dictionary even when they have the same sizeMartin Jansa2012-02-221-21/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | * otherwise it reports lots of changed checksums just because it compares different tasks notice linux-libc-headers_3.1.bb.do_package_write: Task dependency hash changed from 42acced29debf54d598802474c5e9cbb to f438a54f995df77620d0727d6f4b4ce5 (for linux-libc-headers_3.1.bb.do_package_write and linux-gta04_git.bb.do_deploy) Task dependency hash changed from 61f8babe1d10c6e7fb1423112bb04e1e to 1b3e21ff106ecfcb7ddf76a1e29537bb (for linux-nokia900-meego_git.bb.do_deploy and linux-gta04_git.bb.do_package_write) Task dependency hash changed from 512f9d6686d760b318d8b11c8b589226 to 42acced29debf54d598802474c5e9cbb (for linux-nokia900-meego_git.bb.do_package_write and linux-libc-headers_3.1.bb.do_package_write) Task dependency hash changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5 (for systemd-serialgetty.bb.do_package_write and systemd-serialgetty.bb.do_package_write) Task dependency hash changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17 (for systemd_git.bb.do_package_write and systemd_git.bb.do_package_write) with this patch: Dependency on task linux-nokia900-meego_git.bb.do_package_write was added Dependency on task linux-nokia900-meego_git.bb.do_deploy was added Dependency on task linux-gta04_git.bb.do_deploy was removed Dependency on task linux-gta04_git.bb.do_package_write was removed Hash for dependent task systemd_git.bb.do_package_write changed from 730abebf9954794bb440c2f3239f79fe to 413eaebaff27a2fd16f5cf68c1f4ff17 Hash for dependent task systemd-serialgetty.bb.do_package_write changed from 153e91dfd1d2053fda7b98cc08d4b802 to 92a293bdd8ed234932b87a66025038c5 * added test if there is different task with same hash then we don't show it as added/removed dependency, because bitbake doesn't care (Bitbake rev: ca52bf32b479811bd7fed41648bedcc06b00430b) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Handle cases where the input to symmetric_difference() may be NoneRichard Purdie2012-02-101-3/+6
| | | | | | | | [YOCTO #1967] (Bitbake rev: f6cee29bf853874876923a7d6ae8574e3801ae8d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Abstract the runtime task dependency handling code in the generatorsRichard Purdie2012-01-201-7/+18
| | | | | | | | | | This means that custom signature handlers can override specific parts of the code without having to reimplement whole functions allowing them more flexibility. (Bitbake rev: 164195c068a656733cfe7aa07369c5ed6ea62ca5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: Ensure correct runtask dependency output is shown for diffsigsRichard Purdie2012-01-201-15/+21
| | | | | | | | | | | | | The actual task names are discounted for comparison of dependent tasks, only the actual hashes are used. This updates the comparison code to account for this change, attempting heuristic matching for more user friendly output but falling back to showing the changed hashes directly. This avoids some confusing output to users where it looked like tasks had changed when they had not and actually had the same hash but a different taskname. (Bitbake rev: 7da7dff83ed765c9cde1d7f91ee1b65e49520481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: If both sigs have a variable in it's whitelist then don't say ↵Matthew McClintock2011-12-051-3/+3
| | | | | | | | | | | | | it's changed Some BB_HASHBASE_WHITELIST variables are in the lists of variable dependencies for signatures. Ignore those differences in lists since this difference does not matter (Bitbake rev: 71b53a3f0766ca464560a1f6a449f9424fbdf7ae) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen.py: Don't backtrace if the taskhash data isn't presentRichard Purdie2011-11-251-2/+2
| | | | | | | | | | This allows the code to safely fall back to dumping the basehash data if the taskhash data isn't present for some reason. We could effecitvely obsolete the runtime option and use this approach instead exclusively. (Bitbake rev: 5ace320ccc01f4e326f90b7ba060dcbff3380dca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data/siggen: Add vardepvalue mechanism to allow the variable dependency code ↵Richard Purdie2011-11-241-2/+1
| | | | | | | | | | | | | | | | | | | | | | | to be forced to specific values We have a problem if we want to inject specific information into the variable dependency code. There are cases for example where we want a dependency on the value of X but it doesn't matter how X was constructed or what dependencies it might have had, we only care about the absolute value. With the current code, its near enough impossible to do this. This patch adds such a mechanism so the user can trigger this with code like: baselib[vardepvalue] = "${baselib}" It also refactors some of the code so we do variable lookups once instead of doing this in two different functions. [YOCTO #1583] (Bitbake rev: 6c879b44ccf42dc73fe4467076e114700d7ba81b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: sort task hash depedencies with basepathMatthew McClintock2011-11-231-11/+14
| | | | | | | | | | | | | | | Without this patch the tash hash dependencies can be in a order that is dependent upon directory/filesystem layout. With this change the data is sorted the same regardless. Without this the dependent hashes could be in different orders on different systems and consequently final md5 hash would differ as well even though nothing else changed. (Bitbake rev: 9a2029899c946ce9aa8adbc85f2cfe7a85b92182) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: print out symmetric difference when comparing sigsMatthew McClintock2011-11-231-0/+4
| | | | | | | | | | This is useful for really longs lists to pinpoint what has actually changed (Bitbake rev: f1eb6d3dcc10c42bb09383a87bde3afa69bc6ed9) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Include list of variables in hashesRichard Purdie2011-11-181-0/+1
| | | | | | | | | | | | | Ensure that the list of dependencies is included in the hash as well as their contents Prior to this, adding or removing dependencies with values of "None" would not change the hash, despite diffsigs reporting this difference. (Bitbake rev: 727ca945177ce9bd44515cf611e3e95a09466d98) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: Fix diffsigs output for filename comparisionsRichard Purdie2011-11-181-3/+15
| | | | | | | | | | | | | | | When comparing sig files, if the recipe locations had changed, the dependent tasks list would show as changed even if the actual hash had not changed. This updates the code to only compare the base part of the pathnames. It also tweaks some of the output to add newlines to aid comparing two lists of variables as it makes the location of the difference clearer. (Bitbake rev: 165a22ddcc647b945707fb5c483146bb336d5f66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/siggen.py: return a string from noop get_taskhashDmitry Eremin-Solenikov2011-09-051-1/+1
| | | | | | | | | | OpenEmbedded is expecting to get a string from get_taskhash, but noop siggen returns just 0 (number), so OE classes/sstate.bbclass barfs badly. Fix that. (Bitbake rev: 24272dae15ccf641ece11ef5a6e2bfa3ebb6f5f9) Signed-off-by: Dmitry Eremin-Solenikov <dbaryshkov@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: don't choke with traceback when data is NoneChris Larson2011-06-071-3/+5
| | | | | | | | | | | | | Given we use bb.error, not bb.fatal, here, it seems this was intended to be non-fatal, yet we'd end up trying to concatenate None. Fix this by setting an empty task to the empty string, for the purposes of hashing. Also str() the value we get from the datastore, just in case something other than a string was stored there. (Bitbake rev: ec8a5a495b72e061a1e8d7c7449afb26581872c0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* build/siggen: Ensure a task signature file is generated for each task that ↵Richard Purdie2011-05-251-0/+3
| | | | | | | | | | | | | | | | | | is executed Tracing down signature changes and dependencies is hard unless the complete task chain can be evaultated. Since we have the data available at task exeuction time, writing it to disk makes sense. This change is suboptimal internal knowledge stamp_internal() has leaks into make_stamp() and the concepts used by this code need revisiting but the change brings enough usability benefit to warrant its inclusion despite that. Addresses issue [YOCTO #1074] (Bitbake rev: 139b8a625818225c358a1b8363518d7ed6913188) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb.siggen: import osChris Larson2011-03-311-0/+1
| | | | | | | (Bitbake rev: cd274f653c677daa69b2e15ffed803b7b1d25a43) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen: add bb.data, bb.parse importsChris Larson2011-02-211-0/+2
| | | | | | | (Bitbake rev: 8eafb12208fcd073f930c0c74f25831d02c02198) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* siggen.py: better print for task hash comparisonKevin Tian2011-01-211-5/+12
| | | | | | | | | current bitbake-diffsigs simply print out the whole 'runtaskdeps' when there's mismatch, which is not very readable. On the other hand, 'runtaskhashes' comparison is broken which assumes same key existing in two sides. This commit provides better output by figuring out differences from addition, removal or hash change. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* bitbake: Introduce stamp-extra-info task flag into stamp filenamesDongxiao Xu2011-01-181-4/+4
| | | | | | | | | | | For certain tasks, we need additional information in build stamp file other than the task name and file name. stamp-extra-info is introduced as a task flag which is appended to the stamp file name. [Code simplifcations/tweaks from Richard] Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen.py: Fix whitelisted variable handlingRichard Purdie2011-01-121-6/+21
| | | | | | | | | Even when a variable was whitelisted, any dependencies of that variable could still creep into the task hash due to the way the whitelisting code worked. This patch changes thing to ensure that when whitelisted, that whitelisting applies to the variable and any dependencies it has. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen.py: Add a signature generator which adds the hash to the ↵Richard Purdie2011-01-071-0/+11
| | | | | | stamp files Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake build/siggen/runqueue: Fix stampfile parametersRichard Purdie2011-01-061-1/+1
| | | | | | | | The current parameters are not useful to the stampfile generator function as they can't uniquely define a task. This updated things so the parameters can identify unique tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-20/+19
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* siggen.py: print taskname when seeing mismatched hash in cacheKevin Tian2010-12-201-1/+1
| | | | Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* bitbake Revert bitbake exec() and go back to fork() for performace wins ↵Richard Purdie2010-12-171-6/+3
| | | | | | (first draft) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* siggen.py: make sure stamp directory existingKevin Tian2010-12-161-0/+2
| | | | | | or else "bitbake -S" from scratch may report "No such file or directory" error. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* siggen.py: fix the wrong usage on BB_TASKHASH_WHITELISTKevin Tian2010-12-071-4/+8
| | | | | | | | | | BB_TASKHASH_WHITELIST is expected to filter out native tasks from the dependency list for target recipe's checksum. However current code actually implements the opposite. All native sstate packages end up to have empty task dependency while target sstate packages still have native tasks counted into the checksum. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* siggen.py: set 'runtaskdeps' correctlyKevin Tian2010-12-061-6/+2
| | | | | | | | Now 'runtaskdeps' is assigned before taskhash whitelist is filtered out, and thus always contains original task dependencies. This causes problem to diagnose sstate related problem. So fix it by appending filtered dep to 'runtaskdeps' in the fly. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* siggen.py: fix comparison around runtaskdepsKevin Tian2010-11-151-2/+2
| | | | | | | | | two dep lists should be sorted before the comparison, or else it just causes false-positive confusion. Also fix 'taskdeps' to 'runtaskdeps' when printing out the difference. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* bitbake/siggen: Allow siggen code to control final stampfile formatRichard Purdie2010-11-061-0/+3
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen: Provide better warnings about empty tasksRichard Purdie2010-11-021-0/+2
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen: Fix finalise call argumentsRichard Purdie2010-11-011-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/bitbake-sigdiff: Extend to handle dumping single state siginfo filesRichard Purdie2010-09-301-0/+24
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen: Fix issue where excluded dependencies caused exceptionsRichard Purdie2010-09-301-0/+2
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen: Ensure full signature data is not held unless needed, ↵Richard Purdie2010-09-291-11/+18
| | | | | | reducing memory consumption Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Pass task hash information to subprocessesRichard Purdie2010-09-281-1/+5
| | | | | | | Pass task has informaiton to work processes, allowing full manipulation of the hash data in the task context allowing checksums to be usable. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen: Allow generation of siggen data from task contextRichard Purdie2010-09-131-2/+10
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/siggen.py: Improve debugging of checksumsRichard Purdie2010-09-131-7/+9
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Add missing signature filesRichard Purdie2010-09-031-0/+213
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-01 07:57:10 +0000