summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/siggen.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: cooker/siggen: Empty siggen cache during parsingRichard Purdie2020-02-181-0/+11
| | | | | | | | | | | | | | | | | | | When parsing recipes its apparent the memory usage of bitbake rises linearly with number of recipes parsed. It shouldn't. Using tracemalloc (thanks for the tip Joshua Lock) it was clear that the dependency information left behind in siggen was the culprit. Add a new method to allow us to drop this information. We don't need it after the recipe has been parsed and hashes calculated (at runtime its different but only the currently executing task would be in memory). This should give signficant memory usage improvements for bitbake and that in turn should help speed on more constrained systems, as well as when used in multiconfig environments. (Bitbake rev: d411b097d810c386d35dc561f8812bb3f35c9a36) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-06-191-0/+4
| | | | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ac556588fac55e91b7ce4839a975eb9ebb5aa192) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix multiconfig corner caseRichard Purdie2019-02-251-0/+2
| | | | | | | | | | | | | There was already a fix to ignore some multiconfig dependencies but its 'opposite' case wasn't covered. Cover that combination to so as to avoid tracebacks in multiconfig builds. [YOCTO #13090] [YOCTO #13130] (Bitbake rev: c883dfe378af9dfc192a8e392e84325d68648806) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-1/+1
| | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. (Bitbake rev: c1fcc46e2498ddd41425d8756754f814d682aba3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/siggen: Switch md5 -> sha256Richard Purdie2019-01-081-3/+3
| | | | | | | | | | Similarly to the codeparser change, change to sha256 hashes due to worries over collisions. The main impact of this change is slightly slower parsing time as well as longer sstate file names. (Bitbake rev: 66f1b766997d53b4375fdd25719b1175f3828903) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Split out task unique hashJoshua Watt2019-01-031-2/+5
| | | | | | | | | | | | | | Abstracts the function to get the unique hash for a task. This hash is used as in place of the taskhash for the purpose of determine how other tasks depend on this one. Unless overridden, the taskhash is the same as the unique hash, preserving the original behavior. [YOCTO #13030] (Bitbake rev: d0065b34cea81fcadff14b0383779e9559d94508) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Split out stampfile hash fetchJoshua Watt2018-12-071-4/+9
| | | | | | | | | | | | The mechanism used to get the hash for a stamp file is split out so that it can be overridden by derived classes [YOCTO #13030] (Bitbake rev: ce241534d19b2f1c51dbdb3b92419676d234e464) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Adapt colors used by bitbake-diffsigs to support light themesPeter Kjellerstedt2018-11-191-4/+4
| | | | | | | | | | | | | | | | | The colors specified for use with bitbake-diffsigs were adapted for a dark theme, e.g., by setting the background color to black, which made it look very bad when used with a light theme. To make it look good both with a dark or a light theme, it is better to drop the background color. It is also better to leave out the color altogether for the title and just use bold. Finally, dropping bold for the red and green texts indicating removed/added values better matches other colorized diff implementations as, e.g., git diff. (Bitbake rev: f1a2c23520832ee91e85338c1ad8af1fec0d0b19) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Correct format argument substitutionMike Crowe2018-11-071-1/+1
| | | | | | | | | | Use % operator to ensure that the dependency is substituted into the error message correctly. (Bitbake rev: 8e587386b0b3a0ed11edf71dfbe9fb508f60d0b3) Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/siggen: Extract task hash generation code into a functionRichard Purdie2018-10-181-33/+4
| | | | | | | | | By creating a standalone function, we can add better functional testing of this code. (Bitbake rev: 796a20d24dc18479de1975a37b9e52a5ac75c73f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add support for multiconfig dependenciesAlejandro Enedino Hernandez Samaniego2018-08-011-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the capability for tasks from different multiconfigs to depend on one another. These dependencies can be enabled using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to: - cooker: To glue both multiconfigs in one place and make sure the dependencies can be provided. - taskdata: To parse and add a new kind of dependency (mcdepends) to the taskdata object. - runqueue: To differentiate tasks from different multiconfigs, add the specified dependencies to the corresponding tasks, and create a working runqueue that contains tasks from both multiconfigs. - siggen: To avoid looking for tasks from different multiconfigs on objects where they dont belong. The taskdata objects are still not aware of the concept of multiconfig, so each object doesnt know which multiconfig its building, hence why the mcdepends are added to all taskdata objects equally (we really dont expect many of these), but the actual dependencies are added only to the required tasks by the runqueue. (Bitbake rev: da8cb8633504bdc815bdcefc538340b9bce5065d) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Delete trailing whitespaceJoshua Watt2018-07-091-2/+2
| | | | | | | (Bitbake rev: b2fd783fa23403e8f08d998951b05bcabb458af1) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: move reset() definition to base SignatureGenerator classPaul Eggleton2017-09-051-4/+2
| | | | | | | | | | | | If we're implementing reset() in SignatureGenerator at all (and we need to for a basic non-OE BitBake setup where that is the default signature generator), then we need it to be clearing out the internal values properly. (Bitbake rev: 13f52d38fdbcb84c2a0c46f85baa44b22d53fdc1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add reset to SignatureGeneratorCaio Marcelo de Oliveira Filho2017-08-311-0/+3
| | | | | | | | | | | Fix failure after commit "cooker/siggen: Reset siggen when reparsing" (e4c6ca9440f63761560b49bbe12654441f54687e) when executing without specifying a BB_SIGNATURE_HANDLER. (Bitbake rev: 2a78c2d09aea0323632bbc927f370f1d3c9c249e) Signed-off-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/siggen: Reset siggen when reparsingRichard Purdie2017-08-131-0/+3
| | | | | | | | | | | | If we don't do this, we get basehash mismatch errors occurring from the reparse which would then set bitbake's error exit code. This for example would cause oe-selftest -r bbtests.BitbakeTests.test_bbappend_order to fail with a non-zero BB_SERVER_TIMEOUT. (Bitbake rev: e4c6ca9440f63761560b49bbe12654441f54687e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Make calc_taskhash match get_taskhash for file checksumsRichard Purdie2017-06-051-1/+2
| | | | | | | | | | | The code in these two functions is meant to be equivlanet in behaviour but isn't. Add in code to ensure files that don't exist are handled consistently by both functions. Users did report being able to generate tracebacks otherwise. (Bitbake rev: c11b4832879354b387a8c9c6a3096f2d621afeac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/{data, siggen}: Don't warn on SkipRecipe exceptionsNathan Rossi2017-04-121-0/+2
| | | | | | | | | | | | Recipes/variables that raise a SkipRecipe exception are intentionally skipped, and should not generate warnings. [YOCTO #11319] (Bitbake rev: edf6e6094a9f7ad4b2ba06eef8fd34756edbedce) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-diffsigs: colourise outputPaul Eggleton2017-04-101-28/+73
| | | | | | | | | | | | | | | | | | If the output is a TTY, add colour to the output in order to make it easier to read. At the moment this is fairly basic, just add colour to the "titles" of each change and to the diff output. I tried to introduce this without changing the code too much - rather than moving everything over to the new python formatting style, I've introduced a color_format() function which takes care of the colour formatting, either accepting additional format arguments or alternatively leaving the caller to use the old-style formatting (%) to insert values. (Bitbake rev: 04a023c8fdea1e1812fcdcaf00345aab59f9abe1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: show word-diff for single-line values containing spacesPaul Eggleton2017-04-101-1/+37
| | | | | | | | | | | | | | | | If a variable value has changed and either the new or old value contains spaces, a word diff should be appropriate and may be a bit more readable. Import the "simplediff" module and use it to show a word diff (in the style of GNU wdiff and git diff --word-diff). Also use a similar style diff to show changes in the runtaskhashes list. I didn't use an actual word-diff here since it's a little different - we can be sure that the list is a list and not simply a free-format string. (Bitbake rev: 20db6b6553c80e18afc4f43dc2495435f7477822) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: add collapsed mode to compare_sigfiles()Paul Eggleton2017-04-101-23/+28
| | | | | | | | | | | | | | If we just want to drill down to the actual differences then we don't need to see certain things in the output, e.g. basehash changing or the signature of dependent tasks. This will be used for comparing signatures within buildhistory-diff in OE-Core; the default mode as used by bitbake-diffsigs and bitbake -S printdiff remains unchanged for the moment. (Bitbake rev: 6543a59b1ebd3194a7c6421cffc66ebe31a67c62) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: don't show unchanged runtaskdeps listPaul Eggleton2017-04-101-2/+6
| | | | | | | | | | | | If the runtaskdeps list hasn't actually changed (but the signatures of some of the tasks did) then it doesn't make sense to print out the old and new lists as they are both the same and may be very long, e.g. for do_rootfs in OE. (Bitbake rev: cb170543605288b3e8badfac3a54c588f4c95413) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: show a diff when dumping changes to multi-line valuesPaul Eggleton2017-04-101-1/+11
| | | | | | | | | | | | | | | | | | When dumping changes to signatures e.g. output of bitbake -s printdiff, if for example a function has changed, it's much more readable to see a unified diff of the changes rather than just printing the old function followed by the new function, so use difflib to do that. Note: I elected to keep to one item in the returned list per change, rather than one line per line of output, so that the caller can still look at changes individually if needed. Thus I've added some handling to bitbake-diffsigs to split the change into lines so that each line is displayed indented. (Bitbake rev: 4d254ae63a35231c98e3f73f669b040ed1144042) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/siggen: add missing path separator to cleaned pathsPaul Eggleton2017-04-101-1/+1
| | | | | | | | | | Printing "pbzip2pbzip2_1.1.13.bb" is ugly, we need to add a separating slash so that we get "pbzip2/pbzip2_1.1.13.bb" instead. (Bitbake rev: 55cd4045a37afc954f4d5091f524756b266064fc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: use multiple processes to dump signatures.Jianxun Zhang2017-01-171-2/+2
| | | | | | | | | | | | | | | | | | | | | This change significantly shortens the time on reparsing stage of '-S' option. Each file is reparsed and then dumped within a dedicated process. The maximum number of the running processes is not greater than the value of BB_NUMBER_PARSE_THREADS if it is set. The dump_sigs() in class SignatureGeneratorBasic is _replaced_ by a new dump_sigfn() interface, so calls from the outside and subclasses are dispatched to the implementation in the base class of SignatureGeneratorBasic. Fixes [YOCTO #10352] (Bitbake rev: 99d3703edd77a21770b366c6ad65a3c0f5183493) Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix clean_basepath to work with multiconfigRichard Purdie2016-12-221-0/+5
| | | | | | | | | | Tasknames can now start with "multiconfig:" which broke the virtual: comparison code and lead to unpredictable checksums with nativesdk recipes. This adds in handling for the new additional prefix which unbreaks nativesdk builds when using multiconfig. (Bitbake rev: 4da296c0c93391ec37f8b6e2245071657759771e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar calls (take 2)Joshua Lock2016-12-161-1/+1
| | | | | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (a follow on patch to fix up a few recent introductions) (Bitbake rev: f9961fd5beb31d5ab9656a5be59f7ab3effef2f0) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: add means of ignoring basehash mismatchPaul Eggleton2016-12-141-1/+2
| | | | | | | | | | | | If you run the setVariable command to set variables then you end up causing the basehash to not match the previously computed values, which triggers error messages. These mismatches are expected, so add a means of disabling them. (Bitbake rev: 5a80c0e210f26526afbe8f266b7b1a9c03334967) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-7/+7
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure taskhash mismatches don't override existing dataRichard Purdie2016-11-041-9/+9
| | | | | | | | | | | | We recalculate the taskhash to ensure the version we have matches what we think it should be. When we write out a sigdata file, use the calculated value so that we don't overwrite any existing file. This leaves any original taskhash sigdata file intact to allow a debugging comparison. (Bitbake rev: 291353b711670ce2da3d45617fc96520bdf09d3f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Pass basehash to worker processes and sanity check ↵Richard Purdie2016-11-041-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | reparsing result Bitbake can parse metadata in the cooker and in the worker during builds. If the metadata isn't deterministic, it can change between these two parses and this confuses things a lot. It turns out to be hard to debug these issues currently. This patch ensures the basehashes from the original parsing are passed into the workers and that these are checked when reparsing for consistency. The user is shown an error message if inconsistencies are found. There is debug code in siggen.py (see the "Slow but can be useful for debugging mismatched basehashes" commented code), we don't enable this by default due to performance issues. If you run into this message, enable this code and you will find "sigbasedata" files in tmp/stamps which should correspond to the hashes shown in this error message. bitbake-diffsigs on the files should show which variables are changing. (Bitbake rev: 857829048c14338132784326ba98a71f12192db8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix file variable typo in compare_sigfilesJonathan Liu2016-08-201-1/+1
| | | | | | | (Bitbake rev: deab9a30987b225922490ca186c5307c15d45b82) Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Initial multi-config supportRichard Purdie2016-08-181-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the notion of supporting multiple configurations within a single build. To enable it, set a line in local.conf like: BBMULTICONFIG = "configA configB configC" This would tell bitbake that before it parses the base configuration, it should load conf/configA.conf and so on for each different configuration. These would contain lines like: MACHINE = "A" or other variables which can be set which can be built in the same build directory (or change TMPDIR not to conflict). One downside I've already discovered is that if we want to inherit this file right at the start of parsing, the only place you can put the configurations is in "cwd", since BBPATH isn't constructed until the layers are parsed and therefore using it as a preconf file isn't possible unless its located there. Execution of these targets takes the form "bitbake multiconfig:configA:core-image-minimal core-image-sato" so similar to our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND. Implementation wise, the implication is that instead of tasks being uniquely referenced with "recipename/fn:task" it now needs to be "configuration:recipename:task". We already started using "virtual" filenames for recipes when we implemented BBCLASSEXTEND and this patch adds a new prefix to these, "multiconfig:<configname>:" and hence avoid changes to a large part of the codebase thanks to this. databuilder has an internal array of data stores and uses the right one depending on the supplied virtual filename. That trick allows us to use the existing parsing code including the multithreading mostly unchanged as well as most of the cache code. For recipecache, we end up with a dict of these accessed by multiconfig (mc). taskdata and runqueue can only cope with one recipecache so for taskdata, we pass in each recipecache and have it compute the result and end up with an array of taskdatas. We can only have one runqueue so there extensive changes there. This initial implementation has some drawbacks: a) There are no inter-multi-configuration dependencies as yet b) There are no sstate optimisations. This means if the build uses the same object twice in say two different TMPDIRs, it will either load from an existing sstate cache at the start or build it twice. We can then in due course look at ways in which it would only build it once and then reuse it. This will likely need significant changes to the way sstate currently works to make that possible. (Bitbake rev: 5287991691578825c847bac2368e9b51c0ede3f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: properly close files rather than opening them inlinePaul Eggleton2016-08-181-6/+9
| | | | | | | | | | If you don't do this, with Python 3 you get a warning on exit under some circumstances. (Bitbake rev: 49502685df3e616023df352823156381b1f79cd3) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix typoUlrich Ölmann2016-08-111-2/+2
| | | | | | | (Bitbake rev: 40f605199fb3ec2549611508b7576c64d735b2b7) Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Convert to python 3Richard Purdie2016-06-021-12/+7
| | | | | | | | | Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fixes to handle sigdata/siginfo files only containing ↵Richard Purdie2016-05-171-1/+10
| | | | | | | | | | | | | basehash data The signature data file comparison functions are meant to be able to handle data files containing just the base hash data. This had regressed in some places so add fixes to allow these comparisons to be made. The runtime components in the data files are optional. (Bitbake rev: 2a6659fd748e255a02c2f9d047829d6edfe65317) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Update logger.warn() -> logger.warning()Richard Purdie2016-05-111-1/+1
| | | | | | | | | | python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure tainted stamps are accounted for with writing custom ↵Richard Purdie2016-04-031-3/+7
| | | | | | | | | | | stamps sstate.bbclass for example writes siginfo files to a separate location but we need to read taint data from the standard path. (Bitbake rev: da444c9761ee15a59ea8880e3f812a5d3f1a1aaa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix nostamp taint handlingRichard Purdie2016-04-031-3/+7
| | | | | | | | | | | The taint values need to be passed from the server to the workers to ensure they see the same stamp values. Also ensure that the "nostamp:" prefix isn't included in the checksum value to match the server calculation. This ensures the checksums are all consistent. (Bitbake rev: f80ba20e90f3746f7faee3e0ff7f249025fec8ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add checksum recalculation/checking codeRichard Purdie2016-04-031-11/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | In theory all the information to recalcuate the task signatures was written into the siginfo/sigdata files. In reality, some of the information was written into the filename. Firstly this patch duplicates that info into the file itself just for easy of use since its small. Secondly, we abstract out the existing "calculate the checksum" code for the taskhash, and add a function to calculate the bashhash based on the informaiton within the file. Finally, we call these functions when we're writing out the data to check that the data we're writing is consistent. I've found a couple of places it wasn't and its good to know about these in advance, rather than having a siginfo/sigdata file which a given hash in its filename but a contents which give a different result. This should all combine to avoid a certain class of checksum bugs making it into world, and identifying problems in advance. (Bitbake rev: 0f50a18d7a0ea0d68edd8e5217e29111f4b1ea0b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix check calculation problem with file_checksumsRichard Purdie2016-04-031-3/+4
| | | | | | | | | | | | When I enabled debugging of the checksum code, I found the value calculated from siginfo/sigdata files for do_fetch tasks never matched. This was due to an error in the way the data was being stored for these, it wasn't ordered correctly. This patch fixes things so the checksums calculated from siginfo/sigdata files is correct when file checksums are present. (Bitbake rev: 046c1be7594fae2eec3d1f242ba3e9a85f1a1880) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Drop misleading duplicate methodRichard Purdie2016-04-031-3/+0
| | | | | | | | | The real method is a few lines later, this one is incorrect and just causing confusion. Remove it. (Bitbake rev: a896f263300f069400eae533be0daf5dedf41c95) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: SignatureGeneratorBasic: make checksum cache file configurableMarkus Lehtonen2016-02-181-3/+17
| | | | | | | | | | | Define a new bitbake configuration variable BB_HASH_CHECKSUM_CACHE_FILE that can be used to define the cache file to use for file checksum cache. (Bitbake rev: a965b390d6240e279c190b92b17c0573e9bd604c) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: SignatureGenerator: add method for saving the file checksum cacheMarkus Lehtonen2016-02-181-0/+9
| | | | | | | | | | | | Extend the API in order to be able to write out the file checksum cache onto disk. SignatureGeneratorBasic class now implements a method that update the fetcher local files checksum cache with the task file dependency checksums. (Bitbake rev: ecdabd321d48fa367b89ebffc00aa525b6eaa95c) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Change exception note into a warningRichard Purdie2016-01-061-1/+1
| | | | | | | | | This makes no sense as just a note, its at least a warning and useful to get an idea of which codepath is failing. (Bitbake rev: 0194cf0da24dc72dab0612cd54aa5190e6cd92f2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Make it clear why nostamp tasks signatures don't matchRichard Purdie2015-10-011-1/+9
| | | | | | | | | | | | If you run bitbake-diffsigs against two differing sigdata files from nostamp tasks it shows no difference despite the differing checksum. Change the code so this shows up as a nostamp 'taint' and at least makes the issue clearer to the end user. (Bitbake rev: 97679d18955dadaa34f9450564e44da99984d140) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Don't crash if number of task dependencies changeRichard Purdie2015-05-261-6/+10
| | | | | | | | | If the number of task dependencies change you currently get a traceback when using diffsigs. (Bitbake rev: c6798b431571aae18bb8699ac6e3ec75b731d719) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure taskdata default functions exist in base classRichard Purdie2015-01-231-6/+7
| | | | | | | | | | | | The get/set_taskdata functions are now part of the API of the class, ensure they exist in the base class definition so the noop handler works. [YOCTO #7233] (Bitbake rev: 9b5b1bd7d77e3f5886f6c557d3b750de1f6d6025) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache/fetch2/siggen: Ensure we track include history for file checksumsRichard Purdie2015-01-081-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if you reference a file url, its checksum is included in the task hash, however if you change to a different file at a different location, perhaps taking advantage of the FILESPATH functionality, the system will not reparse the file in question and change its checksum to match the new file. To correctly handle this, the system not only needs to know if the existing file still exists or not, but also check the existance of every file it would have looked at when computing the original file. We already do this in the bitbake parsing code for class inclusion. This change uses the same technique to log the file list we looked at and if files in these locations exist when they previously did not, to invalidate and reparse the file. Since data stored in the cache is flattened text, we have to use a string form of the data and split on the ":" character which is ugly, but is an internal detail we can improve later if a better method is found. The cache version changes to trigger a reparse since the previous cache data is now incompatible. [YOCTO #7019] (Bitbake rev: 6c0706a28d72c591f1b75b6e3f3b645859387c7e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: ensure nostamp tasks force dependent tasks to re-executePaul Eggleton2014-12-231-0/+6
| | | | | | | | | | | | | If a nostamp task is depended on by a non-nostamp task, then we want the signature of that task to change such that it re-executes afterwards. This is an unusual situation, but we want this to work in OE in externalsrc.bbclass so that compilation happens every time it is requested. (Bitbake rev: 73498afc3d45beede5b8f24a9acd523a1663b793) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-10 05:58:06 +0000