summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: lib/bb/build: enable access to logger within tasksPaul Eggleton2016-12-142-0/+12
| | | | | | | | | | | | | | | | | | In certain circumstances it can be useful to get access to BitBake's logger within a task; the main example is in OpenEmbedded's image construction code where we want to be able to check the log file for errors and warnings, but we don't want to see any errors or warnings that were emitted through the logger; so we need a way to exclude those. In order to do this, pass the logger object into the task via a BB_TASK_LOGGER variable, and add a logging handler class to bb.utils that can be added to it in order to keep a list of warnings/errors that have been emitted. (Bitbake rev: f1cd6fab604f14d8686b1d783cbfe012d923ee42) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: don't change UI process signal handler on terminatePaul Eggleton2016-12-141-1/+0
| | | | | | | | | | | | | | | | | | | | | On terminating the connection to the server, we were disabling SIGINT - and this is executed on the UI side. I'm not sure whether the intention here was to undo the SIGINT disabling we did in the server, and it was just a mistake that it disabled rather than restored and it's run on the wrong side, or whether we wanted to stop the user from breaking out of the shutdown code - the commit message provides no clues either way. Regardless, we do not want to permanently disable Ctrl+C here - it's legitimate to terminate the connection to the server and then re-establish it within the same process; at least currently, devtool modify by virtue of using tinfoil in two separate parts of the code does this, and the result of this disabling is that during the second tinfoil usage we can potentially be parsing all recipes without the ability to easily interrupt the process. (Bitbake rev: 58c60a951229dcbd8253863fb24228d046c23f6e) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: add means of ignoring basehash mismatchPaul Eggleton2016-12-141-1/+2
| | | | | | | | | | | | If you run the setVariable command to set variables then you end up causing the basehash to not match the previously computed values, which triggers error messages. These mismatches are expected, so add a means of disabling them. (Bitbake rev: 5a80c0e210f26526afbe8f266b7b1a9c03334967) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: enable setVariable command to affect task executionPaul Eggleton2016-12-143-0/+3
| | | | | | | | | | | | Allow the client to set variables with the setVariable command and have those changes take effect when running tasks. This is accomplished by collecting changes made by setVariable separately and pass these to the worker so it can be applied on top of the datastore it creates. (Bitbake rev: 69a3cd790da35c3898a8f50c284ad1a4677682a4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: support serialisationPaul Eggleton2016-12-142-0/+36
| | | | | | | | | | | | | The COW object used within VariableHistory can't be serialised itself, but we can convert it to a dict when serializing and then back when deserialising. This finally allows DataSmart objects to be serialized. NOTE: "serialisation" here means pickling, not over XMLRPC or any other transport. (Bitbake rev: bbbb2a53d5decf3b613a92c4ff77c84bfc5d4903) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: allow buildFile warning to be hidden programmaticallyPaul Eggleton2016-12-143-7/+12
| | | | | | | | | | If we want to use this function/command internally, we don't want this warning shown. (Bitbake rev: 5cfbb60833e7b12d698c1c2970c17ccf2a4971bf) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: pass datastore to server when expanding python referencesPaul Eggleton2016-12-144-11/+18
| | | | | | | | | | | | | | | | | | | | If you're expanding a value that refers to the value of a variable in python code, we need to ensure that the datastore that gets used to get the value of that variable is the client-side datastore and not just the part of it that's on the server side. For example, suppose you are in client code doing the following: d.setVar('HELLO', 'there') result = d.expand('${@d.getVar("HELLO", True)}') result should be "there" but if the client part wasn't taken into account, it would be whatever value HELLO had in the server portion of the datastore (if any). (Bitbake rev: cbc22a0a9aadc8606b927dbac0f1407ec2736b35) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: implement server-side recipe parsingPaul Eggleton2016-12-142-17/+81
| | | | | | | | | | | | | It's not really practical for us to parse recipes on the client side, we need to do it on the server because that's where we have the full python environment (including any "pure" python functions defined in classes). Thus, add some functions to tinfoil do this including a few shortcut functions. (Bitbake rev: 8f635815d191c9d848a92d51fdbf5e9fd3da1727) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: remotedata: enable transporting datastore from the client to the serverPaul Eggleton2016-12-141-0/+42
| | | | | | | | | | | | | | | | | | | | | | For the purposes of server-side parsing and expansion allowing for client-side use of the datastore, we need a means of sending a datastore from the client back to the server, where the datastore probably consists of a remote (server-side) original plus some client-side modifications. To do this we need to take care of a couple of things: 1) xmlrpc can't handle nested dicts, so if you enable memres and simply try passing a serialised datastore then things break. Instead of serialising the entire datastore, just take the naive option of transferring the internal dict alone (as a list of tuples) for now. 2) Change the TinfoilDataStoreConnector object into simply the handle (number) when transmitting; it gets substituted with the real datastore when the server receives it. (Bitbake rev: 784d2f1a024efe632fc9049ce5b78692d419d938) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: rewrite as a wrapper around the UIPaul Eggleton2016-12-145-97/+659
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rewrite tinfoil as a wrapper around the UI, instead of the earlier approach of starting up just enough of cooker to do what we want. This has several advantages: * It now works when bitbake is memory-resident instead of failing with "ERROR: Only one copy of bitbake should be run against a build directory". * We can now connect an actual UI, thus you get things like the recipe parsing / cache loading progress bar and parse error handling for free * We can now handle events generated by the server if we wish to do so * We can potentially extend this to do more stuff, e.g. actually running build operations - this needs to be made more practical before we can use it though (since you effectively have to become the UI yourself for this at the moment.) The downside is that tinfoil no longer has direct access to cooker, the global datastore, or the cache. To mitigate this I have extended data_smart to provide remote access capability for the datastore, and created "fake" cooker and cooker.recipecache / cooker.collection adapter objects in order to avoid breaking too many tinfoil-using scripts that might be out there (we've never officially documented tinfoil or BitBake's internal code, but we can still make accommodations where practical). I've at least gone far enough to support all of the utilities that use tinfoil in OE-Core with some changes, but I know there are scripts such as Chris Larson's "bb" out there that do make other calls into BitBake code that I'm not currently providing access to through the adapters. Part of the fix for [YOCTO #5470]. (Bitbake rev: 3bbf8d611c859f74d563778115677a04f5c4ab43) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: provide a means to shut down from the client in memres modePaul Eggleton2016-12-143-0/+16
| | | | | | | | | | In memory resident mode we don't really want to actually shut down since it's only the client going away. (Bitbake rev: 74db369c46043116359101cab70486afd82372c0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: implement remote datastore functionalityPaul Eggleton2016-12-142-6/+72
| | | | | | | | | | | | | | | | | This allows you to maintain a local reference to a remote datastore. The actual implementation of the remote connection is delegated to a connector object that the caller must define and supply. There is support for getting variable values and expanding python references (i.e. ${@...} remotely, however setting variables remotely is not supported - any variable setting is done locally as if the datastore were a copy (which it kind of is). Loosely based on an earlier prototype implementation by Qing He. (Bitbake rev: a3edc3eefa2d03c4ad5d12187b32fa4dc495082a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/xmlrpc: send back 503 response with correct encodingPaul Eggleton2016-12-141-1/+1
| | | | | | | | | | | | If you send back a string here you get "TypeError: 'str' does not support the buffer interface" errors in bitbake-cookerdaemon.log and "IncompleteRead(0 bytes read, 22 more expected)" errors on the client side. (Bitbake rev: 0d659a7dfe5fb096f8aa4380320f9e2a464b3cb5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: fix --observe-only optionPaul Eggleton2016-12-141-3/+7
| | | | | | | | | | | If we're in observe-only mode then we cannot run commands that would affect the server's state, including getSetVariable, so prevent that from being called in observe-only mode. (Bitbake rev: 2c5a8661430edebff67ab4a108995033d182b5d6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: make quiet option a level optionPaul Eggleton2016-12-142-9/+30
| | | | | | | | | | | | | | | | | Allow you to specify -q / --quiet more than once to reduce the messages even further. It will now operate as follows: Level Option Result ----- ------ ---------------------------------------- 0 Print usual output 1 -q Only show progress and warnings or above 2 -qq Only show warnings or above 3+ -qqq Only show errors (Bitbake rev: 6cf2582e17c28ca04f5cfb59858c4a9778c700d4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: fix resetting of reference on variablehistoryPaul Eggleton2016-12-141-1/+1
| | | | | | | | | | | | | There is no "datasmart" member, only dataroot. This dates back to the original implementation of variable history support - it's surprising we haven't noticed the issue until now, but I guess it's rare to change a copy of a datastore in a manner which using the old reference would cause an issue. (Bitbake rev: febd5534b07edfdef15cedb0578730c582c7373f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle inofity queue overflows more gracefullyRichard Purdie2016-12-081-0/+8
| | | | | | | | | | | If many files change and the inotify queue overflows, rather than print a traceback, invalidate the caches and warn the user. [YOCTO #10676] (Bitbake rev: 058f8517c041b80e8b591ad7d34a68281b2d03fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Fix world taskgraph generation issueRichard Purdie2016-12-081-4/+10
| | | | | | | | | | | | | | The processing of the "do_" prefix to tasks is currently inconsistent and has resulted in "bitbake world -g" being broken as task prefixes don't get handled correctly. Make the "do_" task prefix handling consistent through various codepaths. [YOCTO #10651] (Bitbake rev: 3d7186353e804c9410096c408bc337a98c8b33fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Avoid traceback errorsRichard Purdie2016-12-081-1/+1
| | | | | | | | | | Avoid errors like: ERROR: Exception handler error: 'NoneType' object has no attribute 'decode' (Bitbake rev: 1aeb45abe56061f044c2347889c191d5256ff21f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Send BB_TASKDEPDATA for setscene tasksRichard Purdie2016-12-081-2/+42
| | | | | | | | | | We now have code in OE that needs BB_TASKDEPDATA for setscene tasks. Therefore generate and send this data. In this case its a "pre collapsed" tree but that is fine for the use cases in question. (Bitbake rev: 38b857d086af43af6ea3aa60d3876a2c9b225401) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Add the taskhash to BB_TASKDEPDATARichard Purdie2016-12-081-1/+2
| | | | | | | | | | Its useful to know the task hash in code using TASKDEPDATA so add this data to the data structure. The recipe specific sysroots in OE need this data. (Bitbake rev: 758867e8dc74283bb1f031e158ec54cefdd5c2a6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: convert type which needs to be marshalledSujith H2016-12-071-1/+1
| | | | | | | | | | | | | | We assume that the value taken by variable v can be string, integer or any type which can be marshalled by xmlrpc. This change would help us to convert the non marshallable types to string. So that we don't get exception from xmlrpc. [YOCTO #10740] (Bitbake rev: efb0e47479b3526bc047112f7200087c5844bba4) Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: depexp: fix string formatting.Ismo Puustinen2016-12-071-1/+1
| | | | | | | | | The parameters to Python string formatting need to be inside a tuple. (Bitbake rev: 3c82af11b89cf251c3e56725a1eed2d3f4bd835b) Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: monitordisk: add eventPatrick Ohly2016-12-072-0/+25
| | | | | | | | | | | | | | | | | | | The current disk usage is interesting and may be worth logging over time as part of the build statistics. Instead of re-implementing the code and the configuration option (BB_DISKMON_DIRS), the information gathered by monitordisk.py is made available to buildstats.bbclass via a new event. This has pros and cons: - there is already a useful default configuration for "interesting" directories - no code duplication - on the other hand, users cannot configure recording separately from monitoring (probably not that important) (Bitbake rev: f065ac17d0031dca6309ddbff18c8792630de865) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: monitordisk.py: minor code and comment cleanupPatrick Ohly2016-12-071-9/+6
| | | | | | | | | | | | | There's no need to encode and decode the hash key as a single string, a tuple works just fine. Iterating over entries can be written more concisely. Entries in the stat results are integers, not floating point values. (Bitbake rev: 3c943e989964382c0b819d92de26a0c914ebed33) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: monitor disk space at regular time intervalsPatrick Ohly2016-12-071-5/+19
| | | | | | | | | | | | | | | | | | | | | Hooking the disk monitor into the regular heatbeat event instead of the runqueue solves two problems: - When there is just one long running task which fills up the disk, the previous approach did not notice that until after the completion of the task because _execute_runqueue() only gets called on task state changes. As a result, aborting a build did not work in this case. - When there are many short-lived tasks, disk space was getting checked very frequently. When the storage that is getting checked is on an NFS server, that can lead to noticable traffic to the server. (Bitbake rev: 4547eea26803a9cd355d8b045197bcbdbb36a9ad) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker process: fire heartbeat event at regular time intervalsPatrick Ohly2016-12-074-0/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | The intended usage is for recording current system statistics from /proc in buildstats.bbclass during a build and for improving the BB_DISKMON_DIRS implementation. All other existing hooks are less suitable because they trigger at unpredictable rates: too often can be handled by doing rate-limiting in the event handler, but not often enough (for example, when there is only one long-running task) cannot because the handler does not get called at all. The implementation of the new heartbeat event hooks into the cooker process event queue. The process already wakes up every 0.1s, which is often enough for the intentionally coarse 1s delay between heartbeats. That value was chosen to keep the overhead low while still being frequent enough for the intended usage. If necessary, BB_HEARTBEAT_EVENT can be set to a float specifying the delay in seconds between these heartbeat events. (Bitbake rev: 7cf22ea057d28c54bd98dc1ab7a43402a29ff1f5) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast: remove BBVERSIONS supportRoss Burton2016-11-301-67/+0
| | | | | | | | | | BBVERSIONS is moderately horrible and it doesn't appear to be actually used by anyone, so remove it to simplify the finalise codepaths. (Bitbake rev: 0bb188f01e396052b127e170a25246d79a6d6741) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser.py: support deeply nested tokensPatrick Ohly2016-11-301-12/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | For shell constructs like echo hello & wait $! the process_tokens() method ended up with a situation where "token" in the "name, value = token" assignment was a list of tuples and not the expected tuple, causing the assignment to fail. There were already two for loops (one in _parse_shell(), one in process_tokens()) which iterated over token lists. Apparently the actual nesting can also be deeper. Now there is just one such loop in process_token_list() which calls itself recursively when it detects that a list entry is another list. As a side effect (improvement?!) of the loop removal in _parse_shell(), the local function definitions in process_tokens() get executed less often. Fixes: [YOCTO #10668] (Bitbake rev: d18a74de9ac75ba32f84c40620ca9d47c1ef96a3) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: obey BB_ALLOWED_NETWORKS when checking network accessMarkus Lehtonen2016-11-307-12/+15
| | | | | | | | | [YOCTO #10508] (Bitbake rev: ddd3bc2d64d7240ecb6b6e4a1ae29b1faef6cc22) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: buildinfohelper Clarify log message for build historyMichael Wood2016-11-301-2/+2
| | | | | | | (Bitbake rev: 5accd6c4d1dcdf6609b4ed25c2b5e4faaf7f0909) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: buildinfohelper fix _get_layer_version_for_dependencyMichael Wood2016-11-301-12/+20
| | | | | | | | | | | | | This function is simplified by not trying to handle replacing the regex and just compiling and using it for matching. - Fix typo in logger output with undefined variable - Fix pyflake errors (Bitbake rev: ea298ece8d678889cd5bcde46e00545e9a73edb9) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: buildinfohelper Simplify layer event to toaster layer functionMichael Wood2016-11-301-41/+28
| | | | | | | | | | | | | | | | | | Simplify the layer event information to layer version object in toaster function. Previously this attempted many different methods of trying to obtain the correct layer from toaster by manipulating the data from the event or the data from the known layers to try and match them together. We speed up and simplify this process by making better use of django's orm methods and by working down the most likely matching methods in order of accuracy. [YOCTO #10220] (Bitbake rev: 6935cc06974ea94c9971ede89b6e8f0eae9c195b) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: buildinfohelper toaster-custom-images layerMichael Wood2016-11-301-6/+1
| | | | | | | | | | | | | | | | | | | This fixes the unidentified layers issue by making the toaster-custom-images layer a local layer. By doing this we also fix the git assumptions made for the local layers which stop recipes and other meta data being associated with them. This also removed some of the special casing previously needed when we didn't have the concept of a local (non git) layer. Also rename created flag var to a have a different var for each returned value so that the same value isn't used multiple times. [YOCTO #10220] (Bitbake rev: ba5332d4960d7f4f79aef63136796e2fa67284e3) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-308-15/+15
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-3033-257/+257
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: COW.py: fix sample codesKai Kang2016-11-231-12/+12
| | | | | | | | | | | | | | The call of methods iteritems() and itervalues() in sample codes were replaced by items() and values() to convert to Python 3 by Bitbake rev d0f904d407f57998419bd9c305ce53e5eaa36b24. But the methods iteritems() and itervalues() belong to class COWDictMeta not class dict or set. The modifications should not be made in purpose that it fails to run sample codes, so revert them. (Bitbake rev: d140f0ee6f301264e226914766d9f63558acfd6c) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: taskdata.py: improve handling of depends/rdependsPatrick Ohly2016-11-231-18/+15
| | | | | | | | | | | | | | | | | | | | Error handling only caught the cause where a dependency did not have any colon, but ignored the case where more than one was given. Now "pn:task:garbage" will raise an error instead of ignoring ":garbage". The error message had a misplaced line break (?) with the full stop on the next line. Indenting the explanation with a space might have been intended and is kept. split() was called three times instead of just once. Instead of improving the two instances of the code (one for 'depends', one for 'rdepends'), the common code is now in a helper function. (Bitbake rev: 063d255fdcb3f79b2d1b0badedc80384b295a3f5) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: don't reorder internal bitbake variables when ↵Patrick Ohly2016-11-231-1/+0
| | | | | | | | | | | | | | | | calculating hash Commit 260ced745 added __BBTASKS, __BBANONFUNCS, __BBHANDLERS to the data that gets hashed, but only after reordering these lists. The intention probably was to make the hash deterministic, but that's unnecessary (the content of the variables should already be deterministic) and hides potential reasons that might require re-parsing. (Bitbake rev: 3511d464f3a9d8b4334cda384b35016de69ce49e) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Default to expansion for getVar/getVarFlagsRichard Purdie2016-11-231-2/+2
| | | | | | | | | | | | | We've been building to this for a while, default to return expanded values for getVar/getVarFlags. We can then go through and remove the "True" option to many of the calls to this function, all function calls should have a default by now though since the parameter has been required for a while. (Bitbake rev: caf5bb9b7fe254bca9da077ebcb84a37d1f96dd4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Drop deprecated old style bitbake APIRichard Purdie2016-11-231-53/+0
| | | | | | | | | | | | | | | | The old style bb.data.getVar/setVar API has long since been deprecated in favour of d.getVar/setVar and friends. Now we're about to change the default expansion parameter, drop the old APIs to simplify the transition and ensure everyone is using the new style functions. Conversion is trivial if there are remaining stragglers. I've left bb.data.expand() for now since its more widely used but would make a good follow up patch series. (Bitbake rev: 1825604d46fcd29fad6cfd325f1cb1e1b457d2c9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Don't use deprecated bb.data.getVar/setVar APIRichard Purdie2016-11-239-24/+24
| | | | | | | | | | The old style bb.data.getVar/setVar API is obsolete. Most of bitbake doesn't use it but there were some pieces that escaped conversion. This patch fixes the remaining users mostly in the fetchers. (Bitbake rev: ff7892fa808116acc1ac50effa023a4cb031a5fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npm: conditionally hide NPM_LOCKDOWN / NPM_SHRINKWRAP warningsPaul Eggleton2016-11-231-8/+14
| | | | | | | | | | | | | | | | | | If ud.ignore_checksums is set (which we currently use to suppress the warnings for missing SRC_URI checksums when fetching files from scripts), then if we're fetching an npm package we should similarly suppress the warnings when NPM_LOCKDOWN and NPM_SHRINKWRAP aren't set. At the same time, make any errors reading either of these files actual errors since if the file is specified and could not be found, that should be an error - not the exact same warning. Fixes [YOCTO #10464]. (Bitbake rev: cefb8c93c8299e68352cf7ec5ad9ca50c0d499ed) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/cooker.py: fix for BBFILE_PATTERN = ""Robert Yang2016-11-041-0/+4
| | | | | | | | | | | | | | | | | | | There would be error when BBFILE_PATTERN = None: BBFILE_PATTERN_foo not defined This is the correct behaviour, but when the layer sets BBFILE_PATTERN = "", it would match all the remaining recipes, and cause "No bb files matched BBFILE_PATTERN" warnings for all the layers which behind it. When a layer sets BBFILE_PATTERN = "" (for example, a layer only provides git repos and source tarballs), now it means has no recipes. This is different from BBFILE_PATTERN_IGNORE_EMPTY, the later one means that it *may* not have any recipes. (Bitbake rev: 91c3b34625fac2a0f093a4b46a46e89f813e7972) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toasterui.py: warn if buildstats is missingOlaf Mandel2016-11-041-0/+3
| | | | | | | | | | Like for buildhistory, warn if buildstats is missing from INHERIT. CC: Joshua Lock <joshua.g.lock@linux.intel.com> (Bitbake rev: 3570a8cf94354c8ab07513c304ebae33623fea33) Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: buildinfohelper Handle regex pathsbrian avery2016-11-041-1/+26
| | | | | | | | | | | | | | We were presuming that all the layer dependency information was of the form "^/path/to/layer" to we were just stripping the leading "^" off of the layer information when we were matching the layer priorities to the toaster database. This patch splits out the priorities layer match which gets a regex from the task/recipe match which is gets a path. (Bitbake rev: e23b574fe52f416184ee43838b8ab28b5b8eb71d) Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: fix exception handling in exported_vars()Markus Lehtonen2016-11-041-3/+5
| | | | | | | | | | | | | | | Fix a bug where a totally wrong value of a variable would be exported if an exception happened during d.getVar(). Also, print a warning if an exception happends instead of silently ignoring it. It would probably be best just to raise the exception, instead, but use the warning for now in order to avoid breaking existing builds. [YOCTO #10393] (Bitbake rev: f639f06cfa280adcc25438387567966271b9b2c3) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Ensure taskhash mismatches don't override existing dataRichard Purdie2016-11-041-9/+9
| | | | | | | | | | | | We recalculate the taskhash to ensure the version we have matches what we think it should be. When we write out a sigdata file, use the calculated value so that we don't overwrite any existing file. This leaves any original taskhash sigdata file intact to allow a debugging comparison. (Bitbake rev: 291353b711670ce2da3d45617fc96520bdf09d3f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Pass basehash to worker processes and sanity check ↵Richard Purdie2016-11-041-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | reparsing result Bitbake can parse metadata in the cooker and in the worker during builds. If the metadata isn't deterministic, it can change between these two parses and this confuses things a lot. It turns out to be hard to debug these issues currently. This patch ensures the basehashes from the original parsing are passed into the workers and that these are checked when reparsing for consistency. The user is shown an error message if inconsistencies are found. There is debug code in siggen.py (see the "Slow but can be useful for debugging mismatched basehashes" commented code), we don't enable this by default due to performance issues. If you run into this message, enable this code and you will find "sigbasedata" files in tmp/stamps which should correspond to the hashes shown in this error message. bitbake-diffsigs on the files should show which variables are changing. (Bitbake rev: 857829048c14338132784326ba98a71f12192db8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Ensure we preserve sigbasedata files as well as sigdata onesRichard Purdie2016-11-041-1/+1
| | | | | | | | | We don't remove sigdata files, we also shouldn't remove sigbasedata files as this hinders debugging. (Bitbake rev: 8b879fd81fdcf86645cfabad0f54454ba573df52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-09-21 21:32:33 +0000