summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/data_smart.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: data_smart: Ensure _remove operations on newly set variables are ↵Richard Purdie2017-04-121-0/+2
| | | | | | | | | | | | | | | | cleared We clear append/prepend on newly set variables, we should also clear remove operations. If we don't do this, there is no way we can actually delete a remove operation. Bitbake internally uses parsing=True to avoid these side effects when making its own internal calls. Also add a testcase to bitbake-selftest to ensure we remain consistent going forward from here. (Bitbake rev: 3a319f079d699c870d8531e051ab65e6278d1fa5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: implement missing remote datastore operationsPaul Eggleton2017-03-221-1/+28
| | | | | | | | | | | | | | | | | | | | | | | | Enable the following operations from a remote datastore to affect the other end: * setVarFlag() * delVar() * delVarFlag() * renameVar() In practice I don't expect these to be used much, but they should be present so that the implementation is at least reasonably filled out and that the tests pass. Also add tests for the interface, mostly by subclassing the existing local test classes so that they are using a remote datastore. (These don't actually test remote usage via tinfoil, just that the datastore's interface can be used.) (Bitbake rev: 282dc0719d22a39df746eea762ebe05c66aa8f8a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: fix override handling in remote datastoresPaul Eggleton2017-03-221-7/+8
| | | | | | | | | | | | | | | | | | | There was a huge gap in the remote datastore code introduced in the tinfoil2 rework - we weren't handling overrides at all, since these are stored separately from the actual data in the DataSmart object. Thus, when a datastore actually represents a remote datastore we need to go back to that remote datastore to get the override data as well, so introduce code to do that. To avoid a second round-trip I had to modify the _findVar() function to return the override data as well. This will increase the overhead a little when that data is superfluous, but without making the function even uglier I don't think there's a way to avoid that. (Bitbake rev: 4f9d6f060ed247fb6fa2f45668a892a1788d3f91) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: Drop now unneeded update_data callsRichard Purdie2017-02-151-1/+0
| | | | | | | | | | Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. (Bitbake rev: 2300beb50333bb620013b058a7309e7f2042101d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix unneeded variable manipulationRichard Purdie2017-01-221-7/+8
| | | | | | | | | | | | | | | | | If was pointed out that if we have: XXX = " A" XXX_remove_inactive-override = "YY" then XXX can become "A" and the leading space can be removed. This is because the remove override code changes the variable value even when there is no removals active. In the process it dirties the cache. We don't really need to do this so tweak the code accordingly. (Bitbake rev: 2bc4d35fb32defc59cd6ed1fc87e35924c201a5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: support serialisationPaul Eggleton2016-12-141-0/+13
| | | | | | | | | | | | | The COW object used within VariableHistory can't be serialised itself, but we can convert it to a dict when serializing and then back when deserialising. This finally allows DataSmart objects to be serialized. NOTE: "serialisation" here means pickling, not over XMLRPC or any other transport. (Bitbake rev: bbbb2a53d5decf3b613a92c4ff77c84bfc5d4903) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: pass datastore to server when expanding python referencesPaul Eggleton2016-12-141-1/+1
| | | | | | | | | | | | | | | | | | | | If you're expanding a value that refers to the value of a variable in python code, we need to ensure that the datastore that gets used to get the value of that variable is the client-side datastore and not just the part of it that's on the server side. For example, suppose you are in client code doing the following: d.setVar('HELLO', 'there') result = d.expand('${@d.getVar("HELLO", True)}') result should be "there" but if the client part wasn't taken into account, it would be whatever value HELLO had in the server portion of the datastore (if any). (Bitbake rev: cbc22a0a9aadc8606b927dbac0f1407ec2736b35) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: implement remote datastore functionalityPaul Eggleton2016-12-141-6/+33
| | | | | | | | | | | | | | | | | This allows you to maintain a local reference to a remote datastore. The actual implementation of the remote connection is delegated to a connector object that the caller must define and supply. There is support for getting variable values and expanding python references (i.e. ${@...} remotely, however setting variables remotely is not supported - any variable setting is done locally as if the datastore were a copy (which it kind of is). Loosely based on an earlier prototype implementation by Qing He. (Bitbake rev: a3edc3eefa2d03c4ad5d12187b32fa4dc495082a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: fix resetting of reference on variablehistoryPaul Eggleton2016-12-141-1/+1
| | | | | | | | | | | | | There is no "datasmart" member, only dataroot. This dates back to the original implementation of variable history support - it's surprising we haven't noticed the issue until now, but I guess it's rare to change a copy of a datastore in a manner which using the old reference would cause an issue. (Bitbake rev: febd5534b07edfdef15cedb0578730c582c7373f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-301-2/+2
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-6/+6
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: don't reorder internal bitbake variables when ↵Patrick Ohly2016-11-231-1/+0
| | | | | | | | | | | | | | | | calculating hash Commit 260ced745 added __BBTASKS, __BBANONFUNCS, __BBHANDLERS to the data that gets hashed, but only after reordering these lists. The intention probably was to make the hash deterministic, but that's unnecessary (the content of the variables should already be deterministic) and hides potential reasons that might require re-parsing. (Bitbake rev: 3511d464f3a9d8b4334cda384b35016de69ce49e) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Default to expansion for getVar/getVarFlagsRichard Purdie2016-11-231-2/+2
| | | | | | | | | | | | | We've been building to this for a while, default to return expanded values for getVar/getVarFlags. We can then go through and remove the "True" option to many of the calls to this function, all function calls should have a default by now though since the parameter has been required for a while. (Bitbake rev: caf5bb9b7fe254bca9da077ebcb84a37d1f96dd4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart/utils: Add 'd' to the context used for better_eval in ↵Richard Purdie2016-06-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | python expansion If a line like: foo=${@' '.join([d.getVar('D', True) + x for x in (' '.join([d.getVar('FILES_bash-' + p, True) or '' for p in ['lib', 'dev', 'staticdev', 'doc', 'locale', 'ptest']])).split()])} is added to a function like do_install, it fails with Exception name 'd' is not defined. This is due to a change of behaviour in python 3 compared to python 2. Generator expressions, dict comprehensions and set comprehensions are executed in a new scope but list comprehensions in python 2.x are not. In python 3 they all use a new scope. To allow these kinds of expressions to work, the easiest approach is to add 'd' to the global context. To do this, an extra optional parameter is added to better_eval and we use that to add 'd'. (Bitbake rev: 8f74881037bb01013d3d439dc0c269909a198c1c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Simplify ExpansionError exceptionRichard Purdie2016-06-021-2/+1
| | | | | | | | | | This seemingly convoluted syntax doesn't work in python3. Instead use the chained exception handling syntax which appears to make more sense here. (Bitbake rev: b19a4c5166303b1fa680582adf63e6a5564bfb4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Convert to python 3Richard Purdie2016-06-021-2/+2
| | | | | | | | | Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.data_smart: use iter() for __len__Christopher Larson2016-05-061-1/+1
| | | | | | | | | | | It seems the frozenset constructor in pypy runs len(), so we can't pass the DataSmart instance directly to it, instead pass the iterator. Fixes pypy support. (Bitbake rev: b492836e08745e04bd9ba2fb0b56a680a5fdce79) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Restrict expansion regexp to not include : charactersRichard Purdie2016-04-091-1/+1
| | | | | | | | | | | | Bitbake variables don't include ":" characters so exclude these from the variable expansion regexp. This assists when parsing shell code which does A=${B:-C} as we don't want a dependency on a variable called "B:-C". (Bitbake rev: 14ed41a292123374d94f5c786a619881f2ddea42) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Drop default expand=False to getVarFlag [API change]Richard Purdie2016-02-181-1/+1
| | | | | | | | | | | | | | | | At some point in the future, getVarFlag should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: aa3faebdf6af66ab34f74d328b2113de0b08c7ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Drop default expand=False to getVar [API change]Richard Purdie2016-02-181-1/+1
| | | | | | | | | | | | | | | | At some point in the future, getVar should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: fab717d303df0bcef737661f6917f275f35215a4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Avoid expanding anonymous python functionsRichard Purdie2016-02-101-1/+1
| | | | | | | | | We don't expand anonymous python before execution, so nor should we do this when calculating checksums for them. (Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add expansion parameter to getVarFlagRichard Purdie2016-02-041-4/+4
| | | | | | | | | | | | | | | This sets the scene for removing the default False for expansion from getVarFlag. This would later allow True to become the expand default. On the most part this is an automatic translation with: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` There should be no functional change from this patch. (Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add missing expand parameter to getVar callRichard Purdie2016-02-041-1/+1
| | | | | | (Bitbake rev: 2876019e696ff2af164961d5d4c8e3ea9dfefc23) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Don't show exceptions for EOL literalsRichard Purdie2016-01-221-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If variables are unset, the code simply doesn't expand them, there aren't errors. If the code is a python expression, this can get a bit messy, see the attached test case. The python expansion code sees the } of the unexpanded value rather than the close of the python expression and then raises a SyntaxError exception. Ideally, we'd update the code to match pairs of brackets. I don't know how to do that with the current regex and this is unfortunately a performance sensitive piece of code. We also run the risk of breaking existing code in OE-Core where there are "{" characters but not "}" to close them (PKGE and PE). Rather than raising the exception, matching the existing "just return the expression" behaviour seems more consistent with the standard variable behaviour. This addresses an issue found in the recent image.bbclass code where there are some variables we choose not to expand (TMPDIR/DATETIME). This patch also adds a test case for this behaviour. It wouldn't preclude improved bracket matching code in the future either. (Bitbake rev: d80d39e73223a50fda0090784303d2c57167bb4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Only support lowercase OVERRIDESRichard Purdie2015-11-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our current OVERRIDES handling means we end up caching and checking for a lot of possible override combinations which turn out to very unlikely. A typical example is the SRC_URI variable where we have to check if "URI" is an override. Having spent many hours working in this code, I've realised all the actual overrides we use are lower case and our standard variables are mostly uppercase. This means we could gain quite some speed advantage if we write this into the code, that overrides only consist of lowercase characters. This patch shows how simple this is and the resulting speed gains are significant. This is a significant change but tests show we don't appear to have any users of capitals in overrides in any OE-Core metadata. Before "time bitbake -p": real 2m4.224s user 7m32.312s sys 0m7.116s After "time bitbake -p": real 1m26.009s user 5m10.484s sys 0m4.640s This check could also be made conditional however I'm not seeing a need to do that at present. (Bitbake rev: c9b9443faa76ee7366b1400a56f826f3f9dec1be) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure OVERRIDES dependencies account for contains()Richard Purdie2015-09-181-2/+6
| | | | | | | | | | | | The dependencies of OVERRIDES were not including DEFAULTTUNE in OE-Core. This is pulled in by a bb.utils.contains() reference which the override dependency tracking code wasn't accounting for. This patch ensures we do track contains references too. (Bitbake rev: f3ee534cb0560dbb5f88a0ffe01e9305bae102e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Correctly handle OVERRIDE values set using ??=Richard Purdie2015-09-181-0/+2
| | | | | | | | | Right now, OVERRIDES dependency variables set using ??=, e.g. TARGET_ARCH in OE-Core don't have their dependencies tracked. This is a bug, fix it. (Bitbake rev: 944734503768f9e9223ef041f2d7873455418a54) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: When considering OVERRIDE dependencies, do so recursivelyRichard Purdie2015-09-181-1/+7
| | | | | | | | | | | Sadly its not enough to consider the dependencies of OVERRIDES, we need to resolve their dependencies and so on recursively. If we don't do this, some variable can be changed and the resulting data store is incorrect. (Bitbake rev: 82143ac064d391300e762ba7520ef1f8df18b574) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Expand overrides cache recursivelyRichard Purdie2015-09-181-3/+12
| | | | | | | | | | | | | | | | | | | | If the values that make up OVERRIDES are themselves overridden, we end up into some horrible circular logic. Unfortunately some metadata does depend on this functionality. e.g: DEFAULTTUNE_virtclass-multilib-xxx = Y which changes TUNE_ARCH which changes TARGET_ARCH which changes OVERRIDES As a solution, we iterate override expansion until the values don't change. If we iterate more than 5 times we abort and tell the user to report the issue. (Bitbake rev: 10279697c701e01bf6fdd5e9f92792ef5134807b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Separate out update_overridevars into separate functionRichard Purdie2015-09-181-5/+6
| | | | | | | | | No code changes, just refactoring to allow for functionality changes by moving things to a separate function. (Bitbake rev: 2eb934814179ccf42e3d424dabe26b17d013a7ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.data_smart: retain traceback from ExpansionErrorChristopher Larson2015-09-041-1/+2
| | | | | | | | | | This gives us the needed context of the original ExpansionError, which is invaluable when we have a chain of function calls in the expansion. (Bitbake rev: c514b6fbea77ede1b7871b89592a33ed39b1d71c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-layers: fix mapping files to layersPaul Eggleton2015-08-191-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bitbake-layers needs to map recipe and class files to the layer they came from within the show-recipes and show-overlayed commands. However, it turns out that mapping a file to the layer it came from is not as trivial as it might seem. To do it properly we need to match the path to an entry in BBFILES then map that to the collection name using BBFILE_PATTERN, then map that to the actual layer using variable history. If it doesn't match any entry in BBFILES, then we can fall back to BBFILE_PATTERN (to handle classes and conf files). This fixes the layer name not showing up properly in the output of the show-recipes and show-overlayed commands for recipes in layers such as meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the priority not showing up in show-layers for such layers. As part of this I've added a function to VariableHistory which for a space-separated list variable gives you a dict mapping the items added to the files in which they were added. I've also fixed bb.utils.get_file_layer() and reduced some of the duplication by using this function in bitbake-layers. Also fixes the priority not showing up for layers such as meta-intel Fixes [YOCTO #8160]. (Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override history loggingRichard Purdie2015-07-241-10/+13
| | | | | | | | | | | | | | | | | | Calling record() for each override alteration is slow. Since we now expand overrides dynamically we don't have to record the log data at each alteration, we can instead print it directly from the existing data stores at variable history print time using the exact same data stores. This massively improves performance of the data store when parsing with bitbake -e for example, it will improve memory overhead as well. The only downside is that VariableHistory has to poke into the datastore for some of its data but that seems an acceptable tradeoff rather than double caching. (Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve performance of infer_caller_details()Richard Purdie2015-07-241-8/+17
| | | | | | | | | | | | | | | | | | | | | | As things stand now, bitbake -e (which turns on all the caller tracking) of OE-Core generates around 9.5 million stat calls which is slow and the largest single thing on the profile data. This is because infer_caller_details() calls traceback.extract_stack() which adds line contents to the traceback. This in turn calls python's internal linecache code which calls stat on every file for every callback. We don't even use that info. We only even want a single frame of the stack. Instead, open code for the pieces of information we need. Also, only obtain the stack once for both halves of the infer_caller_details() code. This reduces the number of stat calls to around 0.5 million and significantly improves parsing with bitbake -e. (Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add CoW approach for overridedataRichard Purdie2015-07-231-1/+10
| | | | | | | | | | | | | Using deepcopy() caused a major performance regression. Turns out we can work with a shallow copy as long as we force the recreation of any list that we change, effectively a poor mans CoW. This isn't too invasive and avoids the huge overhead of deepcopy so this seems like the best way to have performance and a working data store. (Bitbake rev: 32dff4749c32f32e947c42c86f8357b8b607354b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Use deepcopy for overridedataRichard Purdie2015-07-211-1/+1
| | | | | | | | | | | | | | | dbus sets RDEPENDS_dbus-ptest_class-target = "X". nativesdk as BBCLASSEXTEND was expanding and setting various RDEPENDS variables which were clearing overrides queued against RDEPENDS. The problem was that this was leaking into other data contexts such as the target case. This was because overridedata was a shallow copy. Replacing the shallow copy with a deep copy avoids this problem of leakage between the data stores at a small performance penalty. (Bitbake rev: 83e2923cd3c8f2e25987132f85b06ce72bc941d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix appendVar/prependVarRichard Purdie2015-07-121-4/+2
| | | | | | | | | | | | | | Now that overrides get expanded 'on the fly', change appendVar and prependVar to work using _append and _prepend, else we'd have to re-implement pieces of getVar and the timing of expansions becomes problematic. Using _append/_prepend equivalence gives the behaviour users likley expect from these functions. (Bitbake rev: 40d661aaf7a563c6447b073310c5f2fdae6ca3d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix data expansion cache issuesRichard Purdie2015-07-121-1/+1
| | | | | | | | | | | The expand cache can be cleared in the middle of getVar now due to the use of operations like delVar. We therefore need to check if variables are in the cache before accessing in case it was cleared. (Bitbake rev: a97ce216cfe44136f742383542954bfce027831e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix keys() missing overridesRichard Purdie2015-07-121-0/+15
| | | | | | | | | | d.keys() was not returning items which had no base content but the variable came into existance through overrides. We have to process self.overridedata to find these other variables. (Bitbake rev: 77fc57c610d7f65f7e328ad555f525b2f8bdf1d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache set(self.overrides)Richard Purdie2015-07-121-5/+9
| | | | | | | | | | | This performs better than continually regeneratiing the set(). Also only use set comparisions when its necessary to save some overhead as issubset() is slower in the single item case. (Bitbake rev: db9a4eedcf78969ec50bf0e4a8defba8ff0daa4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override recursion handlingRichard Purdie2015-07-121-0/+6
| | | | | | | | | | When expanding OVERRIDES, its possible someone might try and override a variable that is used in OVERRIDES. This could lead to infinite recursion. Add in guards against this. (Bitbake rev: 07d773369f571028c2cf82dd1f65d9731af6d00e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Tweak OVERRIDES value cache for performanceRichard Purdie2015-07-121-4/+12
| | | | | | | | | | | | Updating the value of OVERRIDES whenever it changes turns out to be extremely expensve/pointless. Instead, clear its value and re-establish the value when we're going to use it. This gives significant speed back. (Bitbake rev: 41cf8d0c92d2d8a33fdad0921e424a0024914be1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Seperate out the override cacheRichard Purdie2015-07-121-14/+41
| | | | | | | | | | | | | Using an internal flag for override mapping turns out to be slower than is optimal, not least as we don't want the keys list to list variables that have no value other than a potential override expansion. Maintinaing a seperate cache is therefore more optimal from a speed perspective. (Bitbake rev: 1a39ccf66b30b54e1274669cb0e1dc8fc72dba49) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: VariableHistory: Ignore override duplicates and overlap ↵Richard Purdie2015-07-121-0/+5
| | | | | | | | | | | | | | with CoW functions Trying to look up a variable called 'copy' in COW is problematic due to internal implmentation details, at least avoid tracebacks from this. Also don't duplicate override history (which is an atrefact of changed override behaviour) as otherwise the bitbake -e output is convoluted. (Bitbake rev: dddff5b7b8e6c18515b43389cef35503468b843d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast/data_smart: Add parsing flag to getVar/setVarRichard Purdie2015-07-121-15/+21
| | | | | | | | | | | | | When parsing we find problems if we clear prepends/appends when setting variables during the initial parsing phases. Later, we actively want to do this (in what would be post finalisation previously). To handle this, pass a parsing flag to the operations to control the correct behaviour for the context. (Bitbake rev: ae87f5b8bf16191b3201cfb445062938eab992a0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Move override handling to getVarRichard Purdie2015-07-121-70/+61
| | | | | | | | | | | | | | Compeltely remove the replaces functionality and move all overrides handling to getVar time. We can move the cookie cache into a hidden flag within the variables themselves. This removes the need for any of the internal_finalize steps. This obsolete the need for the _seen_overrides COW cache. (Bitbake rev: 2a0b73110bede91777ada54d1d89b45fb6034b6c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache overrides and fix data storeRichard Purdie2015-07-121-8/+27
| | | | | | | | | | | | | Rather than repeatedly expanding the value of OVERRIDES, cache the value and update it when things change. There were also some bugs introduced in the replaces functionality which this approach fixes by ensuring the replaces data is updated at the correct points. (Bitbake rev: f3b7c3e054ce230ea5c2db5813390383e8dfd6db) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix cache clearance problemsRichard Purdie2015-07-121-0/+4
| | | | | | | | | These write operations should clear the expand cache since they can influence returned variable values but currently do not. Fix this. (Bitbake rev: a075332c9e13be35f1ae84adc6b29e9137a487ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override handlingRichard Purdie2015-07-121-96/+60
| | | | | | | | | | | | | | | | | | | Rather than the heavy lifting in internal_finalize, move the bulk of the functionality to getVar and rely on a new internal replaces variable to keep track of any mappings that are needed. This removes the need for the COW _special_values cache. This change in functionality also implies we need to track any changes not only to OVERRIDES but also any variable which OVERIDES depends upon. Add code to handle this. Explicitly list FILE as a value OVERRIDES depends upon since it doesn't automatically get detected and is of key importance to OVERRIDES, otherwise PN doesn't update when FILE changes. (Bitbake rev: a6f1377ce3386d274882072d1ae6da3b1834149b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Remove need for update_data callsRichard Purdie2015-07-121-4/+15
| | | | | | | | | | | | | | Move the update_data functionality into internal data store operations so the main finalize (update_data) call is a nop. To make this work we need to call the internal finalization function whenever OVERRIDES is changed to ensure values get updated correctly. This has performance issues but the subsequant patches look into this. (Bitbake rev: 546d9932d6c4413824319a9d780c0d77d2725f4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-11 10:20:22 +0000