summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/data_smart.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: data_smart: Restrict expansion regexp to not include : charactersRichard Purdie2016-04-091-1/+1
| | | | | | | | | | | | Bitbake variables don't include ":" characters so exclude these from the variable expansion regexp. This assists when parsing shell code which does A=${B:-C} as we don't want a dependency on a variable called "B:-C". (Bitbake rev: 14ed41a292123374d94f5c786a619881f2ddea42) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Drop default expand=False to getVarFlag [API change]Richard Purdie2016-02-181-1/+1
| | | | | | | | | | | | | | | | At some point in the future, getVarFlag should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: aa3faebdf6af66ab34f74d328b2113de0b08c7ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Drop default expand=False to getVar [API change]Richard Purdie2016-02-181-1/+1
| | | | | | | | | | | | | | | | At some point in the future, getVar should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: fab717d303df0bcef737661f6917f275f35215a4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Avoid expanding anonymous python functionsRichard Purdie2016-02-101-1/+1
| | | | | | | | | We don't expand anonymous python before execution, so nor should we do this when calculating checksums for them. (Bitbake rev: 5f10987edda35b08970a6dd6ccf9febad271ce3e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: Add expansion parameter to getVarFlagRichard Purdie2016-02-041-4/+4
| | | | | | | | | | | | | | | This sets the scene for removing the default False for expansion from getVarFlag. This would later allow True to become the expand default. On the most part this is an automatic translation with: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` There should be no functional change from this patch. (Bitbake rev: 7c3b99c6a716095af3ffce0b15110e91fb49c913) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add missing expand parameter to getVar callRichard Purdie2016-02-041-1/+1
| | | | | | (Bitbake rev: 2876019e696ff2af164961d5d4c8e3ea9dfefc23) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Don't show exceptions for EOL literalsRichard Purdie2016-01-221-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If variables are unset, the code simply doesn't expand them, there aren't errors. If the code is a python expression, this can get a bit messy, see the attached test case. The python expansion code sees the } of the unexpanded value rather than the close of the python expression and then raises a SyntaxError exception. Ideally, we'd update the code to match pairs of brackets. I don't know how to do that with the current regex and this is unfortunately a performance sensitive piece of code. We also run the risk of breaking existing code in OE-Core where there are "{" characters but not "}" to close them (PKGE and PE). Rather than raising the exception, matching the existing "just return the expression" behaviour seems more consistent with the standard variable behaviour. This addresses an issue found in the recent image.bbclass code where there are some variables we choose not to expand (TMPDIR/DATETIME). This patch also adds a test case for this behaviour. It wouldn't preclude improved bracket matching code in the future either. (Bitbake rev: d80d39e73223a50fda0090784303d2c57167bb4c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Only support lowercase OVERRIDESRichard Purdie2015-11-161-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our current OVERRIDES handling means we end up caching and checking for a lot of possible override combinations which turn out to very unlikely. A typical example is the SRC_URI variable where we have to check if "URI" is an override. Having spent many hours working in this code, I've realised all the actual overrides we use are lower case and our standard variables are mostly uppercase. This means we could gain quite some speed advantage if we write this into the code, that overrides only consist of lowercase characters. This patch shows how simple this is and the resulting speed gains are significant. This is a significant change but tests show we don't appear to have any users of capitals in overrides in any OE-Core metadata. Before "time bitbake -p": real 2m4.224s user 7m32.312s sys 0m7.116s After "time bitbake -p": real 1m26.009s user 5m10.484s sys 0m4.640s This check could also be made conditional however I'm not seeing a need to do that at present. (Bitbake rev: c9b9443faa76ee7366b1400a56f826f3f9dec1be) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure OVERRIDES dependencies account for contains()Richard Purdie2015-09-181-2/+6
| | | | | | | | | | | | The dependencies of OVERRIDES were not including DEFAULTTUNE in OE-Core. This is pulled in by a bb.utils.contains() reference which the override dependency tracking code wasn't accounting for. This patch ensures we do track contains references too. (Bitbake rev: f3ee534cb0560dbb5f88a0ffe01e9305bae102e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Correctly handle OVERRIDE values set using ??=Richard Purdie2015-09-181-0/+2
| | | | | | | | | Right now, OVERRIDES dependency variables set using ??=, e.g. TARGET_ARCH in OE-Core don't have their dependencies tracked. This is a bug, fix it. (Bitbake rev: 944734503768f9e9223ef041f2d7873455418a54) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: When considering OVERRIDE dependencies, do so recursivelyRichard Purdie2015-09-181-1/+7
| | | | | | | | | | | Sadly its not enough to consider the dependencies of OVERRIDES, we need to resolve their dependencies and so on recursively. If we don't do this, some variable can be changed and the resulting data store is incorrect. (Bitbake rev: 82143ac064d391300e762ba7520ef1f8df18b574) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Expand overrides cache recursivelyRichard Purdie2015-09-181-3/+12
| | | | | | | | | | | | | | | | | | | | If the values that make up OVERRIDES are themselves overridden, we end up into some horrible circular logic. Unfortunately some metadata does depend on this functionality. e.g: DEFAULTTUNE_virtclass-multilib-xxx = Y which changes TUNE_ARCH which changes TARGET_ARCH which changes OVERRIDES As a solution, we iterate override expansion until the values don't change. If we iterate more than 5 times we abort and tell the user to report the issue. (Bitbake rev: 10279697c701e01bf6fdd5e9f92792ef5134807b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Separate out update_overridevars into separate functionRichard Purdie2015-09-181-5/+6
| | | | | | | | | No code changes, just refactoring to allow for functionality changes by moving things to a separate function. (Bitbake rev: 2eb934814179ccf42e3d424dabe26b17d013a7ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.data_smart: retain traceback from ExpansionErrorChristopher Larson2015-09-041-1/+2
| | | | | | | | | | This gives us the needed context of the original ExpansionError, which is invaluable when we have a chain of function calls in the expansion. (Bitbake rev: c514b6fbea77ede1b7871b89592a33ed39b1d71c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-layers: fix mapping files to layersPaul Eggleton2015-08-191-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bitbake-layers needs to map recipe and class files to the layer they came from within the show-recipes and show-overlayed commands. However, it turns out that mapping a file to the layer it came from is not as trivial as it might seem. To do it properly we need to match the path to an entry in BBFILES then map that to the collection name using BBFILE_PATTERN, then map that to the actual layer using variable history. If it doesn't match any entry in BBFILES, then we can fall back to BBFILE_PATTERN (to handle classes and conf files). This fixes the layer name not showing up properly in the output of the show-recipes and show-overlayed commands for recipes in layers such as meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the priority not showing up in show-layers for such layers. As part of this I've added a function to VariableHistory which for a space-separated list variable gives you a dict mapping the items added to the files in which they were added. I've also fixed bb.utils.get_file_layer() and reduced some of the duplication by using this function in bitbake-layers. Also fixes the priority not showing up for layers such as meta-intel Fixes [YOCTO #8160]. (Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override history loggingRichard Purdie2015-07-241-10/+13
| | | | | | | | | | | | | | | | | | Calling record() for each override alteration is slow. Since we now expand overrides dynamically we don't have to record the log data at each alteration, we can instead print it directly from the existing data stores at variable history print time using the exact same data stores. This massively improves performance of the data store when parsing with bitbake -e for example, it will improve memory overhead as well. The only downside is that VariableHistory has to poke into the datastore for some of its data but that seems an acceptable tradeoff rather than double caching. (Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve performance of infer_caller_details()Richard Purdie2015-07-241-8/+17
| | | | | | | | | | | | | | | | | | | | | | As things stand now, bitbake -e (which turns on all the caller tracking) of OE-Core generates around 9.5 million stat calls which is slow and the largest single thing on the profile data. This is because infer_caller_details() calls traceback.extract_stack() which adds line contents to the traceback. This in turn calls python's internal linecache code which calls stat on every file for every callback. We don't even use that info. We only even want a single frame of the stack. Instead, open code for the pieces of information we need. Also, only obtain the stack once for both halves of the infer_caller_details() code. This reduces the number of stat calls to around 0.5 million and significantly improves parsing with bitbake -e. (Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add CoW approach for overridedataRichard Purdie2015-07-231-1/+10
| | | | | | | | | | | | | Using deepcopy() caused a major performance regression. Turns out we can work with a shallow copy as long as we force the recreation of any list that we change, effectively a poor mans CoW. This isn't too invasive and avoids the huge overhead of deepcopy so this seems like the best way to have performance and a working data store. (Bitbake rev: 32dff4749c32f32e947c42c86f8357b8b607354b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Use deepcopy for overridedataRichard Purdie2015-07-211-1/+1
| | | | | | | | | | | | | | | dbus sets RDEPENDS_dbus-ptest_class-target = "X". nativesdk as BBCLASSEXTEND was expanding and setting various RDEPENDS variables which were clearing overrides queued against RDEPENDS. The problem was that this was leaking into other data contexts such as the target case. This was because overridedata was a shallow copy. Replacing the shallow copy with a deep copy avoids this problem of leakage between the data stores at a small performance penalty. (Bitbake rev: 83e2923cd3c8f2e25987132f85b06ce72bc941d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix appendVar/prependVarRichard Purdie2015-07-121-4/+2
| | | | | | | | | | | | | | Now that overrides get expanded 'on the fly', change appendVar and prependVar to work using _append and _prepend, else we'd have to re-implement pieces of getVar and the timing of expansions becomes problematic. Using _append/_prepend equivalence gives the behaviour users likley expect from these functions. (Bitbake rev: 40d661aaf7a563c6447b073310c5f2fdae6ca3d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix data expansion cache issuesRichard Purdie2015-07-121-1/+1
| | | | | | | | | | | The expand cache can be cleared in the middle of getVar now due to the use of operations like delVar. We therefore need to check if variables are in the cache before accessing in case it was cleared. (Bitbake rev: a97ce216cfe44136f742383542954bfce027831e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix keys() missing overridesRichard Purdie2015-07-121-0/+15
| | | | | | | | | | d.keys() was not returning items which had no base content but the variable came into existance through overrides. We have to process self.overridedata to find these other variables. (Bitbake rev: 77fc57c610d7f65f7e328ad555f525b2f8bdf1d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache set(self.overrides)Richard Purdie2015-07-121-5/+9
| | | | | | | | | | | This performs better than continually regeneratiing the set(). Also only use set comparisions when its necessary to save some overhead as issubset() is slower in the single item case. (Bitbake rev: db9a4eedcf78969ec50bf0e4a8defba8ff0daa4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override recursion handlingRichard Purdie2015-07-121-0/+6
| | | | | | | | | | When expanding OVERRIDES, its possible someone might try and override a variable that is used in OVERRIDES. This could lead to infinite recursion. Add in guards against this. (Bitbake rev: 07d773369f571028c2cf82dd1f65d9731af6d00e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Tweak OVERRIDES value cache for performanceRichard Purdie2015-07-121-4/+12
| | | | | | | | | | | | Updating the value of OVERRIDES whenever it changes turns out to be extremely expensve/pointless. Instead, clear its value and re-establish the value when we're going to use it. This gives significant speed back. (Bitbake rev: 41cf8d0c92d2d8a33fdad0921e424a0024914be1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Seperate out the override cacheRichard Purdie2015-07-121-14/+41
| | | | | | | | | | | | | Using an internal flag for override mapping turns out to be slower than is optimal, not least as we don't want the keys list to list variables that have no value other than a potential override expansion. Maintinaing a seperate cache is therefore more optimal from a speed perspective. (Bitbake rev: 1a39ccf66b30b54e1274669cb0e1dc8fc72dba49) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: VariableHistory: Ignore override duplicates and overlap ↵Richard Purdie2015-07-121-0/+5
| | | | | | | | | | | | | | with CoW functions Trying to look up a variable called 'copy' in COW is problematic due to internal implmentation details, at least avoid tracebacks from this. Also don't duplicate override history (which is an atrefact of changed override behaviour) as otherwise the bitbake -e output is convoluted. (Bitbake rev: dddff5b7b8e6c18515b43389cef35503468b843d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast/data_smart: Add parsing flag to getVar/setVarRichard Purdie2015-07-121-15/+21
| | | | | | | | | | | | | When parsing we find problems if we clear prepends/appends when setting variables during the initial parsing phases. Later, we actively want to do this (in what would be post finalisation previously). To handle this, pass a parsing flag to the operations to control the correct behaviour for the context. (Bitbake rev: ae87f5b8bf16191b3201cfb445062938eab992a0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Move override handling to getVarRichard Purdie2015-07-121-70/+61
| | | | | | | | | | | | | | Compeltely remove the replaces functionality and move all overrides handling to getVar time. We can move the cookie cache into a hidden flag within the variables themselves. This removes the need for any of the internal_finalize steps. This obsolete the need for the _seen_overrides COW cache. (Bitbake rev: 2a0b73110bede91777ada54d1d89b45fb6034b6c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache overrides and fix data storeRichard Purdie2015-07-121-8/+27
| | | | | | | | | | | | | Rather than repeatedly expanding the value of OVERRIDES, cache the value and update it when things change. There were also some bugs introduced in the replaces functionality which this approach fixes by ensuring the replaces data is updated at the correct points. (Bitbake rev: f3b7c3e054ce230ea5c2db5813390383e8dfd6db) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix cache clearance problemsRichard Purdie2015-07-121-0/+4
| | | | | | | | | These write operations should clear the expand cache since they can influence returned variable values but currently do not. Fix this. (Bitbake rev: a075332c9e13be35f1ae84adc6b29e9137a487ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve override handlingRichard Purdie2015-07-121-96/+60
| | | | | | | | | | | | | | | | | | | Rather than the heavy lifting in internal_finalize, move the bulk of the functionality to getVar and rely on a new internal replaces variable to keep track of any mappings that are needed. This removes the need for the COW _special_values cache. This change in functionality also implies we need to track any changes not only to OVERRIDES but also any variable which OVERIDES depends upon. Add code to handle this. Explicitly list FILE as a value OVERRIDES depends upon since it doesn't automatically get detected and is of key importance to OVERRIDES, otherwise PN doesn't update when FILE changes. (Bitbake rev: a6f1377ce3386d274882072d1ae6da3b1834149b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Remove need for update_data callsRichard Purdie2015-07-121-4/+15
| | | | | | | | | | | | | | Move the update_data functionality into internal data store operations so the main finalize (update_data) call is a nop. To make this work we need to call the internal finalization function whenever OVERRIDES is changed to ensure values get updated correctly. This has performance issues but the subsequant patches look into this. (Bitbake rev: 546d9932d6c4413824319a9d780c0d77d2725f4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Defer append/prepend handlingRichard Purdie2015-07-121-5/+22
| | | | | | (Bitbake rev: b1ce9975ef96f2506042832f4518cde73f6be917) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: split expanded removal values when handling _removeRoss Burton2015-02-031-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Given these assignments: TEST="a b c d" TEST_remove = "b d" TEST evaluates to "a c". However, if the _remove override is given as a variable: TEST="a b c d" FOO = "b d" TEST_remove = "${FOO} TEST evaluates to "a b c d", because when FOO is expanded it isn't split into a list. Solve this by splitting all members of removeactive once they've been expanded. [ YOCTO #7272 ] (Bitbake rev: 207013b6dde82f9654f9be996695c8335b95a288) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Don't use mutable objects as default argsRichard Purdie2015-01-291-1/+6
| | | | | | | | | | | | | These only have one instance created which means your subsequent datastores can contain echos of previous ones. Obviously this is not the behaviour we want/expect. It doesn't affect bitbake too badly as we only have one datastore, it does massively potentially break our selftests though. Thanks to Tim Amsell for pointing out the now obvious problem! (Bitbake rev: 9facf3604759b00e8fe99f929353d46f8b8ba5cb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure d.keys() doesn't list deleted variablesRichard Purdie2014-12-251-0/+4
| | | | | | | | | If you copy the datastore, then delete a variable, it still shows up in d.keys() when it should not. This patch addresses the issue. (Bitbake rev: f28ee1bb03cb32d3757fbef67c9fbe143e3dadfa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: rename defaultval to _defaultvalRoss Burton2014-12-031-3/+3
| | | | | | | | | | | | | The defaultval field is intended to be internal and the only use of that field outside of data.py is to skip over it when iterating over a value's flags. For clarity and convenience, rename the field to _defaultval so that it is considered internal and not exposed through the data API. (Bitbake rev: 2800958dadaa5c055ba21d52c98d842d360f0785) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: fix variable splitting at _remove mechanismStefan Müller-Klieser2014-11-081-1/+1
| | | | | | | | | | If we split variables only at whitespaces, a slipped in tab will render a value unremovable. (Bitbake rev: 9f171ea755644ecd9d2b3d7ed13bf8ec09ec917a) Signed-off-by: Stefan Müller-Klieser <s.mueller-klieser@phytec.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Clarify what 'computed' means in the data store history ↵Richard Purdie2014-09-161-1/+1
| | | | | | | | context (Bitbake rev: a2ca038dd1d0be4e0a0b20ae16a467d5a0075514) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix remove operator and its interaction with data expansionRichard Purdie2014-09-131-1/+2
| | | | | | | | | | | | | | | | | | | If you have: FOO = "${bindir}/X Y" FOO_remove = "${bindir}/X" the expected result is "Y". Currently this doesn't work since the removed expressions are not expanded first. This patch adjusts things so the expressions are expanded before being processed for removal. Also add a test to ensure this case continues to work. [YOCTO #6624] (Bitbake rev: 72a1ca4a104ccab73d6abcbd44db9c2636a58572) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Add SkipRecipe event to replace SkipPackageRichard Purdie2014-06-011-1/+1
| | | | | | | | | | | | In the depths of time we were rather confused about naming. bb files are recipes, the event to skip parsing them should be SkipRecipe, not SkipPackage. This changes bitbake to use the better name but leaves the other around for now. We can therefore start removing references to it from the metadata. (Bitbake rev: 98d9e6e0f514a7cb7da1d99bf4bd5602b89426d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix an unusual variable reference bugRichard Purdie2014-05-211-1/+1
| | | | | | | | | | | | | | | | | | | | If you try: Y = "" Y_remove = "X" in OE-Core, bitbake will crash with a KeyError during expansion. The reason is that no expansion of the empty value is attempted but removal from is it and hence no varparse data is present for it in the expand_cache. If the value is empty, there is nothing to remove so the best fix is simply not to check for None but check it has any value. Also add a test for this error so it doesn't get reintroduced. (Bitbake rev: af3ce0fc0280e6642fa35de400f75fdbabf329b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure all possible overrides are cached including ↵Richard Purdie2014-05-071-1/+6
| | | | | | | | | | | | | | | | | | those with '_' in the name Unfortunately we've been neglecting to pay the correct tributes to the cookie monster and hence the datastore is malfunctioning. Currently tributes are only paid on the last part of a variable after the last "_" character. We need to split by *all* "_" characters since an override may contain the character. This fixes the code so the correct number of tributes are made. Paradoxically parsing appears to be faster after this change. (Bitbake rev: d1c712fd3a59fa804e6fd451612c30487671f3a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix caching issue for double remove referencesRichard Purdie2014-03-251-0/+4
| | | | | | | | | | | | | | | | | | FOO = "foo bar" FOO_remove = "bar" FOO_FOO = "${FOO} ${FOO}" would show FOO_FOO = "foo foo bar" rather than the expected "foo foo". This is actually a cache bug, this patch ensures the right value is put into the cache. The preceeding patch adds a test case to ensure we don't regress in future. [YOCTO #6037] (Bitbake rev: 2a80735183e8faa110b4c6d8d85c4707f28e03a1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix hash corruption issueRichard Purdie2013-12-131-1/+1
| | | | | | | | | | | | | We were accidentally using references to sets in the contains functionality instead of creating a copy. This could cause data corruption and corruption of the resulting sstate checksums. This patch fixes this to make a copy of the set and resolved the corruption issue. (Bitbake rev: 8f4733257ad665aa7c7e7061c543379d5e4e3af2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser/data_smart: Optimise parsing speedRichard Purdie2013-12-031-4/+6
| | | | | | | | | | | | | | | | | | | | | The previous "contains" changes caused a ~3% parsing speed impact. Looking at the cause of those changes was interesting: * Use of defaultdict was slower than just checking for missing entries and setting them when needed. * Even the "import collections" adversely affects parsing speed * There was a missing intern function for the contains cache data * Setting up a log object for each variable has noticeable overhead due to the changes in the code paths uses, we can avoid this. * We can call getVarFlag on "_content" directly within VariableParse for a noticeable speed gain since its a seriously hot code path. This patch therefore tweaks the code based on the above observations to get some of the speed back. (Bitbake rev: fca802187a2a30686a8a07d2b6b16a3e5716e293) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/codeparser: Improve handling of contains functionsRichard Purdie2013-11-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | One of the current frustrations with the sstate checksums is that code like base_contains('X', 'y',...) adds a full dependency on X and varies depend even on whitespace changes in X. This patch adds special handling of the contains functions to expand the first parameter and check for the flag specified by the second parameter (assuming its a string). The result is then appended to the value of the variable with a "Set" or "Unset" status. If the flag is added/removed, the stored variable value changes and hence the checksum changes. No dependency on X is added so it is free to change with regard to other flags or whitespace. This code is far from ideal, ideally we'd have properly typed variables however it fixes a major annoyance of the current checksums and is of enough value its worth adding in a stopgap solution. It shouldn't significantly restrict any propely typed variable implementation in future. (Bitbake rev: ed2d0a22a80299de0cfd377999950cf4b26c512e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/hob: removing extra parameters from conf files using hobCristiana Voicu2013-10-071-2/+6
| | | | | | | | | | | | | In Hob settings, there is a tab to add/remove extra settings. This patch implements a way to "remove" variables from conf files, through bitbake. But, to keep the history assigment of the variables synchronized, instead of removing, it replaces the lines with blank lines. [YOCTO #5284] (Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Variable references don't contain newlines, spaces or tabsRichard Purdie2013-09-221-1/+1
| | | | | | | | | | | The code is happily trying to expand variable names containing newlines, spaces and tabs which are illegal characters in variable names. This patch stops it doing this. This will change dependency checksums since some rather weird dependencies were being attempted to be expanded. (Bitbake rev: 37e13b852b33d98fa40f49dc1e815b3bbe912ff0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-17 12:55:28 +0000