summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/data_smart.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: data_smart: Fix hash corruption issueRichard Purdie2013-12-131-1/+1
| | | | | | | | | | | | | We were accidentally using references to sets in the contains functionality instead of creating a copy. This could cause data corruption and corruption of the resulting sstate checksums. This patch fixes this to make a copy of the set and resolved the corruption issue. (Bitbake rev: 8f4733257ad665aa7c7e7061c543379d5e4e3af2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser/data_smart: Optimise parsing speedRichard Purdie2013-12-031-4/+6
| | | | | | | | | | | | | | | | | | | | | The previous "contains" changes caused a ~3% parsing speed impact. Looking at the cause of those changes was interesting: * Use of defaultdict was slower than just checking for missing entries and setting them when needed. * Even the "import collections" adversely affects parsing speed * There was a missing intern function for the contains cache data * Setting up a log object for each variable has noticeable overhead due to the changes in the code paths uses, we can avoid this. * We can call getVarFlag on "_content" directly within VariableParse for a noticeable speed gain since its a seriously hot code path. This patch therefore tweaks the code based on the above observations to get some of the speed back. (Bitbake rev: fca802187a2a30686a8a07d2b6b16a3e5716e293) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data/codeparser: Improve handling of contains functionsRichard Purdie2013-11-261-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | One of the current frustrations with the sstate checksums is that code like base_contains('X', 'y',...) adds a full dependency on X and varies depend even on whitespace changes in X. This patch adds special handling of the contains functions to expand the first parameter and check for the flag specified by the second parameter (assuming its a string). The result is then appended to the value of the variable with a "Set" or "Unset" status. If the flag is added/removed, the stored variable value changes and hence the checksum changes. No dependency on X is added so it is free to change with regard to other flags or whitespace. This code is far from ideal, ideally we'd have properly typed variables however it fixes a major annoyance of the current checksums and is of enough value its worth adding in a stopgap solution. It shouldn't significantly restrict any propely typed variable implementation in future. (Bitbake rev: ed2d0a22a80299de0cfd377999950cf4b26c512e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/hob: removing extra parameters from conf files using hobCristiana Voicu2013-10-071-2/+6
| | | | | | | | | | | | | In Hob settings, there is a tab to add/remove extra settings. This patch implements a way to "remove" variables from conf files, through bitbake. But, to keep the history assigment of the variables synchronized, instead of removing, it replaces the lines with blank lines. [YOCTO #5284] (Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Variable references don't contain newlines, spaces or tabsRichard Purdie2013-09-221-1/+1
| | | | | | | | | | | The code is happily trying to expand variable names containing newlines, spaces and tabs which are illegal characters in variable names. This patch stops it doing this. This will change dependency checksums since some rather weird dependencies were being attempted to be expanded. (Bitbake rev: 37e13b852b33d98fa40f49dc1e815b3bbe912ff0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix variable reference issuesRichard Purdie2013-09-181-4/+3
| | | | | | | | | | | | | | | | The change to use the expansion cache in VariableParse was incorrect as it was adding in references it shouldn't have been. This patch corrects the codepaths and ensures the references are correct. The cache version is bumped since the previous bug could have leave to invalid checksum calculations and a clean cache is therefore desireable. The impact of the bug was that sstate was not getting reused when it should and some tasks were also being rerun when they should not have been. (Bitbake rev: 8a42d082315bd6ce091d006bf83476db257fa48b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache the fact a variable accesses another even if its ↵Richard Purdie2013-09-171-1/+1
| | | | | | | | | | | | unset If a variable references another but it isn't set at present, the reference wasn't stored. It really should be marked as a reference and the higher level dependency code can handle as appropriate. (Bitbake rev: b05b748b2153c941b95cd36fb22aaafc4dbf3791) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Allow flags to use the expand cacheRichard Purdie2013-09-171-1/+3
| | | | | | (Bitbake rev: a0122ab80df21597291ff32ff7fbaa4de0347a6f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Allow expansion of flags in getVarFlagsRichard Purdie2013-09-171-2/+3
| | | | | | | | | | Allow a list of flags to expand to be passed into getVarFlags. This is useful within bitbake itself to optimise performance of the dependency generation code. (Bitbake rev: a3ae7efdf750fc5bb9ff5a75defbcfdab1912dbe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add explict None checksAlexandru DAMIAN2013-09-171-2/+2
| | | | | | | | | | | Simple if xxx checks end up calling len(xxx). We're interested in the specific case of None which means we can break out the iterator much earlier after the first item. This adds in the specific tests for None in what is a hot path in the data store code which gives small performance gains. (Bitbake rev: a4d81e44a7cd3dafb0bf12f7cac5ff511db18e60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Cache an list of export variablesRichard Purdie2013-09-171-0/+7
| | | | | | | | | Compute a cache of the list of potential export variables so that we don't have to compute the list from scratch. (Bitbake rev: f41f46f7eaa6889edeb3a4e4ddedc07084686c60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: use the expand_cache in VariableParseRichard Purdie2013-09-171-0/+5
| | | | | | | | | | When in VariableParse, use the expand_cache if possible rather than looking up data. Ultimately it would come from the same place but this short cuts a heavily used code block for speed improvements. (Bitbake rev: f682b8b83d21d576160bac8dc57c4c989b4dc555) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve variable expansion regexpRichard Purdie2013-09-171-1/+1
| | | | | | | | | | | Debugging showed the variable expansion regexp was catching python expressions (starting with @). Since these are caught by their own dedicated regexp, stop matching these for the plain variable expansion for small performance improvements. (Bitbake rev: c630d564285f55f9db10c18269bd310df797430e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Account for changes in append/prepend/remove in the ↵Richard Purdie2013-09-121-3/+6
| | | | | | | | | | | | | | | config hash bitbake wasn't reparsing when _remove items were added to its configuration and equally, appends/prepends were also being badly tracked. This change enrures these variables are accounted for in the configuration hash. [YOCTO #5172] (Bitbake rev: 62914f9208ef2427a34daa523af857f4027900eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Move getVar expand cache handing to fix _remove operationsRichard Purdie2013-09-061-7/+6
| | | | | | | | | | | | | DISTRO_FEATURES_remove = "opengl" wasn't working as expected. The reason turned out the be the indirect reference to opengl and the fact _remove was operating on unexpanded data. This patch rearranges some code to ensure we operate on expanded data by moving the expand cache handing into getVarFlags instead of getVar. (Bitbake rev: 181899bd9665f74f8d1b22d2453616ad30d26d9e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: allow removal of multiple words at once with _removeChristopher Larson2013-08-281-1/+1
| | | | | | | | | | FOO = "foo bar baz" FOO_remove = "foo baz" (Bitbake rev: 04127dec207d6dfc0ada56c5cc67ec9ad30517a8) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: use a split/filter/rejoin for _removeChristopher Larson2013-08-281-7/+3
| | | | | | | | | | | This is more idiomatic, and from the limited performance testing I did, is faster as well. See https://gist.github.com/kergoth/6360248 for the naive benchmark. (Bitbake rev: 1aa49226d5a2bac911feeb90e3d9f19529bc1a3e) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add _remove operatorRichard Purdie2013-08-261-5/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | There are long standing complaints about the fact its very difficult to remove a portion of a variable. The immediate request is for a -= and =- operator. The trouble is that += and =+ are "immediate" operators and are applied straight away. Most people would expect -= and =- to be deferred to have the effect most people desire and therefore implementing -= and =- would just make the situation more confusing. This deferred operation is much more similar to the override syntax which happens at data store finalisation. The _remove operator is therefore in keeping with the _append and _prepend operations. This code is loosely based on a patch from Peter Seebach although it has been rewritten to be simpler, more efficient and avoid some potential bugs. The code currently only works on space delimited variables, which are by far the most commom type. If bitbake is ehanced to support types natively in future, we can adjust this code to adapt to that. (Bitbake rev: 9c91948e10df278dad4832487fa56888cd58d187) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix bug with overrides and weak default valuesRichard Purdie2013-06-211-5/+12
| | | | | | | | | | | | | | | | | | | | (aka pay the cookie monster for weak defaults) If you have code like: MYVAR = "a" MYVAR_override ??= "b" then MYVAR will get the value "a" even when override is in OVERRIDES. The reason is that the value of ??= is set as a flag not a value and the cookie monster isn't paid. The fix is to ensure appropriate payment is made for a defaultval varflag matching the usual setVar case. (Bitbake rev: 3d8044bc79c482c5ea008ddf12a8128dcd1527ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Ensure variable flags are accounted for in config data hashRichard Purdie2013-06-211-0/+6
| | | | | | | | | | | | | | | | | | | | | | | Currently if the flags set against a variable in the base data store change, it doesn't automatically trigger a reparse when it really should. For example with the blacklist class setting: PNBLACKLIST[qemu] = "bar" PNBLACKLIST[bash] = "foo" will not trigger a reparse if only one entry is changed and a blacklisted recipe can still be built. I did consider using BB_SIGNATURE_EXCLUDE_FLAGS in here however it doesn't make sense, we want to trigger a reparse when any of the flags change too (which is different to the sstate signatures which we wouldn't want to change in those cases). [YOCTO #4627] (Bitbake rev: ed74ea50043f6feb698c891e571feda2b9f8513d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart.py: add some internal bitbake variables to configuration ↵Laurentiu Palcu2013-05-031-0/+10
| | | | | | | | | | | | | | hash Take __BBTASKS, __BBHANDLERS and __BBANONFUNCS into account when computing the configuration hash. [YOCTO #4447] (Bitbake rev: 260ced7452405fc43ce3d9dd6798236aa07cc716) Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake & hob: implement functions to assure consistency for ↵Cristiana Voicu2013-01-311-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | configuration files Added a new command in bitbake to save a variable in a file; added a function in cooker which is called by this command. Added new command in bitbake to enable/disable data tracking. The function saveConfigurationVar from cooker.py saves a variable in the file that is received by argument. It checks all the operations made on that variable, using the history. If it's the first time when it does some changes on a variable,it comments the lines where an operation is made on it, and it sets it in a line to the end of file. If it's not the first time(it has a comment before), it replaces the line. Made some changes in hob to save the variables from bblayers.conf and local.conf using the bitbake command. [YOCTO #2934] (Bitbake rev: 55b814ccfa413d461d12956896364ab63eed70a8) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: data_smart.py and friends: Track variable historyPeter Seebach2013-01-181-37/+219
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds tracking of the history of variable assignments. The changes are predominantly localized to data_smart.py and parse/ast.py. cooker.py and data.py are altered to display the recorded data, and turn tracking on for the bitbake -e case. The data.py update_data() function warns DataSmart.finalize() to report the caller one further back up the tree. In general, d.setVar() does what it used to do. Optionally, arguments describing an operation may be appended; if none are present, the operation is implicitly ignored. If it's not ignored, it will attempt to infer missing information (name of variable, value assigned, file and line) by examining the traceback. This slightly elaborate process eliminates a category of problems in which the 'var' member of the keyword arguments dict is set, and a positional argument corresponding to 'var' is also set. It also makes calling much simpler for the common cases. The resulting output gives you a pretty good picture of what values got set, and how they got set. RP Modifications: a) Split from IncludeHistory to separate VariableHistory b) Add dedicated copy function instead of deepcopy c) Use COW for variables dict d) Remove 'value' loginfo value and just use 'details' e) Desensitise code for calling order (set 'op' before/after infer_caller_details was error prone) f) Fix bug where ?= "" wasn't shown correctly g) Log more set operations as some variables mysteriously acquired values previously h) Standardise infer_caller_details to be triggered from .record() where at all possible to reduce overhead in non-enabled cases i) Rename variable parameter names to match inference code j) Add VariableHistory emit() function to match IncludeHistory k) Fix handling of appendVar, prependVar and matching flag ops l) Use ignored=True to stop logging further events where appropriate (Bitbake rev: f00524a3729000cbcb3317fee933ac448fae5e2d) Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: data_smart.py and friends: Track file inclusions for ↵Peter Seebach2013-01-181-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bitbake -e This code adds inclusion history to bitbake -e output, showing which files were included, in what order. This doesn't completely resolve timing questions, because it doesn't show you which lines of a file were processed before or after a given include, but it does let you figure out what the path was by which a particular file ended up in your build at all. How it works: data_smart acquires a .history member, which is an IncludeHistory; this represents the inclusion of a file and all its inclusions, recursively. It provides methods for including files, for finishing inclusion (done as an __exit__), and for dumping the whole tree. The parser is modified to run includes inside a with() to push and pop the include filename. RP Modifications: a) Split Include and Variable tracking b) Replace deepcopy usage with dedicated copy function c) Simplify some variable and usage (Bitbake rev: b2dda721262da8abb7dc32d019e18fbc32ed8860) Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart/BBHandler: Fix SkipParse exception handlingRichard Purdie2012-12-141-0/+2
| | | | | | | | | | If SkipParse is raised from something which isn't anonymous python, it wasn't being handled correctly. This improves the handling for example from within inline python. (Bitbake rev: 7467d7d66b24cc8f43ab168e65895e7c4aee6092) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve get_hash to account for overrides and key expansionRichard Purdie2012-12-031-3/+7
| | | | | | | | | | | | | | | | | | | | | | An issue was uncovered where changing: IMAGE_INSTALL_append = "X" to IMAGE_INSTALL_append = "X Y" in local.conf would not get noticed by bitbake. The issue is that the configuration hash doesn't account for overrides or key expansion. This patch improves get_hash to account for these. This means the hash does account for changes like the above. [YOCTO #3503] (Bitbake rev: 86bf1f5603e8f98019544e45f51bd0db9a48112a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/data.py: improve output for expansion errorsPaul Eggleton2012-09-271-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Instead of logging the function/variable separately as a NOTE when failing to expand, re-raise ExpansionError with more contextual information. This means that the full details are reported in Hob as well as actually reporting the original error message in any UI where we previously did not. For example, we used to get this with tab/space indentation issues in a python function: NOTE: Error expanding variable populate_packages ERROR: Unable to parse /path/to/recipename.bb Now, we will get this: ERROR: ExpansionError during parsing /path/to/recipename.bb: Failure expanding variable populate_packages: IndentationError: unindent does not match any outer indentation level (<string>, line 4) Fixes [YOCTO #3162]. (Bitbake rev: ce5c7a95a359cdaecab7c4a519ad4f9df029da82) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: skip all interal (underscored) flags in getVarFlagsRoss Burton2012-08-221-1/+1
| | | | | | | (Bitbake rev: 9f631e29a2eebb96a8291839dd8b39aa9126a10e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data-smart: rename the 'content' internal variable to '_content'Ross Burton2012-08-221-8/+8
| | | | | | | (Bitbake rev: 684cf09aed09ec82c8afb99895f92d73cd0519df) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix unanchored regexp causing strange parsing issueRichard Purdie2012-08-031-1/+1
| | | | | | | | | | | | | | | If this regular expression is unanchored, it would accept strings like: do_install_append1 do_install_appendsomelongstring and treat them like they were do_install_append. Clearly this isn't desirable. Only one instance of this type of issue was found in OE-Core and has been fixed so correcting the regexp should be safe to do. (Bitbake rev: 23bd5300b4a99218a15f4f6b0ab4091d63a602a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Fix multiple override interaction with append and ↵Richard Purdie2012-07-051-1/+6
| | | | | | | | | | | | | | | | | | | | | prepend operators Variables which used multiple overrides and the append/prepend operators were not functioning correctly. This change fixes that. This fixes the testcase: OVERRIDES = "linux:x86" TESTVAR = "original" TESTVAR_append_x86 = " x86" TESTVAR_append_x86_linux = " x86+linux" TESTVAR_append_linux_x86 = " linux+x86" [YOCTO #2672] (Bitbake rev: dc35a2e506e15fb7ddbf74c3b3280e9e83ab33bb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/data_smart.py: don't report variable in ExpansionError if not setPaul Eggleton2012-05-301-1/+4
| | | | | | | | | | If the variable name is not specified then don't confuse the error message by starting off with "Failure expanding variable None...". (Bitbake rev: 9cb16f3c73751e7cf6d495586a6193f06eb97b1f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart: Improve the calculation of config hashDongxiao Xu2012-04-171-3/+4
| | | | | | | | | | | For config hash, we put the keys in structure of "set()", which is not order sensitive. Therefore when calculating the md5 value for config hash, we need to identify the order of the keys. (Bitbake rev: 0f1b142a3f6b8125bf023c2e5ec269618869abf7) Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/classes: Convert to use appendVar and appendVarFlagsRichard Purdie2012-03-051-1/+1
| | | | | | (From OE-Core rev: 3b57de68e70e77dbc03c0616a83a29a2e99e40b4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart.py: Uses BB_HASHCONFIG_WHITELIST to filter unnecessary variablesDongxiao Xu2012-02-271-12/+4
| | | | | | | | | | | | | Adopt the BB_HASHCONFIG_WHITELIST as a exclusion list for variables that are not needed in cache hash calculation. (Bitbake rev: ae8cf138b5eb8f1f28a7143b8d67ad06cbe43061) Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> CC: Christopher Larson <kergoth@gmail.com> CC: Martin Jansa <martin.jansa@gmail.com> CC: Andreas Oberritter <obi@opendreambox.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cache: Use configuration's hash value to validate cacheDongxiao Xu2012-02-231-0/+21
| | | | | | | | | | | | | | | Previously we use the file time stamp to judge if a cache is valid. Here this commit introduce a new method, which calculates the total hash value for a certain configuration's key/value paris, and tag it into cache filename, for example, bb_cache.dat.xxxyyyzzz. This mechanism also ensures the cache's correctness if user dynamically setting variables from some frontend GUI, like HOB. (Bitbake rev: 1c1df03a6c4717bfd5faab144c4f8bbfcbae0b57) Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart: Add appendVar/prependVar functionsRichard Purdie2011-11-101-0/+16
| | | | | | | | | | | | | | | | | | | This patch adds appendVar and prependVar functions to the data store meaning python code would no longer have to do the getVar, append and the setVar dance that much of the current python code does. It also adds corresponding variants for flags. Currently there is no spacing added by these functions. That could be added as a parameter if desired. If these functions turn out to be hotspots in the code, there are tricks that could potentially be used to increase the speed of these specific operations within the datastore. (Bitbake rev: 4a4046268f84b85559eea2c4b6a6004ad8cccb77) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: silence non-literal warnings for vardepsChristopher Larson2011-11-101-1/+7
| | | | | | | | | | If the vardeps flag is not None, we now silence the warnings about non-literal usage for that variable. (Bitbake rev: e724b9f417d1baf898f5afc6376c73c1a2ad8db9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: accept a name for better messagesChristopher Larson2011-11-101-1/+1
| | | | | | | | | | | | | | | | | | | | - If a name is passed to the parser, prepend the messages with "while parsing <name>:". This gives a bit more context. - Tweak the warning messages slightly (they had to be altered anyway to inject the variable being parsed). Before: DEBUG: Warning: in call to 'bb.data.getVar': argument ''%s' % var' is \ not a literal After: DEBUG: while parsing emit_pkgdata, in call of bb.data.getVar, argument \ ''%s' % var' is not a string literal (Bitbake rev: 1060193ae4d54e667735dbff5d1d2be49a3f95c9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart.py: make use of expand cache in getVar()Dongxiao Xu2011-08-291-1/+6
| | | | | | | | | | | | | | | Currently if passing expand=True to getVar() function, it will pass the handling to getVarFlag(), which doesn't get any benefit from the expand cache. Call the expand() function separately in getVar() to make use of the expand cache, which can decrease the parsing time by 40%. (from current 49s to 27s) (Bitbake rev: 6555a77c199f41bf35460138764e03e30c56d29f) Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/ast: Fix ??= vs. ?= handlingRichard Purdie2011-07-051-8/+4
| | | | | | | | | | As the code stands, setting a variable with ??= could result in a ?= variable not overriding it. This patch fixes the issue by allowing the ast to make lookups that ignore any ??= set variables. (Bitbake rev: 32fee2e650dfdd3aa9a7572dad1251e0c24ca34b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Don't track overrides in deleted variable namesRichard Purdie2011-06-281-1/+4
| | | | | | | | | | | | | When we delete a variable we no longer expect it to override other variables. To do this we remove it from the list of active overrides at deletion time. It turns out we already had to do this at override expansion time so this cleans up the code to be more consistent as an added bonus. (Bitbake rev: d924ff9ede57c3dea6e1c738ba3633f18d460b14) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Don't export deleted/empty entries in the list of keysRichard Purdie2011-06-281-0/+2
| | | | | | | | | | | | | | | If you d.delVar(), you expect the variable to be gone. Even empty variables continue to exist in the datastore and are still user visible unfortunately. The COW siutation means you can't just remove it since it might unmask a variable from an inner copy. This patch therefore stops empty variables from appearing in key lists exposed to the external world making empty variables an internal implementation detail only. (Bitbake rev: 2b5548c591d4cfde9238d2cc0959c42cfc08f09c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Optimise the data store iteratorRichard Purdie2011-06-281-10/+12
| | | | | | | | | | Since we're going to creat the seen set() anyway, we might as well use it directly. If we don't do this, we see thousands of function calls with associated overhead on profiles. (Bitbake rev: 9d43e3279895639ee4899df635f2546c7ee13737) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Change overrides behaviour to remove expanded variables ↵Richard Purdie2011-06-091-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | from the datastore Currently if you do: OVERRIDES = "z" DEPENDS_prepend = "a " DEPENDS = "b" DEPENDS_z = "c" d.update_data() d.getVar("DEPENDS") gives "a c" d.update_data() d.getVar("DEPENDS") then gives "c" This patch changes the behaviour such that at the time bitbake expands the DEPENDS_z override, it removes "DEPENDS_z" from the data store. In the above example this would mean that it wouldn't matter how often you call d.update_data(), you'd always get "a c" back. See the bitbake-devel mailing list for further discussion and analysis of the potential impact of this change. (Bitbake rev: 899d45b90061eb3cf3e71029072eee42cd80930c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data_smart: Adding overrides of zero length is pointless so lets notRichard Purdie2011-06-011-3/+4
| | | | | | (Bitbake rev: 67434496108efc3aba9cb1e3640bc712658b1408) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* More 'is' fixupsChris Larson2011-03-311-2/+2
| | | | | | | (Bitbake rev: 967cd1aa2c59f15d805862bd9935f507c635c762) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Improve Variable expansion error handlingRichard Purdie2011-02-231-2/+14
| | | | | | | | | | | | | | | | If expanding a variable triggers an exception the caller currently has no way to supress the error message or otherwise handle the siutation. An example of where this is a problem is "bitbake -e" showing tracebacks and errors for variables like SRCPV in OE/Poky. Secondly in a chained expansion fails, log mesages are recorded for every step of the expansion, not just the innermost error which is where the real failure occured. To fix this we introduce a new exception ExpansionError which callers can handle as appropriate. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data_smart: Improve the way lazyassignment worksRichard Purdie2011-02-231-0/+2
| | | | | | | | | | | | | | | | | | | | Currently, if a variable has been set with ??= and the code looks it up before the data finalisation phase, no value is found. This is causes serious problems for anonymous python functions which manipulate data, or for the fetcher revision handling code where revisions can be set with ??=. There is also a significant performance implication for processing lazy assignment in finalise. Moving the check for a default value into getVarFlag addresses both the timing issue and the performace. This change gives a 7% real time performance improvement to parsing the Poky metadata. The cost of the check at this point is minimal since we have all the data flags available. This should also fix Yocto bug 752. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-13 21:51:14 +0000