summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/data.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: data: Ensure dependencies of subfunctions are accounted forRichard Purdie2013-05-221-0/+1
| | | | | | | | | | | | Currently we account of the top level function's vardeps but not those of any subfunction. This would imply we'd have to manually write the dependencies of all parent functions which would be crazy. This patch adds the dependencies to fix the issue. (Bitbake rev: 658008cc6a8acd5c1f26877c9444c96002db01e7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: fix performance regressionPaul Eggleton2013-04-151-2/+2
| | | | | | | | | | | | | | BitBake commit 7c568132c54a21161de28907159f902462f1e2bb resulted in a fairly serious performance regression during parsing, almost doubling the time taken to do a full parse and almost certainly impacting performance during building. The expandKeys function is called frequently, and if we avoid using keys() and instead just use the normal variable lookup mechanism, performance is restored. (Bitbake rev: 034b392e9877309f15940b258fc2c16f16fb40b5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data.py: Add a warning when expandKeys overwrites an existing keyMark Hatle2013-04-101-0/+5
| | | | | | | | | | | | | | | | | | | When two variables are defined as: ${var} = "bar" foo = "foobar" The value of 'foo' when ${var} == foo becomes indeterminate. We want to warn a user when this situation has been encountered so they can take corrective actions. In the above example usually foo == bar, unless multilibs are enabled. Then ml-foo = "ml-foobar". (Bitbake rev: 7c568132c54a21161de28907159f902462f1e2bb) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: data_smart.py and friends: Track variable historyPeter Seebach2013-01-181-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds tracking of the history of variable assignments. The changes are predominantly localized to data_smart.py and parse/ast.py. cooker.py and data.py are altered to display the recorded data, and turn tracking on for the bitbake -e case. The data.py update_data() function warns DataSmart.finalize() to report the caller one further back up the tree. In general, d.setVar() does what it used to do. Optionally, arguments describing an operation may be appended; if none are present, the operation is implicitly ignored. If it's not ignored, it will attempt to infer missing information (name of variable, value assigned, file and line) by examining the traceback. This slightly elaborate process eliminates a category of problems in which the 'var' member of the keyword arguments dict is set, and a positional argument corresponding to 'var' is also set. It also makes calling much simpler for the common cases. The resulting output gives you a pretty good picture of what values got set, and how they got set. RP Modifications: a) Split from IncludeHistory to separate VariableHistory b) Add dedicated copy function instead of deepcopy c) Use COW for variables dict d) Remove 'value' loginfo value and just use 'details' e) Desensitise code for calling order (set 'op' before/after infer_caller_details was error prone) f) Fix bug where ?= "" wasn't shown correctly g) Log more set operations as some variables mysteriously acquired values previously h) Standardise infer_caller_details to be triggered from .record() where at all possible to reduce overhead in non-enabled cases i) Rename variable parameter names to match inference code j) Add VariableHistory emit() function to match IncludeHistory k) Fix handling of appendVar, prependVar and matching flag ops l) Use ignored=True to stop logging further events where appropriate (Bitbake rev: f00524a3729000cbcb3317fee933ac448fae5e2d) Signed-off-by: Peter Seebach <peter.seebach@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler/data: Standardise some setVar access formatsRichard Purdie2013-01-181-3/+3
| | | | | | (Bitbake rev: e12c1a485f96a4701144ac81179ae1af348e5bf3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Ensure emit_func honours vardeps flagRichard Purdie2012-12-171-0/+1
| | | | | | | | | | | | | | | Currently, calling a shell function using exec_func may result in dependent functions being missing, if the shell parser can't detect the dependency, even if it was specified with the vardeps flag. This patch ensures the function looks at the flag and considers it when deciding which other functions need to be output. [YOCTO #3561] (Bitbake rev: a906c94328b040d5076571bd593b57be7ee403a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBhandler/data: Fix __inherit_cache duplicationRichard Purdie2012-12-111-2/+4
| | | | | | | | | | | | The inherits cache contains duplicate entries, some with the full patch, some just starting classes/xxx. This is a waste of parse time and potentially error prone. This patch fixes various pieces of code so the absolute paths are always preferred and work correctly. The inherits_class function did not work with full paths so the patch fixes this. (Bitbake rev: f3a71e509af196391ec126d079cf1bd178e62ad5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/data.py: improve output for expansion errorsPaul Eggleton2012-09-271-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Instead of logging the function/variable separately as a NOTE when failing to expand, re-raise ExpansionError with more contextual information. This means that the full details are reported in Hob as well as actually reporting the original error message in any UI where we previously did not. For example, we used to get this with tab/space indentation issues in a python function: NOTE: Error expanding variable populate_packages ERROR: Unable to parse /path/to/recipename.bb Now, we will get this: ERROR: ExpansionError during parsing /path/to/recipename.bb: Failure expanding variable populate_packages: IndentationError: unindent does not match any outer indentation level (<string>, line 4) Fixes [YOCTO #3162]. (Bitbake rev: ce5c7a95a359cdaecab7c4a519ad4f9df029da82) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Warn upon finding tab indentation in python functionsRichard Purdie2012-07-191-0/+2
| | | | | | | | | | | | Mixed spacing in python functions can cause subtle issues and generally confuses users. We've standardised on 4 space indentation, adding this warning helps ensure consistency and avoid bugs. It also makes _prepend and _append operations on python functions slightly less risky. (Bitbake rev: c51cfaf48d3b12a19b01e824b6ba4230376bcad4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: include varflags in checksumsPaul Eggleton2012-05-301-1/+19
| | | | | | | | | | | | | | | | | | | | | | Add a dependency to each variable on a filtered list of its varflags. This is intended to catch things such as SRC_URI checksums, varflags controlling extra functionality from classes (e.g. the recently updated update-alternatives class in OE-Core), etc. and ensure their values influence the sstate checksums. There is an exclusion list which needs to be set via bitbake.conf (BB_SIGNATURE_EXCLUDE_FLAGS), if this is not set then the functionality is disabled. The existing vardepsexclude mechanism can also be used to exclude undesired varflags, but they must be fully specified, e.g.: do_patch[vardepsexclude] += "do_patch[someflag]" Implements [YOCTO #2517]. (Bitbake rev: 56c1ab18fcaf4ac245dcb412ed55e8e0af07883b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data/runqueue.py: Fix commented out test code after variable name changesRichard Purdie2012-01-271-1/+1
| | | | | | (Bitbake rev: 175c5c8a5e51365aa127562e8598e84f5cd36495) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data/siggen: Add vardepvalue mechanism to allow the variable dependency code ↵Richard Purdie2011-11-241-9/+14
| | | | | | | | | | | | | | | | | | | | | | | to be forced to specific values We have a problem if we want to inject specific information into the variable dependency code. There are cases for example where we want a dependency on the value of X but it doesn't matter how X was constructed or what dependencies it might have had, we only care about the absolute value. With the current code, its near enough impossible to do this. This patch adds such a mechanism so the user can trigger this with code like: baselib[vardepvalue] = "${baselib}" It also refactors some of the code so we do variable lookups once instead of doing this in two different functions. [YOCTO #1583] (Bitbake rev: 6c879b44ccf42dc73fe4467076e114700d7ba81b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: silence non-literal warnings for vardepsChristopher Larson2011-11-101-5/+9
| | | | | | | | | | If the vardeps flag is not None, we now silence the warnings about non-literal usage for that variable. (Bitbake rev: e724b9f417d1baf898f5afc6376c73c1a2ad8db9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: accept a name for better messagesChristopher Larson2011-11-101-4/+4
| | | | | | | | | | | | | | | | | | | | - If a name is passed to the parser, prepend the messages with "while parsing <name>:". This gives a bit more context. - Tweak the warning messages slightly (they had to be altered anyway to inject the variable being parsed). Before: DEBUG: Warning: in call to 'bb.data.getVar': argument ''%s' % var' is \ not a literal After: DEBUG: while parsing emit_pkgdata, in call of bb.data.getVar, argument \ ''%s' % var' is not a string literal (Bitbake rev: 1060193ae4d54e667735dbff5d1d2be49a3f95c9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Ensure only the filtered environment variables are inherited from the OSJoshua Lock2011-08-151-7/+8
| | | | | | | | | | | | | | | | | | The recent change which modified inheritFromOS to use the intial environment, rather than the current environment, introduced a bug such that variables which had been cleaned from the environment where still set in the data store. This patch changes things such that a list of approved environment variables is saved after the environment is cleaned and only the variables in this list are inherited in inheritFromOS. CC: James Limbouris <james.limbouris@gmail.com> CC: Chris Larson <clarson@kergoth.com> (Bitbake rev: cb6c07054e8baf94614713ec257c643b22266d75) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* data|cooker: use saved environment variables when calling inheritFromOSJoshua Lock2011-07-261-4/+4
| | | | | | | | | | Now that we have a pristine copy of the variables available at launch time we can use them when looking to inherit the OS's environment. (Bitbake rev: 21c21fcc5871e81d8d497b6baed605cdd74c4571) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-1/+1
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data/runqueue: Sync up with upstream to clean up environment ↵Richard Purdie2011-06-081-16/+11
| | | | | | variable handling Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/siggen.py: Fix whitelisted variable handlingRichard Purdie2011-01-121-4/+1
| | | | | | | | | Even when a variable was whitelisted, any dependencies of that variable could still creep into the task hash due to the way the whitelisting code worked. This patch changes thing to ensure that when whitelisted, that whitelisting applies to the variable and any dependencies it has. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync environment handling function names and comments with upstream ↵Richard Purdie2011-01-101-1/+1
| | | | | | alterations Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data.py: corrected the output for shell syntax.Lianhao Lu2011-01-071-1/+3
| | | | | | | | | | | [BUGID# 645], modify the emit_var() 1. Added "#" to the beginning of each line if the comment contains multiple lines. 2. Added "\" to the end of each line if the shell variable value contains multiple lines. Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
* data: fewer newlines for (un)exportBernhard Reutner-Fischer2011-01-041-2/+2
| | | | | | | | | | | Previously we emitted two newlines for export and unexport. One newline for export and unexport is enough (and makes the scripts look better and a tad smaller). (Bitbake rev: ba060160fdf1278a273fb2b77d36b8c681807ecf) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-2/+2
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: add optional expansion to getVarFlag()Paul Eggleton2010-12-141-2/+2
| | | | | | | | Add a parameter to getVarFlag() to auto-expand the value of the flag. This makes getVarFlag() more consistent with getVar(), and allows expansion of vardeps and vardepsexclude (which has been done in this commit). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* bitbake: allow excluding dependencies of variablesPaul Eggleton2010-12-141-0/+1
| | | | | | | | | Adds a vardepsexclude flag that can be used to exclude a dependency of a variable (the opposite of vardeps). This will allow the exclusion of variables from the hash generation much more selectively than blanket whitelisting using BB_HASHBASE_WHITELIST. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* bitbake: Overhaul environment handlingRichard Purdie2010-12-091-1/+9
| | | | | | | | | | | | | | Currently, anything whitelisted in the environment makes it into the worker processes. This is undesireable and the worker environment should be as clean as possible. This patch adapts bitbake sosme variables are loaded into bitbake's datastore but not exported by default. Any variable can be exported by setting its export flag. Currently, this code only finalises the environment in he worker as doing so in the server means variables are unavailable in the worker. If we switch back to fork() calls instead of exec() this code will need revisting. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/data.py: Allow variables to suplement dependencies using the vardep flagRichard Purdie2010-09-131-0/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/data.py: Add emit_func() and generate_dependencies() functionsRichard Purdie2010-08-311-0/+75
| | | | | | | | These functions allow generation of dependency data between funcitons and variables allowing moves to be made towards generating checksums and allowing use of the dependency information in other parts of bitbake. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/data.py: Handle exceptions in export_bars in the same way as emit_var()Richard Purdie2010-08-201-3/+8
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run ↵Richard Purdie2010-08-191-0/+9
| | | | | | shell and python under a fakeroot environment Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Don't emit vars starting with __, as those are internalChris Larson2010-07-021-1/+1
| | | | | | | (Bitbake rev: 211f44ce0e15c7c419606e9ef3568d9cea6775b7) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Enhance the expansion exception messageChris Larson2010-07-021-6/+3
| | | | | | | (Bitbake rev: 1cc11164a85a5b30d6b9184cb5a8f7f7ac1aaff2) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Allow emission of non-string objects in emit_var, instead running str() on ↵Chris Larson2010-07-021-4/+2
| | | | | | | | | it first (Bitbake rev: 5b09e28cb4a2c6c99ad58e71fb803160922a7e1e) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* emit_env: clean up, iterate onceChris Larson2010-07-021-11/+7
| | | | | | | (Bitbake rev: db718ec6f67c7c0d0efb4ba0b5b24384f707dcf5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Kill unnecessary usages of the types moduleChris Larson2010-07-021-2/+2
| | | | | | | | | | | | | types.IntType -> int types.StringType -> basestring ... Also moves our ImmutableTypes tuple into our own namespace. (Bitbake rev: 83674a3a5564ecb1f9d2c9b2d5b1eeb3c31272ab) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Apply some 2to3 transforms that don't cause issues in 2.6Chris Larson2010-07-021-1/+1
| | | | | | | (Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Drop unused bb.data.expandData functionChris Larson2010-07-021-16/+0
| | | | | | | (Bitbake rev: 188af3ef5fdb5cd86b74147bf5eafc085ac489be) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Add some missing docstringsChris Larson2010-07-021-0/+3
| | | | | | | (Bitbake rev: 20b6f2d1d2d4541e612e8cffbdf1ca4822a3e394) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Drop some unused functionsChris Larson2010-07-021-28/+0
| | | | | | | (Bitbake rev: eb83df5bc7077685f314df01949cf06850e4693c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Drop some unused classesChris Larson2010-07-021-3/+0
| | | | | | | (Bitbake rev: 5cc47a49518c69c94f8b78e9dda6d542ba17b1e8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Formatting cleanupsChris Larson2010-07-021-18/+17
| | | | | | | (Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Drop doctest, as we aren't utilizing itChris Larson2010-07-021-150/+13
| | | | | | | | | | | We should add back in some proper unit testing. Doctest is rather limited, and we haven't actually made use of it since the original implementation of the datastore when the project started, as far as I'm aware. (Bitbake rev: 3a11c2807972bbbddffde2fa67fc380d159da467) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move update_data into the DataSmart class as a finalize() methodChris Larson2010-07-021-136/+2
| | | | | | | (Bitbake rev: ff801397785567cb84b3615de86bff764d65decf) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move MalformedUrl, VarExpandError into appropriate modules.Chris Larson2010-03-221-0/+3
| | | | | | | (Bitbake rev: e616483b237dafff7f90ba1c09e9ee7c383a2e47) Signed-off-by: Chris Larson <clarson@kergoth.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Switch to bitbake-dev version (bitbake master upstream)Richard Purdie2010-01-201-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Merge further fixes from upstream 1.8 branchRichard Purdie2009-05-111-0/+2
| | | | | | | | * Make the test functionality work * Optimise BBPATH handling when changing directory * Optimise file globing for BBFILES Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Make sure existing environment variables are properly exportedRichard Purdie2009-05-091-0/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Refactor the environment variable handlingRobert Bradford2008-10-171-2/+0
| | | | | | | Refactor the environment variable handling so that it more closes matches the changes that have been made in bitbake-dev. git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5527 311d38ba-8fff-0310-9ca6-ca027cbcb966
* bitbake/bitbake-dev: Allow much better control of which variable influence ↵Richard Purdie2008-09-301-15/+6
| | | | | | bitbake from the environment git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5347 311d38ba-8fff-0310-9ca6-ca027cbcb966
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-13 22:23:44 +0000