summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/codeparser.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: refactor out codeparser cache into a separate classPaul Eggleton2012-05-231-134/+57
| | | | | | | | | | We want to be able to reuse most this functionality for the file checksum cache. (Bitbake rev: 0fe3cb1438d297f90dd0fc6b26362ecbff75c76d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: Call intern over the set contents for better cache performanceRichard Purdie2012-03-121-0/+21
| | | | | | | | See the comment in the code in the commit for more information. (Bitbake rev: 2d56dc7b1f0d186e14c4c8a949b280b6b3fc31de) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: Compute extra cache components on the fly rather than laterRichard Purdie2012-03-121-21/+20
| | | | | | | | | | | | | | | | | | | | In the initial implementation there was a relcutance on my part to generate incremental cache components on the fly since it would lead to some duplicate code. We are now seeing problems where each thread reading in the saved cache file causes significant overhead and can make the process appear to hang on a many core build, particularly when the cache file is large. This patch changes the code to maintain the delta in a separate dict right from the start. The code duplication isn't too bad and could be mitigated in other ways if it becomes an issue. [YOCTO #2039 partial] (Bitbake rev: cdd5d0dee6ab12326b252b6b505a316a52638cac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: silence non-literal warnings for vardepsChristopher Larson2011-11-101-5/+23
| | | | | | | | | | If the vardeps flag is not None, we now silence the warnings about non-literal usage for that variable. (Bitbake rev: e724b9f417d1baf898f5afc6376c73c1a2ad8db9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: drop expand trackingChristopher Larson2011-11-101-11/+0
| | | | | | | | | | | | | | | | | | | | | | There are two usual cases involving bb.data.expand: - Calling it with a string literal -- "bb.data.expand('${FOO}/${BAZ}/bleh', d)". - Calling it on getVar results (legacy) -- "bb.data.expand(bb.data.getVar('FOO', d), d)" Nothing in any of the usual layers uses it in any other way, and I'm having trouble coming up with any real use cases beyond this. The first of the above cases is already tracked, via the expandWithRefs called on the python code string. The second didn't emit a warning anyway, since the getVar was already handled. Given this, I see no reason for us to maintain explicit expansion tracking. Further, we weren't using its results anyway (the var_expands member). (Bitbake rev: 405dfe69e6a608826e599ebf2f83ef8cf5083b96) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: accept a name for better messagesChristopher Larson2011-11-101-8/+10
| | | | | | | | | | | | | | | | | | | | - If a name is passed to the parser, prepend the messages with "while parsing <name>:". This gives a bit more context. - Tweak the warning messages slightly (they had to be altered anyway to inject the variable being parsed). Before: DEBUG: Warning: in call to 'bb.data.getVar': argument ''%s' % var' is \ not a literal After: DEBUG: while parsing emit_pkgdata, in call of bb.data.getVar, argument \ ''%s' % var' is not a string literal (Bitbake rev: 1060193ae4d54e667735dbff5d1d2be49a3f95c9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: simplify how we compare the called node namesChristopher Larson2011-11-101-52/+20
| | | | | | | | | | | | | | With the previous method, using the compare_name methods, we split the requested match name by '.', reversed it, then compared them piecemeal during the node traversal. The new method walks the nodes and hands back the name of what's being called, and then we check that. This also consolidates the two different implementations of traversal of the attribute/name nodes (one in compare_name, one for the execs). (Bitbake rev: 84e535b5165c7e936c5b1486bdf4626ed3649f5f) Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: merge the nested python parsing classesChristopher Larson2011-11-101-102/+87
| | | | | | | | | | The split is even less necessary now that we use ast.walk rather than an actual NodeVisitor subclass. (Bitbake rev: d6c44fac184abae8395bfa7078f06675218aa534) Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: make var_expands actually hold useful informationChristopher Larson2011-11-041-1/+1
| | | | | | | | | | | Previously, it was calling var_expands.update() rather than add(), with a string argument, resulting in adding each character of that string to the var_expands set, rather than the string itself. (Bitbake rev: 8e4e75383e43d6da2c16ec5286186a0d0569b0f8) Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: When loading the cache, ignore ValueErrorRichard Purdie2011-06-151-1/+1
| | | | | | (Bitbake rev: 9bff182a4ba9571679985b45b309990a6eddad14) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/codeparser: Improve cache handlingRichard Purdie2011-05-271-10/+70
| | | | | | | | | | | | | | | | | | | The current codeparser cache handling hurts performance badly even over a couple of cores and certainly on many core systems, it can spent huge amounts of time in the codeparser cache save functions. This patch reworks the cache handling so that each parsing thread saves out its own "differences" file compared to any existing core cache and then the main bitbake thread picks these up and merges things back together. This was tested on systems with small and large numbers of cores and was found to perform orders of magnitude better in all cases despite the more complex code. (Bitbake rev: 9f27563d66523f5af1028f173d53ee75e0877d46) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser.py: fix syntax error in exception handlingOtavio Salvador2011-05-241-1/+1
| | | | | | | | | | Commit 036cf3cd11b3a6836b77f5ffa760ceee6b71b1ef missed the needed brackets to handle more then a type of exception. (Bitbake rev: abecbb4c0af83c6b4ee248b0f03b779f84b13390) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser.py: Ignore incomplete cache filesRichard Purdie2011-05-121-1/+1
| | | | | | (Bitbake rev: 036cf3cd11b3a6836b77f5ffa760ceee6b71b1ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/codeparser: Correctly handle a missing/empty cache fileRichard Purdie2011-05-051-2/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker/codeparser: Ensure the code parser cache is saved for each ↵Richard Purdie2011-05-051-0/+14
| | | | | | | | | | | | | | parsing process Before this change, the codeparser cache was only being saved for the main server process. This is suboptimal as it leaves code being re-evaluated at task execution time and increases parse time. We use the multiprocess Finalize() functionality to ensure each process saves out its cache. We need to update the cache save function to be multiprocess friendly with locking. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* codeparser: use ==, not 'is' to compare stringsChris Larson2011-03-311-2/+2
| | | | | | | (Bitbake rev: 8f5cf3a9975d8e6878e403be0e6edc22cc44f396) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/codeparser: fix raising of ShellSyntaxErrorJoshua Lock2011-02-251-2/+2
| | | | | | | To raise the ShellSyntaxError we need to import it's module and reference it by namespace. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* codeparser: fix spacing in diagnostic messagesBernhard Reutner-Fischer2011-01-121-2/+2
| | | | | | | (Bitbake rev: 8c5555f5ed6d61db57de80d2820c8cec64a27239) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/codeparser: Fix import to match upstream bitbakeRichard Purdie2011-01-101-1/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-17/+23
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move the pysh package into the bb packageChris Larson2011-01-041-1/+1
| | | | | | | | | | The pysh we're using is modified, and we don't want to risk it conflicting with one from elsewhere. (Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* codeparser.py: Fix storing of hash values as object references can be corruptedRichard Purdie2010-09-281-8/+11
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/codeparser: Deal with functions with trailing whitespaceRichard Purdie2010-09-071-1/+8
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/codeparser: Ensure cached sheel entries return the correct dependenciesRichard Purdie2010-09-031-1/+1
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/codeparser: Implement persistent cacheRichard Purdie2010-08-311-10/+56
| | | | | | | For a given input to this code, the output doesn't change to implement a persistent cache of the data to speed up parsing. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Add codeparser for parsing shell and python functionsRichard Purdie2010-08-311-0/+273
This commit is derived from Chris Larson's checksum work, turned into a standalone piece of code for parsing python and shell functions. The deindent code has been replaced with code to work around indentation for speed. The original NodeVisitor in the ast was replaced with a faster class walk call. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-09 20:11:12 +0000