summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/utils.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: bitbake: cooker: properly fix bitbake.lock handlingRichard Purdie2015-09-011-4/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the PR server or indeed any other child process takes some time to exit (which it sometimes does when saving its database), it can end up holding bitbake.lock after the UI exits, which led to errors if you ran bitbake commands successively - we saw this when running the PR server oe-selftest tests in OE-Core. The recent attempt to fix this wasn't quite right and ended up breaking memory resident bitbake. This time we close the lock file when cooker shuts down (inside the UI process) instead of unlocking it, and this is done in the cooker code rather than the actual UI code so it doesn't matter which UI is in use. Additionally we report that we're waiting for the lock to be released, using lsof or fuser if available to list the processes with the lock open. The 'magic' in the locking is due to all spawned subprocesses of bitbake holding an open file descriptor to the bitbake.lock. It is automatically unlocked when all those fds close the file (as all the processes terminate). We close the UI copy of the lock explicitly, then close the server process copy, any remaining open copy is therefore some proess exiting. (The reproducer for the problem is to set PRSERV_HOST = "localhost:0" and add a call to time.sleep(20) after self.server_close() in lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ). Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>. This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and e97a9f1528d77503b5c93e48e3de9933fbb9f3cd. (Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf) (Bitbake rev: ed30f4ee1cef8db9ea422c5e54b2375c4f3b1d6f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Conflicts: bitbake/lib/bb/tinfoil.py
* bitbake: lib/bb/utils: fix several bugs in edit_metadata_file()Paul Eggleton2015-05-231-3/+11
| | | | | | | | | | | | | | * Fix unchanged assignments being dropped if other lines changed * Fix not passing variable name from single-line assignments to the function * Fix not trimming the trailing quote from values (Bitbake master rev: 0b0c82f49cf2de887967d305768cbd95314bb171) (Bitbake rev: cd92e5dce5f5d61ecb7838bf964a7812e905509a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils: add function to get layer containing a filePaul Eggleton2015-05-231-0/+16
| | | | | | | | | | | | | | In certain contexts it can be useful to find the layer that a file (e.g. a recipe) appears in. Implements [YOCTO #7723]. (Bitbake master rev: 3bf9c8830c5d5eea5502230d5af84ebd87ad5849) (Bitbake rev: 8dcd83e5edb1cf3307b610d3bb8d54733ea7356d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils: add safeguard against recursively deleting things we ↵Paul Eggleton2015-04-241-0/+21
| | | | | | | | | | | | | | | | shouldn't Add some very basic safeguard against recursively deleting paths such as / and /home in the event of bugs or user mistakes. Addresses [YOCTO #7620]. (Bitbake master rev: 56cddeb9e1e4d249f84ccd6ef65db245636e38ea) (Bitbake rev: 9178a708ecd75758be360a5a6bb04c77b69be5d8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: rework LAYERDEPENDS versioning so that it is actually usefulPaul Eggleton2015-02-161-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We've had versioned dependency support in LAYERDEPENDS for quite a long time, but I can say with pretty good certainty that almost nobody has used it up to now because it was too strict - the specified version had to exactly match the version in your configuration or you would get an error; there was no "greater than or equal" option, which is usually what you will want given that LAYERVERSION does get bumped from time to time. However, users mismatching layer branches and then having their builds fail later on with some incomprehensible error is still a pretty common problem. We can't simply use the git branch because not everyone is always on a branch and the branch names don't always match up (and that's not an issue). To provide a practical means to address branch mismatching, I have reworked LAYERDEPENDS version specifications to use the more familiar "dependency (>= version)" syntax as used with package dependencies, support non-integer versions, and clarified the error message a little. If we then take care to bump the version on every breaking change, it is at least possible to have layers depend on these changes when they update to match; we can now even support a major.minor scheme to allow retrospectively adding a version limiter to old branches when a new branch is created and yet still allow the old branch minor version to be bumped if needed. Fixes [YOCTO #5991]. (Bitbake rev: 408be9cdf2b1e32e64ea488d8051a546fb54c144) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: ensure explode_dep_versions2 raises an exception on ↵Paul Eggleton2015-02-161-0/+4
| | | | | | | | | | | | | invalid/missing operator We really want an error rather than the version to just be silently skipped when the operator is missing (e.g. "somepackage (1.0)" was specified instead of "somepackage (>= 1.0)".) (Bitbake rev: b6dc946f477adc40d68da16e2f2580cb3b4a10db) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: make edit_metadata_file() handle VARIABLE +=Paul Eggleton2014-12-301-1/+1
| | | | | | | | | | This is a little crude as the usage basically ignores that you're doing an append operation, but for a lot of cases it will be sufficient. (Bitbake rev: 24a28205ab680b6cc645d97b76c9855920608229) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-layers: fix error handling in add-layer / remove-layerPaul Eggleton2014-12-301-0/+8
| | | | | | | | | | * Fix add-layer error message when a layer is already in BBLAYERS * Ensure we show an error message if we can't find BBLAYERS at all (Bitbake rev: 1c743fd2103730e27699dd55efc6914d3b0c3702) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: add basic metadata manipulation functionsPaul Eggleton2014-12-231-0/+127
| | | | | | | | | | | | | | * Add a generic edit_metadata_file() function to modify variable assignments in any metadata file (conf, bb, bbappend) using a callback for flexibility * Add a specific edit_bblayers_conf() function to modify conf/bblayers.conf and add and/or remove layers from the BBLAYERS value within it. (Bitbake rev: aa03a28b442549dd8ffe92ae4d6390f62202a76a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: add exec_flat_python_func()Paul Eggleton2014-12-091-0/+22
| | | | | | | | | | Add a function that allows executing a flat python function (defined with def funcname(args): ...). (Bitbake rev: 20e6939ebcb62e08a9a7ad586a915dfe368136a0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: don't use len() for truth value testing.Oscar Utbult2014-10-281-1/+1
| | | | | | | (Bitbake rev: 4bdfeab7845bdcd62a4928200dd13701414a464e) Signed-off-by: Oscar Utbult <oscar@oscr.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Improve profile log processingRichard Purdie2014-08-281-6/+1
| | | | | | | | | This stream redirection of stdout is horrible. pstats takes a stream argument so lets use that instead. (Bitbake rev: 93d155f4766e27e7b004d13569aa03961fe89e3b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/*.py: Typo fixes/grammar/comment fixes, nothing functional.Robert P. J. Day2014-08-251-2/+2
| | | | | | | (Bitbake rev: 587b144ee409d444494d8d7f2d1c53ede8f7c953) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Add workaround for multiprocessing bugRichard Purdie2014-08-231-0/+12
| | | | | | | | | | | | Our usage of multitprocessing is problematic. In particular, there is a bug in python 2.7 multiprocessing where signals are not handled until command completion instead of immediately. This adds a workaround into our wrapper function to deal with the issue. (Bitbake rev: a16185e602b39b71475aa7e9ee80ad2b1f28d0f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/utils: fix contains_any()Ross Burton2014-06-121-1/+1
| | | | | | | | | | | | | | | | "set1 in set2" doesn't do what you'd expect, and if it did do a subset test that's not the logic required by contains_any(). Instead get the intersection of checkvalues and val and check if the resulting set is empty or not (by evaluating it in a boolean context), which tells us if there are any elements in common. Based on a patch by Valentin Popa <valentin.popa@intel.com>. (Bitbake rev: 2e742c03e8dfdfa67899e7f5d579ed14bd87e139) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/utils: Fix broken exception handlingRichard Purdie2014-06-011-7/+2
| | | | | | | | | | Checking for explicit exception names is bad, we also want to be able top rely on inheritance. Fix these checks to be part of the real except clauses so SkipPackage is recognised as being inherited from SkipRecipe. (Bitbake rev: b131229145e1f2c372d6230a7b554e436c13c3f9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Add SkipRecipe event to replace SkipPackageRichard Purdie2014-06-011-1/+1
| | | | | | | | | | | | In the depths of time we were rather confused about naming. bb files are recipes, the event to skip parsing them should be SkipRecipe, not SkipPackage. This changes bitbake to use the better name but leaves the other around for now. We can therefore start removing references to it from the metadata. (Bitbake rev: 98d9e6e0f514a7cb7da1d99bf4bd5602b89426d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: avoid printing traceback on ExpansionError during parsingPaul Eggleton2014-05-301-0/+2
| | | | | | | | | | | If an ExpansionError occurs during better_exec() we should just raise it instead of printing the traceback, so that recipe errors (such as broken URLs in SRC_URI) are more easily comprehensible. (Bitbake rev: 5b0da8932c318813138c113d2bb20498145dbd42) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.utils, bb.codeparser: Add bb.utils.contains_anyOtavio Salvador2014-04-291-0/+13
| | | | | | | | | | | | | | | | | This includes contains_any in the special handling code for sstate. It does not take into account the equivalence of the values. In current code, considering 'bb.utils.contains_any("A", "foo bar", ...)': A = "foo" A = "bar" A = "foo bar" All those will get different signatures. (Bitbake rev: d1e3345d715e488ec3f5515fb0e1fb39366346bc) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ConfHander/BBHandler/utils: Fix cache dependency bugsRichard Purdie2013-11-301-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently bitbake only adds files to its dependency list if they exist. If you add 'include foo.inc' to your recipe and the file doesn't exist, then later you add the file, the cache will not be invalidated. This leads to another bug which is that if files don't exist and then you add them and they should be found first due to BBPATH, again the cache won't invalidate. This patch adds in tracking of files we check for the existence of so that if they are added later, the cache correctly invalidates. This necessitated a new version of bb.utils.which which returns a list of files tested for. The patch also adds in checks for duplicate file includes and for now prints a warning about this. That will likely become a fatal error at some point since its never usually desired to include a file twice. The same issue is also fixed for class inheritance. Now when a class is added which would be found in the usual search path, it will cause the cache to be invalidated. Unfortunately this is old code in bitbake and the patch isn't the neatest since we have to work within that framework. [YOCTO #5611] [YOCTO #4425] (Bitbake rev: 78d285871e4b8c54ccc4602d571e85f922e37ccd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Don't show stack traces for BBHandledExceptionRichard Purdie2013-10-181-0/+3
| | | | | | | | | A BBHandledException means we already showed an error to the user so we shouldn't show a stack trace as this just confuses things further. (Bitbake rev: 8a8bafc8ded98364a31878b23c64503a53affcd1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: use logger.warn instead of print in copyfileRoss Burton2013-10-071-5/+5
| | | | | | | | | print disappears into the ether, so use logger.warn and clean up the messages. (Bitbake rev: 90f91f7402ff69f3fe9fba5f94a53d371303ce34) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Fix BB_PRESERVE_ENVRichard Purdie2013-09-111-0/+2
| | | | | | | | | | | BB_PRESERVE_ENV wasn't working since data.inheritFromOS wasn't getting a correct list of keys to import into the data store. This fixes things so it does add all environment variables into the data store when BB_PRESERVE_ENV is used. (Bitbake rev: 843e9339c5ee3c99657a40a0e2c7dbd777b6ef06) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Fix multiprocessing pool referenceRichard Purdie2013-06-141-1/+1
| | | | | | | | | Interestingly the previous version comparision was broken and we were always using the compatibility code, masking this bug. Oops. (Bitbake rev: d48e8bcb24e8fa5d4fd60fd2c9927a95976d8d8b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: compat/server/utils: Jettison pre python 2.7.3 workaroundsRichard Purdie2013-06-141-5/+2
| | | | | | | | | Now we've moved to require python 2.7.3, we can jettison the compatibility workarounds/hacks for older python versions. (Bitbake rev: a51c402304f2080a76720f9b31d6dfdbed393bba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/cookerdata/utils: Improve context managementRichard Purdie2013-05-241-8/+18
| | | | | | | | | | | | | | | The current execution context management for bitbake is ugly and the use of a global variable is nasty. Fixing that is hard, however we can improve things to start to establish an API for accessing and changing that context. This patch also adds in an explicit reset of the context when we reparse the configuration data which starts to improve the lifecycle of the data in setups like hob. (Bitbake rev: 6c3281a140125337fc75783973485e16785d05a1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/utils: Clean up importsRichard Purdie2013-05-171-3/+4
| | | | | | | | | Move various random imports to the start of the modules as cleanup and avoid an import issue with bb.process on python 2.6. (Bitbake rev: aed4adfbe3a591ca4f8e41fb763c9f961bf2e6d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: Clean up various file access syntaxRichard Purdie2013-05-091-4/+10
| | | | | | | | | | | | | Python 3 is stricter about how files are accessed. Specficially: * Use open(), not file() * Use binary mode for binary files (when checksumming) * Use with statements to ensure files get closed * Add missing file close statements (Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Improve better_compile error messageRichard Purdie2013-05-091-7/+9
| | | | | | | | | Similarly to the better_exec improvements, improve the compile failure messages to be more user readable. (Bitbake rev: 9bc92d0210e13e4cc98727f6c9ec2f47c2221e77) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Improve better_exec traceback handlingRichard Purdie2013-05-091-39/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current bitbake tracebacks are hard to read/confusing and sometimes incomplete. This patch attempts to do better by: * Moving the note about the exact exception to the end to make things read in sequence * Merged the initial stack trace to become part of the code dump * Added handling for "/xxxx" file paths since we can load these files and include the data as part of the trace * Dropped the ERROR: prefix to every line, allowing the error messages to be spacially accosicated in the UIs * Moved the "From:" line to the top of each code block and ensured its present consistently With the complexity now in this funciton, I've added try/except wrapping around it to ensure we catch exceptions in the exception handler too. Example before: """ ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb: TypeError: 'filter' object is not subscriptable ERROR: The stack trace of python calls that resulted in this exception/failure was: ERROR: File "do_populate_lic", line 13, in <module> ERROR: ERROR: File "do_populate_lic", line 6, in do_populate_lic ERROR: ERROR: File "license.bbclass", line 99, in find_license_files ERROR: ERROR: File "/media/build1/poky/meta/lib/oe/license.py", line 38, in visit_string ERROR: if pos > 0 and license_pattern.match(elements[pos-1]): ERROR: ERROR: The code that was being executed was: ERROR: 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) ERROR: 0010: copy_license_files(lic_files_paths, destdir) ERROR: 0011: ERROR: 0012: ERROR: *** 0013:do_populate_lic(d) ERROR: 0014: ERROR: [From file: 'do_populate_lic', lineno: 13, function: <module>] ERROR: 0002:def do_populate_lic(d): ERROR: 0003: """ ERROR: 0004: Populate LICENSE_DIRECTORY with licenses. ERROR: 0005: """ ERROR: *** 0006: lic_files_paths = find_license_files(d) ERROR: 0007: ERROR: 0008: # The base directory we wrangle licenses to ERROR: 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) ERROR: 0010: copy_license_files(lic_files_paths, destdir) ERROR: [From file: 'do_populate_lic', lineno: 6, function: do_populate_lic] ERROR: 0095: lic_files_paths.append((os.path.basename(path), srclicfile)) ERROR: 0096: ERROR: 0097: v = FindVisitor() ERROR: 0098: try: ERROR: *** 0099: v.visit_string(license_types) ERROR: 0100: except oe.license.InvalidLicense as exc: ERROR: 0101: bb.fatal('%s: %s' % (d.getVar('PF', True), exc)) ERROR: 0102: except SyntaxError: ERROR: 0103: bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True))) ERROR: [From file: 'license.bbclass', lineno: 99, function: find_license_files] ERROR: Function failed: do_populate_lic ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.17442 """ Example after: """ ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'do_populate_lic', lineno: 13, function: <module> 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) 0010: copy_license_files(lic_files_paths, destdir) 0011: 0012: *** 0013:do_populate_lic(d) 0014: File: 'do_populate_lic', lineno: 6, function: do_populate_lic 0002:def do_populate_lic(d): 0003: """ 0004: Populate LICENSE_DIRECTORY with licenses. 0005: """ *** 0006: lic_files_paths = find_license_files(d) 0007: 0008: # The base directory we wrangle licenses to 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) 0010: copy_license_files(lic_files_paths, destdir) File: 'license.bbclass', lineno: 99, function: find_license_files 0095: lic_files_paths.append((os.path.basename(path), srclicfile)) 0096: 0097: v = FindVisitor() 0098: try: *** 0099: v.visit_string(license_types) 0100: except oe.license.InvalidLicense as exc: 0101: bb.fatal('%s: %s' % (d.getVar('PF', True), exc)) 0102: except SyntaxError: 0103: bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True))) File: '/media/build1/poky/meta/lib/oe/license.py', lineno: 38, function: visit_string 0034: new_elements = [] 0035: elements = filter(lambda x: x.strip(), license_operator.split(licensestr)) 0036: for pos, element in enumerate(elements): 0037: if license_pattern.match(element): *** 0038: if pos > 0 and license_pattern.match(elements[pos-1]): 0039: new_elements.append('&') 0040: element = '"' + element + '"' 0041: elif not license_operator.match(element): 0042: raise InvalidLicense(element) Exception: TypeError: 'filter' object is not subscriptable ERROR: Function failed: do_populate_lic ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.3275 ERROR: Task 9 (/media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb, do_populate_lic) failed with exit code '1 """ (Bitbake rev: c5de66b870406d9bd1161a9b7e2b04fe6eb065fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: fix BB_ENV_WHITELISTRobert Yang2013-03-191-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The BB_ENV_WHITELIST doesn't work well and flushes BB_ENV_EXTRAWHITE, here is an example: $ export BB_ENV_WHITELIST $ export BB_NUMBER_THREADS=10 (or other value) Edit conf/local.conf, change "BB_NUMBER_THREADS =" to "BB_NUMBER_THREADS ?=" $ bitbake -e | grep '^BB_NUMBER_THREADS =' we will notice that BB_NUMBER_THREADS' value doesn't change, though BB_NUMBER_THREADS in both BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE. This is because the "approved" inside the function approved_variables doesn't include BB_ENV_WHITELIST or BB_ENV_EXTRAWHITE when BB_ENV_WHITELIST is set (they are incuded by preserved_envvars()), so the BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE will be removed from the env in the first call from bin/bitbake, and when it is called again by cooker.py, their value will be None, then the vars inside them will be removed from the env. Add BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE to the "approved" would fix the problem. [YOCTO #4031] (Bitbake rev: d2b07e6516dd308d0045a7fdb72b588af9d676ad) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Improve remove funciton to handle whitespaceRichard Purdie2013-02-191-2/+2
| | | | | | | | | | | Improve the remove function to better handle cases where path contains special shell chars like whitespaces, '$', ';' or'\'. Thanks to Enrico Scholz for the fix. (Bitbake rev: 617511c9f86cc4ef52457653c8adff582d94bce3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Use rm -rf in remove()Richard Purdie2013-02-151-4/+7
| | | | | | | | | Whilst shutils.rmtree() is pythonic, its also slow. Its faster to use rm -rf which makes optimial use of the right syscalls. (Bitbake rev: 96088ebdec08e49ba9e8dbcac437bfcdc21f5983) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/utils: Improve environment handling to allow UIs access to ↵Richard Purdie2013-02-111-4/+6
| | | | | | | | | | | | | | | | original environment We need to empty out the environment whilst we build the cooker but we need the environment for the UIs since hob uses DISPLAY and other session variables. This patch adapts the utils functions to return removed environment components so we can reinject them for use by the UI, allowing hob to work again. (Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/utils: Drop custom 'interactive' variables list and store ↵Richard Purdie2013-02-061-26/+1
| | | | | | | | | | | | environment in BB_ORIGENV instead Maintining hardcoded lists of user environmental variables is hacky, replace these with the savedenv datastore. Allow access to that through the BB_ORIGENV variable. (Bitbake rev: 0a99563a4ea270594fd9a61da46f9387fb79dc66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: compat/utils: Add copy of python multiprocessing pool for pre 2.7.3 ↵Richard Purdie2013-02-061-0/+8
| | | | | | | | | | | | issues python 2.7 shows hangs with issues in its pool implmenetation. Rather than try and hack around these, add a copy of the working pool implementation to the compat module from 2.7.3. (Bitbake rev: c9eb742637131e8dbd526d2ad9b458abea0a2d87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Add function for processing profile outputRichard Purdie2013-01-281-0/+20
| | | | | | (Bitbake rev: 0df64810e8d40e7761cfd5059c0617dda31a6641) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Add explode_dep_versions2 to replace explode_dep_versionsRichard Purdie2012-10-021-6/+21
| | | | | | | | | | | | | | | | The API for explode_dep_versions is flawed since there can only be one version constraint against any given dependency. This adds a new function with an API without this limitation. explode_dep_versions() is maintained with a warning printed when its used in a situation where information is lost. This should allow a simple transition to the new API to fix the lost dependency information. join_deps() is updated to deal with data in either format. (Bitbake rev: babeeded21827d8d3e7c7b785a62332ee9d45d4f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Allow explode_dep_versions comparisons to have arbitrary ↵Mark Hatle2012-10-021-15/+47
| | | | | | | | | | | | | | | | | | | whitespace Refactor the explode_dep_versions to be more lenient on whitespace values. The required format is: foo (= 1.10) foo (=1.10) foo ( = 1.10) foo ( =1.10) foo ( = 1.10 ) foo ( =1.10 ) (Bitbake rev: 39c1c12c58fadd854098cf14ebe92f4d307a36dd) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Check for duplicate dependency entriesMark Hatle2012-10-021-1/+4
| | | | | | | | | | | | | | | | | | | | | | explode_dep_versions is not able to have duplicate entries. Previously duplicate entries ended up with the last item being the one returned to the caller. We now detect a collision. We do allow an empty item to have a comparison added to it, or a duplicate with the same comparison without error. When a collision is detected a ValueError exception is thrown. Allowed: foo foo (= 1.12) foo Invalid: foo (= 1.12) foo (= 1.13) (Bitbake rev: d40448f0483a2959e9dcaac9b6dd35839f396a6e) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Once we've printed errors messages, suppress the backtraceRichard Purdie2012-08-231-3/+5
| | | | | | | | | | Adding a generic backtrace to better_compile and better_exec is pointless, therefore reraise the exception as a bb.BBHandledException so the generic code doesn't confuse the user even more. (Bitbake rev: b3d97130e1e70fe969399277dcd7cccd888103d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Try harder to extract good traceback information by ↵Richard Purdie2012-08-231-4/+13
| | | | | | | | | | | | | | | | | | querying the datastore Currently as soon as execution passes outside the code fragment being executed by better_exec, we don't get any good traceback information, just a likely obscure reference to some function name which may or may not be identifiable. This patch adds code to query the datastore if present, allowing a more meaningful back trace to be displayed in many cases. [YOCTO #2981] (Bitbake rev: 0edf8431f9ff52581afe0d3ef525c59909af02ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/ast: Use better_exec instead of simple_execRichard Purdie2012-08-221-1/+3
| | | | | | | | | | | | This improves the stacktraces dumped by bitbake when for example anonymous python functions fail. Also default to passing code strings to better_exec to match the behaviour of simple_exec to aid the transition. (Bitbake rev: 7e8205929ae953731a6854ea80b197847cff5771) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils.py: remove unnecessary preserved variablesPaul Eggleton2012-06-251-2/+0
| | | | | | | | | | | | | | | | | | | | _ and LANG no longer need to be preserved from the external environment. The value of _ changes between non-pseudo-wrapped and pseudo-wrapped invocations (e.g. between "bitbake -p" and "bitbake target") and this will currently trigger a full reparse in the absence of a whitelist entry in BB_HASHCONFIG_WHITELIST, which is not ideal. LANG used to be preserved in order to ensure the C locale was being used for tools invoked by bitbake, however we now set LC_ALL in bitbake.conf to take care of this. Second part of the fix for [YOCTO #2600]. Acked-by: Jason Wessel <jason.wessel@windriver.com> (Bitbake rev: 1c531dff2fb055ecab2d462027eecec3fabc2a44) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Add function to set nonblocking operation on a file ↵Richard Purdie2012-06-251-0/+5
| | | | | | | | descriptor (Bitbake rev: ab6d71ebfcfb7bedc064b25f84647c8815096e5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Ensure utils.which() returns full pathsRichard Purdie2012-05-251-0/+2
| | | | | | | | | | | | If the path passed to which contains empty elements, it will search the current working directory for the file which is correct baheviour. Various pieces of code assume the path returned is a full path though. This commit ensures we don't return relative paths. (Bitbake rev: 4de24ccc10e40cc088b8515095df59f69b12715d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils: Convert vercmp_string() to use vercmp internallyRichard Purdie2012-05-041-124/+4
| | | | | | | | | | | | | | | Having two different version comparision algorithms in bitbake has never seemed like a sensible idea. Worryingly, they also return different results to each other. The vercmp_string API is relatively unused with no users in OE-Core or BitBake itself for example. This patch converts it to use vercmp internalls, bringing consitency to the comparisions which is easy now we have other recently added functions. Yes, this changes behaviour but in this case I'd prefer we were consistent than having two different comparisions. (Bitbake rev: a569c816e016447d60624c59a750709d59a0f455) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb.utils: Modifed vercmp() to meet Debian rules.Lianhao Lu2012-04-131-24/+16
| | | | | | | | | | | | | | | | The version compare function vercmp() was not exatcly conforming to Debian rules, e.g. it reported 'r1' > 'r1.1' but the Debian rules says 'r1' < 'r1.1'; it didn't support the "~" either. Modified the vercmp() to meet Debian rules, so that it's compatible to the rules used in opkg. This part of the buf fixing of [YOCTO #2233]. (Bitbake rev: 97b610c54c60b5a40fa7f6a09fa23ce17b38f93a) Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Add split_version function for splitting a version into its componentsRichard Purdie2012-03-201-0/+14
| | | | | | (Bitbake rev: b7b7c768761d205bc5a60922ff709c9c184d4158) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* command.py: add new API to get the cpu count on the serverShane Wang2012-02-231-0/+4
| | | | | | | | | Add a new API in command.py to get the cpu count in order to set the appropriate default BB_NUMBER_THREADS and PARALLEL_MAKE variables. (Bitbake rev: 335047b2e440e65713e88fabb24b47a9c82f939b) Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-02 22:43:58 +0000