linux/poky.git/bitbake/lib/bb/utils.py, branch 1.5_M4.rc2

bitbake: utils: Fix multiprocessing pool reference

2013-06-14T21:01:09+00:00

Interestingly the previous version comparision was broken and we were always using the compatibility code, masking this bug. Oops. (Bitbake rev: d48e8bcb24e8fa5d4fd60fd2c9927a95976d8d8b) Signed-off-by: Richard Purdie

bitbake: compat/server/utils: Jettison pre python 2.7.3 workarounds

2013-06-14T16:26:30+00:00

Now we've moved to require python 2.7.3, we can jettison the compatibility workarounds/hacks for older python versions. (Bitbake rev: a51c402304f2080a76720f9b31d6dfdbed393bba) Signed-off-by: Richard Purdie

bitbake: cooker/cookerdata/utils: Improve context management

2013-05-24T09:34:52+00:00

The current execution context management for bitbake is ugly and the use of a global variable is nasty. Fixing that is hard, however we can improve things to start to establish an API for accessing and changing that context. This patch also adds in an explicit reset of the context when we reparse the configuration data which starts to improve the lifecycle of the data in setups like hob. (Bitbake rev: 6c3281a140125337fc75783973485e16785d05a1) Signed-off-by: Richard Purdie

bitbake: fetch2/utils: Clean up imports

2013-05-17T09:42:08+00:00

Move various random imports to the start of the modules as cleanup and avoid an import issue with bb.process on python 2.6. (Bitbake rev: aed4adfbe3a591ca4f8e41fb763c9f961bf2e6d5) Signed-off-by: Richard Purdie

bitbake: lib: Clean up various file access syntax

2013-05-09T21:28:04+00:00

Python 3 is stricter about how files are accessed. Specficially: * Use open(), not file() * Use binary mode for binary files (when checksumming) * Use with statements to ensure files get closed * Add missing file close statements (Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f) Signed-off-by: Richard Purdie

bitbake: utils: Improve better_compile error message

2013-05-09T13:06:52+00:00

Similarly to the better_exec improvements, improve the compile failure messages to be more user readable. (Bitbake rev: 9bc92d0210e13e4cc98727f6c9ec2f47c2221e77) Signed-off-by: Richard Purdie

bitbake: utils: Improve better_exec traceback handling

2013-05-09T13:06:52+00:00

The current bitbake tracebacks are hard to read/confusing and sometimes incomplete. This patch attempts to do better by: * Moving the note about the exact exception to the end to make things read in sequence * Merged the initial stack trace to become part of the code dump * Added handling for "/xxxx" file paths since we can load these files and include the data as part of the trace * Dropped the ERROR: prefix to every line, allowing the error messages to be spacially accosicated in the UIs * Moved the "From:" line to the top of each code block and ensured its present consistently With the complexity now in this funciton, I've added try/except wrapping around it to ensure we catch exceptions in the exception handler too. Example before: """ ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb: TypeError: 'filter' object is not subscriptable ERROR: The stack trace of python calls that resulted in this exception/failure was: ERROR: File "do_populate_lic", line 13, in ERROR: ERROR: File "do_populate_lic", line 6, in do_populate_lic ERROR: ERROR: File "license.bbclass", line 99, in find_license_files ERROR: ERROR: File "/media/build1/poky/meta/lib/oe/license.py", line 38, in visit_string ERROR: if pos > 0 and license_pattern.match(elements[pos-1]): ERROR: ERROR: The code that was being executed was: ERROR: 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) ERROR: 0010: copy_license_files(lic_files_paths, destdir) ERROR: 0011: ERROR: 0012: ERROR: *** 0013:do_populate_lic(d) ERROR: 0014: ERROR: [From file: 'do_populate_lic', lineno: 13, function: ] ERROR: 0002:def do_populate_lic(d): ERROR: 0003: """ ERROR: 0004: Populate LICENSE_DIRECTORY with licenses. ERROR: 0005: """ ERROR: *** 0006: lic_files_paths = find_license_files(d) ERROR: 0007: ERROR: 0008: # The base directory we wrangle licenses to ERROR: 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) ERROR: 0010: copy_license_files(lic_files_paths, destdir) ERROR: [From file: 'do_populate_lic', lineno: 6, function: do_populate_lic] ERROR: 0095: lic_files_paths.append((os.path.basename(path), srclicfile)) ERROR: 0096: ERROR: 0097: v = FindVisitor() ERROR: 0098: try: ERROR: *** 0099: v.visit_string(license_types) ERROR: 0100: except oe.license.InvalidLicense as exc: ERROR: 0101: bb.fatal('%s: %s' % (d.getVar('PF', True), exc)) ERROR: 0102: except SyntaxError: ERROR: 0103: bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True))) ERROR: [From file: 'license.bbclass', lineno: 99, function: find_license_files] ERROR: Function failed: do_populate_lic ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.17442 """ Example after: """ ERROR: Error executing a python function in /media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb: The stack trace of python calls that resulted in this exception/failure was: File: 'do_populate_lic', lineno: 13, function: 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) 0010: copy_license_files(lic_files_paths, destdir) 0011: 0012: *** 0013:do_populate_lic(d) 0014: File: 'do_populate_lic', lineno: 6, function: do_populate_lic 0002:def do_populate_lic(d): 0003: """ 0004: Populate LICENSE_DIRECTORY with licenses. 0005: """ *** 0006: lic_files_paths = find_license_files(d) 0007: 0008: # The base directory we wrangle licenses to 0009: destdir = os.path.join(d.getVar('LICSSTATEDIR', True), d.getVar('PN', True)) 0010: copy_license_files(lic_files_paths, destdir) File: 'license.bbclass', lineno: 99, function: find_license_files 0095: lic_files_paths.append((os.path.basename(path), srclicfile)) 0096: 0097: v = FindVisitor() 0098: try: *** 0099: v.visit_string(license_types) 0100: except oe.license.InvalidLicense as exc: 0101: bb.fatal('%s: %s' % (d.getVar('PF', True), exc)) 0102: except SyntaxError: 0103: bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF', True))) File: '/media/build1/poky/meta/lib/oe/license.py', lineno: 38, function: visit_string 0034: new_elements = [] 0035: elements = filter(lambda x: x.strip(), license_operator.split(licensestr)) 0036: for pos, element in enumerate(elements): 0037: if license_pattern.match(element): *** 0038: if pos > 0 and license_pattern.match(elements[pos-1]): 0039: new_elements.append('&') 0040: element = '"' + element + '"' 0041: elif not license_operator.match(element): 0042: raise InvalidLicense(element) Exception: TypeError: 'filter' object is not subscriptable ERROR: Function failed: do_populate_lic ERROR: Logfile of failure stored in: /media/build1/poky/build/tmp/work/i586-poky-linux/eglibc-initial/2.17-r3/temp/log.do_populate_lic.3275 ERROR: Task 9 (/media/build1/poky/meta/recipes-core/eglibc/eglibc-initial_2.17.bb, do_populate_lic) failed with exit code '1 """ (Bitbake rev: c5de66b870406d9bd1161a9b7e2b04fe6eb065fe) Signed-off-by: Richard Purdie

bitbake: utils.py: fix BB_ENV_WHITELIST

2013-03-19T13:56:03+00:00

The BB_ENV_WHITELIST doesn't work well and flushes BB_ENV_EXTRAWHITE, here is an example: $ export BB_ENV_WHITELIST $ export BB_NUMBER_THREADS=10 (or other value) Edit conf/local.conf, change "BB_NUMBER_THREADS =" to "BB_NUMBER_THREADS ?=" $ bitbake -e | grep '^BB_NUMBER_THREADS =' we will notice that BB_NUMBER_THREADS' value doesn't change, though BB_NUMBER_THREADS in both BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE. This is because the "approved" inside the function approved_variables doesn't include BB_ENV_WHITELIST or BB_ENV_EXTRAWHITE when BB_ENV_WHITELIST is set (they are incuded by preserved_envvars()), so the BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE will be removed from the env in the first call from bin/bitbake, and when it is called again by cooker.py, their value will be None, then the vars inside them will be removed from the env. Add BB_ENV_WHITELIST and BB_ENV_EXTRAWHITE to the "approved" would fix the problem. [YOCTO #4031] (Bitbake rev: d2b07e6516dd308d0045a7fdb72b588af9d676ad) Signed-off-by: Robert Yang Signed-off-by: Richard Purdie

bitbake: utils: Improve remove funciton to handle whitespace

2013-02-19T22:33:05+00:00

Improve the remove function to better handle cases where path contains special shell chars like whitespaces, '$', ';' or'\'. Thanks to Enrico Scholz for the fix. (Bitbake rev: 617511c9f86cc4ef52457653c8adff582d94bce3) Signed-off-by: Richard Purdie

bitbake: utils: Use rm -rf in remove()

2013-02-15T15:53:41+00:00

Whilst shutils.rmtree() is pythonic, its also slow. Its faster to use rm -rf which makes optimial use of the right syscalls. (Bitbake rev: 96088ebdec08e49ba9e8dbcac437bfcdc21f5983) Signed-off-by: Richard Purdie