summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/utils.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: utils: Improve remove funciton to handle whitespaceRichard Purdie2013-02-191-2/+2
| | | | | | | | | | | Improve the remove function to better handle cases where path contains special shell chars like whitespaces, '$', ';' or'\'. Thanks to Enrico Scholz for the fix. (Bitbake rev: 617511c9f86cc4ef52457653c8adff582d94bce3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Use rm -rf in remove()Richard Purdie2013-02-151-4/+7
| | | | | | | | | Whilst shutils.rmtree() is pythonic, its also slow. Its faster to use rm -rf which makes optimial use of the right syscalls. (Bitbake rev: 96088ebdec08e49ba9e8dbcac437bfcdc21f5983) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/utils: Improve environment handling to allow UIs access to ↵Richard Purdie2013-02-111-4/+6
| | | | | | | | | | | | | | | | original environment We need to empty out the environment whilst we build the cooker but we need the environment for the UIs since hob uses DISPLAY and other session variables. This patch adapts the utils functions to return removed environment components so we can reinject them for use by the UI, allowing hob to work again. (Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/utils: Drop custom 'interactive' variables list and store ↵Richard Purdie2013-02-061-26/+1
| | | | | | | | | | | | environment in BB_ORIGENV instead Maintining hardcoded lists of user environmental variables is hacky, replace these with the savedenv datastore. Allow access to that through the BB_ORIGENV variable. (Bitbake rev: 0a99563a4ea270594fd9a61da46f9387fb79dc66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: compat/utils: Add copy of python multiprocessing pool for pre 2.7.3 ↵Richard Purdie2013-02-061-0/+8
| | | | | | | | | | | | issues python 2.7 shows hangs with issues in its pool implmenetation. Rather than try and hack around these, add a copy of the working pool implementation to the compat module from 2.7.3. (Bitbake rev: c9eb742637131e8dbd526d2ad9b458abea0a2d87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Add function for processing profile outputRichard Purdie2013-01-281-0/+20
| | | | | | (Bitbake rev: 0df64810e8d40e7761cfd5059c0617dda31a6641) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Add explode_dep_versions2 to replace explode_dep_versionsRichard Purdie2012-10-021-6/+21
| | | | | | | | | | | | | | | | The API for explode_dep_versions is flawed since there can only be one version constraint against any given dependency. This adds a new function with an API without this limitation. explode_dep_versions() is maintained with a warning printed when its used in a situation where information is lost. This should allow a simple transition to the new API to fix the lost dependency information. join_deps() is updated to deal with data in either format. (Bitbake rev: babeeded21827d8d3e7c7b785a62332ee9d45d4f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Allow explode_dep_versions comparisons to have arbitrary ↵Mark Hatle2012-10-021-15/+47
| | | | | | | | | | | | | | | | | | | whitespace Refactor the explode_dep_versions to be more lenient on whitespace values. The required format is: foo (= 1.10) foo (=1.10) foo ( = 1.10) foo ( =1.10) foo ( = 1.10 ) foo ( =1.10 ) (Bitbake rev: 39c1c12c58fadd854098cf14ebe92f4d307a36dd) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Check for duplicate dependency entriesMark Hatle2012-10-021-1/+4
| | | | | | | | | | | | | | | | | | | | | | explode_dep_versions is not able to have duplicate entries. Previously duplicate entries ended up with the last item being the one returned to the caller. We now detect a collision. We do allow an empty item to have a comparison added to it, or a duplicate with the same comparison without error. When a collision is detected a ValueError exception is thrown. Allowed: foo foo (= 1.12) foo Invalid: foo (= 1.12) foo (= 1.13) (Bitbake rev: d40448f0483a2959e9dcaac9b6dd35839f396a6e) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Once we've printed errors messages, suppress the backtraceRichard Purdie2012-08-231-3/+5
| | | | | | | | | | Adding a generic backtrace to better_compile and better_exec is pointless, therefore reraise the exception as a bb.BBHandledException so the generic code doesn't confuse the user even more. (Bitbake rev: b3d97130e1e70fe969399277dcd7cccd888103d6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Try harder to extract good traceback information by ↵Richard Purdie2012-08-231-4/+13
| | | | | | | | | | | | | | | | | | querying the datastore Currently as soon as execution passes outside the code fragment being executed by better_exec, we don't get any good traceback information, just a likely obscure reference to some function name which may or may not be identifiable. This patch adds code to query the datastore if present, allowing a more meaningful back trace to be displayed in many cases. [YOCTO #2981] (Bitbake rev: 0edf8431f9ff52581afe0d3ef525c59909af02ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/ast: Use better_exec instead of simple_execRichard Purdie2012-08-221-1/+3
| | | | | | | | | | | | This improves the stacktraces dumped by bitbake when for example anonymous python functions fail. Also default to passing code strings to better_exec to match the behaviour of simple_exec to aid the transition. (Bitbake rev: 7e8205929ae953731a6854ea80b197847cff5771) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils.py: remove unnecessary preserved variablesPaul Eggleton2012-06-251-2/+0
| | | | | | | | | | | | | | | | | | | | _ and LANG no longer need to be preserved from the external environment. The value of _ changes between non-pseudo-wrapped and pseudo-wrapped invocations (e.g. between "bitbake -p" and "bitbake target") and this will currently trigger a full reparse in the absence of a whitelist entry in BB_HASHCONFIG_WHITELIST, which is not ideal. LANG used to be preserved in order to ensure the C locale was being used for tools invoked by bitbake, however we now set LC_ALL in bitbake.conf to take care of this. Second part of the fix for [YOCTO #2600]. Acked-by: Jason Wessel <jason.wessel@windriver.com> (Bitbake rev: 1c531dff2fb055ecab2d462027eecec3fabc2a44) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Add function to set nonblocking operation on a file ↵Richard Purdie2012-06-251-0/+5
| | | | | | | | descriptor (Bitbake rev: ab6d71ebfcfb7bedc064b25f84647c8815096e5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Ensure utils.which() returns full pathsRichard Purdie2012-05-251-0/+2
| | | | | | | | | | | | If the path passed to which contains empty elements, it will search the current working directory for the file which is correct baheviour. Various pieces of code assume the path returned is a full path though. This commit ensures we don't return relative paths. (Bitbake rev: 4de24ccc10e40cc088b8515095df59f69b12715d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils: Convert vercmp_string() to use vercmp internallyRichard Purdie2012-05-041-124/+4
| | | | | | | | | | | | | | | Having two different version comparision algorithms in bitbake has never seemed like a sensible idea. Worryingly, they also return different results to each other. The vercmp_string API is relatively unused with no users in OE-Core or BitBake itself for example. This patch converts it to use vercmp internalls, bringing consitency to the comparisions which is easy now we have other recently added functions. Yes, this changes behaviour but in this case I'd prefer we were consistent than having two different comparisions. (Bitbake rev: a569c816e016447d60624c59a750709d59a0f455) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb.utils: Modifed vercmp() to meet Debian rules.Lianhao Lu2012-04-131-24/+16
| | | | | | | | | | | | | | | | The version compare function vercmp() was not exatcly conforming to Debian rules, e.g. it reported 'r1' > 'r1.1' but the Debian rules says 'r1' < 'r1.1'; it didn't support the "~" either. Modified the vercmp() to meet Debian rules, so that it's compatible to the rules used in opkg. This part of the buf fixing of [YOCTO #2233]. (Bitbake rev: 97b610c54c60b5a40fa7f6a09fa23ce17b38f93a) Signed-off-by: Lianhao Lu <lianhao.lu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Add split_version function for splitting a version into its componentsRichard Purdie2012-03-201-0/+14
| | | | | | (Bitbake rev: b7b7c768761d205bc5a60922ff709c9c184d4158) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* command.py: add new API to get the cpu count on the serverShane Wang2012-02-231-0/+4
| | | | | | | | | Add a new API in command.py to get the cpu count in order to set the appropriate default BB_NUMBER_THREADS and PARALLEL_MAKE variables. (Bitbake rev: 335047b2e440e65713e88fabb24b47a9c82f939b) Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-3/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Nothing uses USERNAME, remove it - can cause sstate-cache conflictsMatthew McClintock2011-11-251-1/+0
| | | | | | | | | | USER is the correct variable to use, also this can affect sstate cache as well. (Bitbake rev: d7f9edda65dae2e046871afa275c5a51dff48fc4) Signed-off-by: Matthew McClintock <msm@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: Fix lockfile retry handlingRichard Purdie2011-11-101-1/+1
| | | | | | | | | | The lockfile retry parameter is expected to return immediately after attempting to take the lock. There was a bug in the logic which this patch fixed to ensure it does that. (Bitbake rev: f421ef819f00ac659504d9af41bcc8323422ff8c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Ensure only the filtered environment variables are inherited from the OSJoshua Lock2011-08-151-7/+16
| | | | | | | | | | | | | | | | | | The recent change which modified inheritFromOS to use the intial environment, rather than the current environment, introduced a bug such that variables which had been cleaned from the environment where still set in the data store. This patch changes things such that a list of approved environment variables is saved after the environment is cleaned and only the variables in this list are inherited in inheritFromOS. CC: James Limbouris <james.limbouris@gmail.com> CC: Chris Larson <clarson@kergoth.com> (Bitbake rev: cb6c07054e8baf94614713ec257c643b22266d75) Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/logging: Overhaul internal logging processRichard Purdie2011-08-151-15/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At the moment it bugs me a lot that we only have one effective logging level for bitbake, despite the logging module having provision to do more advanced things. This patch: * Changes the core log level to the lowest level we have messages of (DEBUG-2) so messages always flow through the core logger * Allows build.py's task logging code to log all the output regardless of what output is on the console and sets this so log files now always contain debug level messages even if these don't appear on the console * Moves the verbose/debug/debug-domains code to be a UI side setting * Adds a filter to the UI to only print the user requested output. The result is more complete logfiles on disk but the usual output to the console. There are some behaviour changes intentionally made by this patch: a) the -v option now controls whether output is tee'd to the console. Ultimately, we likely want to output a message to the user about where the log file is and avoid placing output directly onto the console for every executing task. b) The functions get_debug_levels, the debug_levels variable, the set_debug_levels, the set_verbosity and set_debug_domains functions are removed from bb.msg. c) The "logging" init function changes format. d) All messages get fired to all handlers all the time leading to an increase in inter-process traffic. This could likely be hacked around short term with a function for a UI to only request events greater than level X. Longer term, having masks for event handlers would be better. e) logger.getEffectiveLevel() is no longer a reliable guide to what will/won't get logged so for now we look at the default log levels instead. [YOCTO #304] (Bitbake rev: 45aad2f9647df14bcfa5e755b57e1ddab377939a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils: 'Fix' bb.utils.contains() behaviourRichard Purdie2011-07-261-7/+6
| | | | | | | | | | | | | | | | | Currently bb.utils.contains(X, "A", true, false) will return true for substring matches, e.g. if X = "ABC". This is not what most users expect from the function. In the common OE use of this function there is the case of "touchscreen" and "screen" being used as independent variables. Whilst it could be argued there isn't a problem in that specific case (touchscreens are usually on screens), there is no substring usage of this function is OE-Core so this patch changes the behaviour to match only full strings. It also fixes a bug where duplicate entries would confuse multiple matches, e.g. contains(X, ["A", "B"], ...) would match X = "A A" which is clearly wrong. (Bitbake rev: 3d8647b68a8e66c7b240ed5fed7406e1b78fabf6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils: Add contains helper function from lib.oe.utilsRichard Purdie2011-07-221-0/+14
| | | | | | | | | | | This function is needed by some of the early .conf setup we need to improve the machine/tune files in Openembedded. We need to add it here since the location in oe.utils can't be accessed until after base.bbclass parses which is too late for our needs. (Bitbake rev: abc67ed6921c98ed581f101ec1acc589fd9ce7e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Add option to lockfiles to return immediately rather than waitRichard Purdie2011-05-271-1/+5
| | | | | | | | | There are usecases where we don't want to block waiting for a lockfile so enhance the lockfile handling functions to support this. (Bitbake rev: 97e8adf03e5fab1fd40c3d53c48f7b333bc2e145) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: fix typo in lockfileChris Larson2011-03-311-1/+1
| | | | | | | (Bitbake rev: 53a10b6793c5bdb45854483abe5da791058dfd84) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lockfile: ask for forgiveness, not permissionChris Larson2011-03-311-7/+6
| | | | | | | | | | | | | | | | | | | | Create the lockfile directory if it doesn't exist, rather than erroring out if it doesn't exist (was also racy). Also improve the wording of the error message shown when the lockfile's directory is not writable. Note for the future, this function should be improved, particularly with regard to its exception handling. It should be catching the *exact* exception(s) it will encounter when the file is locked, and continuing in that case only. If it did that, there'd be no need for the proactive directory writability check, as bb.utils.lockfile() would raise an appropriate IOError for that case. (Bitbake rev: 238151441c74db53d6e4d4753f4f96c32f6f13b6) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Only try and add read access to a file if we don't have itRichard Purdie2011-02-251-4/+11
| | | | | | | | | A file we're copying might be on a readonly filesystem so if we can already read it, don't try and add read permission. Fixes BUGID #771 in Yocto. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Allow join_deps to return a list that isn't comman separatedRichard Purdie2011-02-251-2/+5
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: unpack revisionMark Hatle2011-02-121-0/+12
| | | | | | | | | | | | Revise the unpack function to have a way to disable the unpack. This is based on the work from "Andreas Oberritter <obi@opendreambox.org>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380 In addition, the to_boolean function comes from the work of "Chris Larson <chris_larson@mentor.com>", see http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54 Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
* Export KRB5CCNAME variableJavier Martin2011-02-101-0/+1
| | | | | | | | | | This allows fetching git repositories using Kerberos authentication. (Bitbake rev: d761cf98284b02eb3d3a1f879782c501c284b698) Signed-off-by: Javier Martin <javier.martin@vista-silicon.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: add glob name matching to removeSaul Wold2011-02-081-8/+9
| | | | Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/unlockfile: Fix exception handlingRichard Purdie2011-01-261-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils: Ignore OSError in unlockfileRichard Purdie2011-01-241-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Teach unlockfile about shared mode lockfilesRichard Purdie2011-01-191-1/+7
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Add option of holding shared lockfilesRichard Purdie2011-01-191-2/+6
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* *: use utils.remove() some moreBernhard Reutner-Fischer2011-01-121-1/+3
| | | | | | | (Bitbake rev: d3489b141cac1197324661680fe38b8a88bc49b4) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: show the actual exception in better_execChris Larson2011-01-101-6/+4
| | | | | | | (Bitbake rev: a148e6a63c842ac586ac1dddbd9008f93cdea297) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/utils.py: Drop unused function extend_deps()Richard Purdie2011-01-101-11/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync environment handling function names and comments with upstream ↵Richard Purdie2011-01-101-12/+12
| | | | | | alterations Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-8/+8
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* utils: fix typo in error messageBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 99cdb61b30d0c75c8f831c78346cc9f8ca7945dd) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* utils: don't overwrite builtin dir() functionBernhard Reutner-Fischer2011-01-041-2/+2
| | | | | | | (Bitbake rev: 6dc863f714beda6ca7ff8cd3e830a9bc8a39123d) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* utils: drop unnecessary debug msgs from mkdirhierChris Larson2011-01-041-2/+0
| | | | | | | (Bitbake rev: 4b36be445a05a544cd3a456acd1dcc8f962359e2) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* utils: fix 'lock' variable referenceChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: f57f8f3cc980e3ae1693c2e065227e951ed0b1c0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* utils: fix calls to close() in the lock/unlock functionsChris Larson2011-01-041-8/+9
| | | | | | | (Bitbake rev: 4262c26d36d1c1b6801ac422716a227c1f6b9589) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* build: use a contextmanager for locksChris Larson2011-01-041-0/+13
| | | | | | | | | | Also don't bother passing logfile to exec_func_python, at least until we start adding the logfile as a file handler to the bitbake logger. (Bitbake rev: f99ee4680c9f67b7ed13fc06044ba2382f9a782c) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* build: use bb.process instead of os.systemChris Larson2011-01-041-0/+11
| | | | | | | | | | | | (Bitbake rev: 53740977521bc81ffa37adfa7bbeb8f2a80ea165) build: write logfiles per task, not per function Based on d14f9bf6 from poky, reworked for master and other cleanup. (Bitbake rev: beadff2eca1eb95f0411115dd72ddb4c3c44c604) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-09 01:10:08 +0000