summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/utils.py
Commit message (Collapse)AuthorAgeFilesLines
* lib: Add copyright statements to files without oneRichard Purdie2022-08-121-0/+2
| | | | | | | | | Where there isn't a copyright statement, add one to make it explicit. Also add license identifiers as MIT if there isn't one. (From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: remove the ThreadedPoolJose Quaresma2022-07-151-64/+0
| | | | | | | | | | | | | | | The ThreadedPool in OE-core is mainly because python2 doesn't have threaded pools but python2 is dead for some time now and python3 have a ThreadPoolExecutor. The only local in OE-core where this ThreadedPool is in use is on the sstate.bbclass that is ported to the python3 ThreadPoolExecutor. (From OE-Core rev: aa8fd5e7c2a1125895accfd55ce9320819a10959) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Improve internal variable namingSaul Wold2022-03-101-2/+2
| | | | | | | | | Update internal variable names to improve the terms used. (From OE-Core rev: f408068e5d7998ae165f3002e51bc54b380b8099) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* prelink: Drop support for itRichard Purdie2022-02-051-11/+0
| | | | | | | | | | | | | | Prelink is being dropped by glibc in 2.36. It already causes issues with binary corruption, has a number of open bugs and is of questionable benefit without disabling load address randomization and PIE executables. We disabled it by default a while back but left people able to use it. We would be unable to maintain it alone without glibc support so remove the remaining pieces. (From OE-Core rev: 23c0be78106f1d1e2bb9c724174a1bb8c56c2469) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: by default cap cpu_count() to 64 coresRoss Burton2021-11-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Larger systems may have large numbers of cores, but beyond a certain point they can't all be used for compiling: whilst purely compute-intensive jobs can be parallelised to hundreds of cores, operations such as compressing (needs lots of RAM) or compiling (lots of I/O) don't scale linearly. For example, the Marvel ThunderX2 has 32 cores, each capable of executing four threads, and can be configured with two sockets, making 256 CPUs according to Linux. Zstd using 256 threads has been seen to fail to allocate memory during even small recipes such as iso-codes. Add a default cap of 64 CPUs to the cpu_count() method so that extreme parallisation is limited. 64 is high enough that meaningful gains beyond it are unlikely, but high enough that most systems won't be effected. (From OE-Core rev: 765d0f25ce48636b1838a5968e2dc15de2127428) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: allow naming threads in ThreadedPoolRoss Burton2021-11-251-6/+5
| | | | | | | | | | When looking at logs involving thread pools it is useful if the threads can be named. (From OE-Core rev: 18342945b021608794d83ecf567afd43f4379b24) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: log exceptions in ThreadedWorker functionsRoss Burton2021-10-181-1/+2
| | | | | | | | | | | | | | If the function a ThreadedWorker is executing raises an exception, don't use print() as that mostly disappears. Instead, output it to the logger. This is done using bb.mainlogger.debug directly instead of bb.debug() as this allows us to pass the exception instance directly, which is then incorporated into the log stream. (From OE-Core rev: 2f1ea25c222b344dd8b784b2bc73a6540ab30274) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory: Label packages providing per-file dependencies in depends.dotAndres Beltran2021-09-011-2/+30
| | | | | | | | | | | | | | | | Currently, depends.dot includes per-file dependencies but not the packages providing those files. This makes it hard to obtain all package dependencies by just looking at depends.dot. Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of their values to the package providing the component. Include runtime packages as dependencies in depends.dot, together with the component provided by the package as a label. (From OE-Core rev: 2ba33093017574bbe29eeba699eb90628614d03a) Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: add directory size functionRoss Burton2021-03-281-0/+31
| | | | | | | | | | | | | | | | | | | For the purpose of image construction using du on a rootfs directory isn't entirely satisfactory. Bare "du" will report the actual disk usage so file systems which can compress the data will report less than the actual space required. Using "du --apparent-size" will report the actual space used, but as this simply sums the bytes used for content across an entire file system can result in significant under-reporting due to block size overhead. Attempt to solve these problems by implementing our own function to calculate how large a rootfs will be. This function handles hardlinks correctly but rounds up all sizes to multiples of the block size (currently, 4KB is the hard-coded block size). (From OE-Core rev: 6ca53ad7b26ee2b4e6d2c121c6f6d6eed7f6b56f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Return empty string in parallel_makeTomasz Dziendzielski2021-01-161-1/+1
| | | | | | | | | | | | | In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make function and if PARALLEL_MAKE is set to empty string then this variable is exported as "None" causing cmake to fail with: "'CMAKE_BUILD_PARALLEL_LEVEL' environment variable invalid number 'None' given." (From OE-Core rev: 2f790ded554a52ac18d1c28002142f9c62abec8b) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: respect scheduler affinity in cpu_count()Ross Burton2020-09-301-2/+1
| | | | | | | | | | | | | | | cpu_count() returns multiprocessing.cpu_count() but that is simply returns os.cpu_count() so we could use that directly. However this returns the number of CPUs on the host, not the number of usable CPUs on the host. If the user is using scheduler affinity then the number of usable CPUs may be less, so when determining how many cores we can use check the affinity instead. (From OE-Core rev: e2e8ccbe410b2f38bcd9525982b2261cf71aab60) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils.py: add get_host_compiler_version()Tim Orling2020-04-011-0/+31
| | | | | | | | | | | | | | Add helper function to get the host compiler and version. Do not assume compiler is gcc. NOTE: cannot set env to d.getVar("PATH") as that does not contain the session PATH which was set by environment-setup-... which breaks the install-buildtools use-case (From OE-Core rev: 88712929354ff9c876bb1e48b6f15c33af5f2bbc) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: fix gcc 10 version detectionCharles-Antoine Couret2020-03-291-1/+1
| | | | | | | | | | | | Utils can not detect GCC 10 correctly due to wrong regex. It generates this error "ERROR: Can't get compiler version from gcc --version output" Sub-version numbers should be 1 or more digits instead of 1 only. (From OE-Core rev: 186fe4a3d390a52b87282c3e694ce3251e45ee78) Signed-off-by: Charles-Antoine Couret <charles-antoine.couret@mind.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: allow to set a lower bound on returned cpu_count()André Draszik2020-03-061-2/+3
| | | | | | | | | | | | | | | | | | | This will be needed for making xz compression more deterministic, as xz archives are created differently in single- vs multi-threaded modes. This means that due to bitbake's default of using as many threads as there are cores in the system, files compressed with xz will be different if built on a multi-core system compared to single-core systems. Allowing cpu_count() here to return a lower bound, will allow forcing xz to always use multi-threaded operation. (From OE-Core rev: c6dda204e15c5b27559f7617b18a7b3ce604e2f8) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cmake: prefer CMAKE_BUILD_PARALLEL_LEVELDaniel McGregor2020-02-041-4/+7
| | | | | | | | | | | | cmake 3.12 introduced this environment variable. Prefer it to passing PARALLEL_MAKE and PARALLEL_MAKEINST on the cmake command line, because it gets passed to second stage cmake invocations while command-line arguments do not (for example, multi-stage clang builds) (From OE-Core rev: cdd44c93f02bb8cc2fa773e13c8ce36e3da23921) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Remove bb.build.FuncFailedRichard Purdie2019-08-061-1/+1
| | | | | | | | | | | | | Whilst seemingly a good idea, this exception doesn't really serve any purpose that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't pythonic. Its not used in many places, lets clean up those and remove usage of it entirely. It may ultimately be dropped form bitbake entirely. (From OE-Core rev: efe87ce4b2154c6f1c591ed9d8f770c229b044ad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-091-0/+4
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils/multiprocess_launch: Improve failing subprocess outputRichard Purdie2019-04-261-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Output before this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.: Traceback (most recent call last): File "/media/build1/poky/meta/lib/oe/utils.py", line 272, in run ret = self._target(*self._args, **self._kwargs) File "/media/build1/poky/meta/classes/package_ipk.bbclass", line 230, in ipk_write_pkg shell=True) File "/usr/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Note how stdout/stderr from the failing command isn't shown. After this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:Foobar *** Error: Package name Foobar contains illegal characters, (other than [a-z0-9.+-]) opkg-build: Please fix the above errors and try again. We suddenly get a much more usable error message. The traceback is supressed as its distracting from the real problem in this case. Ideally python itself would handle this but it doesn't so we have to wrap the exception. We already do this in bitbake itself for the same reason. (From OE-Core rev: 09276dc76a8bda237b0b0b6d117a1980ae9dbfcc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Make prune_suffix prune a suffixAndre Rosa2019-04-091-3/+3
| | | | | | | | | | | ... instead of replacing a substring that could happen more than once and not only when it ends with it. Do the same for the prefix. See related https://github.com/openembedded/bitbake/pull/24 . There it stops replacing sufixes once first one is matched but not here. (From OE-Core rev: 610ac84170f8a91cc3321edfc336a9e39f24ebe3) Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: added sh_quote() functionEnrico Scholz via Openembedded-core2019-04-041-0/+3
| | | | | | | | | | | This function is a wrapper around "shlex.quote()" and can be used in "${@...}" context where shlex (or pipes, which provides similar functionality) is unavailable. (From OE-Core rev: 127141f5023a7e3fc3963dc7d76cfce9067a9e8a) Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Fix hang in multiprocess_launch()Richard Purdie2019-02-171-0/+4
| | | | | | | | | | | | | | If large results values are returned by the subprocesses, we can hit a deadlock where the subprocess is trying to write data back to the parent, the pipe is full and the parent is waiting for the child to exit. Avoid this by calling the update() method which would trigger reading a result from the child, avoiding the deadlock. The issue is described in https://bugs.python.org/issue8426 (From OE-Core rev: 0035e8066ecbbff94d6a1994a9f72b1368d660d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: add -src to system package name blacklistRoss Burton2019-02-151-1/+1
| | | | | | | | | | | oe.utils.packages_filter_out_system() returns PACKAGES after removing "system" packages but it doesn't handle ${PN}-src as generated by PACKAGE_DEBUG_SPLIT_STYLE=debug-with-srcpkg. (From OE-Core rev: 162632d3d1e40c83ed9c5d49a026edf3912860a0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-2/+2
| | | | | | | | | | | | | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. Note that some show up as: """ meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.   """ where the problem isn't on 1293 in package.bbclass but in some _prepend to a package.bbclass function in a different file like mesa.inc, often from do_package_split() calls. (From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ccache.bbclass: Refactor it to make it more reliableRobert Yang2019-01-141-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous ccache.bbclass has the following problems: - It uses host's ccache for native recipes, but this may not work on some hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always build failures (m4-native failed at do_configure, and others will also be failed if I disable CCACHE for m4-native) - native/nativesdk/cross/crosssdk recipes use host's ccache, but target uses ccache-native, this may confuse user. - The target recipes may use both host's ccache and ccache-native, this may cause unexpected problems and be hard to debug. This is because ccache-native is in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when rebuild target recipes, and then it would use hosttools/ccache, but the previous ccache files were generated by ccache-native. - Target recipes can't use ccache when no ccache is installed on the host: CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}" After refactored: All types recipes (native, target and others) will use ccache-native except ccache-native itself, host's cache won't be used any more. It is more reliable now, which will work everywhere when ccache-native can be built. And now we need use "CCACHE_DISABLE = '1'" to disable ccache for the recipe rather than "CCACHE = ''" since we set CCACHE in anonymous function, and d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we can't check whether CCACHE is set or not in anonymous function since it is always set. Use CCACHE_DISABLE to disable it would be more clear. (From OE-Core rev: b25271b65262f70d849a4861da216c9be6c54d53) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Set stderr for host_gcc_version()Robert Yang2018-12-141-1/+2
| | | | | | | | | | | | | | | | | | | Fixed: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc $ rm -fr tmp/hosttools/ && bitbake -p [snip] ERROR: Error running gcc --version: It didn't print the error message, now it is: ERROR: Error running gcc --version: ccache: error: Could not find compiler "gcc" in PATH For the error itself, it is because ccache is not in my HOSTTOOLS, so this is an expected error. (From OE-Core rev: 91955caae584b4f75118e04411851b1a3d783fec) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Improve multiprocess_lauch exception handlingRichard Purdie2018-12-051-2/+3
| | | | | | | | | | | | | | We've seen a cryptic: "ERROR: Fatal errors occurred in subprocesses, tracebacks printed above" message from oe-selftest with no other traceback information. Improve the traceback logging to try and give a better indication of any errors that is ocurring. (From OE-Core rev: 521dd3d00979a27b6932e58d5497de68abac26e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: add eol to format_pkg_list()grygorii tertychnyi via Openembedded-core2018-10-111-1/+7
| | | | | | | | | | | | | | | | | Append '\n' to the non-empty formatted string before return. If you write it to the (manifest) file, it will ensure file ends with a newline. Many GNU utilities have problems processing the last line of a file if it is not '\n' terminated. E.g. if the last line is not terminated by a newline character, then "read" will read it but return false, leaving the broken partial line in the read variable(s). It can also break or adversely affect some text processing tools, that operate on the file. (From OE-Core rev: ee4d0c879713ba50dc6cc3300f44647faebee2e0) Signed-off-by: grygorii tertychnyi <gtertych@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils.py: Add vartrue()Robert Yang2018-09-131-0/+8
| | | | | | | | | | It can be used to simplify code like: "${@['iffalse', 'iftrue'][var]}" (From OE-Core rev: fc5a5af7bc3619f575988a75efc0c4fe15478b2d) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* populate_base_sdk: Stop running gcc --version all the timeRichard Purdie2018-09-051-1/+4
| | | | | | | | | | Running 'gcc --version' for every image recipe is slow and increases parsing time/resource usage for no good reason. Only compute the value in when we're really running the task/function. (From OE-Core rev: bf49316bb9913b7c89de64d6a194be31aa66e16b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Fix get_multilib_datastore to work for original tuneRichard Purdie2018-08-231-0/+3
| | | | | | | | | | | Currently the original datastore returned by this function doesn't always work as the tune isn't set back to the original. Fix it to work like all_multilib_tune_list() in utils.bbclass and correct the data returned. (From OE-Core rev: 4e1dc858fbf671ef27089a2b9bcdc965fe19d698) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Drop now unused multiprocess_execRichard Purdie2018-07-241-34/+0
| | | | | | (From OE-Core rev: f3b30def2cd1c9ede7630489c3949a45b6eba6ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Add multiprocess_launch API and testcaseRichard Purdie2018-07-241-0/+70
| | | | | | | | | | | | | | | | | | | | | The current methods of spawning processes for parallel execution have issues around collection of results or exceptions. Take the code from package_ipk/deb, make it generic, add a results collection mechanism, fix the exception handling and for it into a standard library function. Also add a test case which tests both the success and failure modes of operation to stop this functionality regressiing again. In particular, compared to multiprocess_exec, this fork off the parent approach means we can pass in the datastore and functions work in the same scope as the parent. This removes some of the complexities found trying to scale multiprocess_exec to wider use. (From OE-Core rev: 88f0c214e593a45566df5131bda4c946f5ccc8c2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Improve get_multilib_datastoreRichard Purdie2018-07-021-3/+9
| | | | | | | | | | | | | Currently this function assumes that no multilib is applied and that we're applying a multilib. This means if we're in multilib context and want the non-multilib context we can't obtain it (and no other function exists for this either). Improve the function to allow this to be requested. (From OE-Core rev: 295c5a3d19834a2fac255346d0a373449cfdd776) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: Fix parallel_make limitJoshua Watt2018-02-241-1/+1
| | | | | | | | | | | parallel_make_argument() was incorrectly taking the maximum of the limit and the calculated value instead of the minimum. (From OE-Core rev: 45205be547967c84dff96403c3a6825a62e3ca6a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: add parallel make helpersJoshua Watt2018-02-161-0/+43
| | | | | | | | | | | | | | | | | | | The code to extract the integer number of parallel build threads and construct a new argument from them has started to be copied in multiple locations, so create two new helper utilities to aid recipes. The first helper (parallel_make()) extracts the integer number of parallel build threads from PARALLEL_MAKE. The second (parallel_make_argument()) does the same and then puts the result back into a format string, optionally clamping it to some maximum value. Additionally, rework the oe-core recipes that were manually doing this to use the new helper utilities. (From OE-Core rev: ccd1142d22b31ed85d8823b1bc9e11ccfd72b61f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Make get_multilib_datastore available from lib/oeRichard Purdie2018-01-041-0/+8
| | | | | | | | | Currently we can't access this function from lib/oe as its a class function. Move it to allow such access. (From OE-Core rev: b241a666f2867ffa425f6d43763d7c3c17941dcf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: remove param_bool()Ross Burton2017-12-021-11/+0
| | | | | | | | | | This function is not used by any classes or recipes that I can find, so lets delete it. (From OE-Core rev: a7cd9d1183be603777fc9c8c448281fe01224f7b) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Handle exceptions in multiprocess_execRichard Purdie2017-09-021-8/+13
| | | | | | | | | | | | Currently exceptions that happen in pool commands are ignored. Any errors would be printed on the console but everything else is silent. Switch to use pool.map_async which allows for an error_callback which we can use to detect exceptions and make sure these errors are handled. (From OE-Core rev: 7f2f9b3ff011b340b5d23bb7c47b12c357dc9f02) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils.py: helper function for optional include filesPatrick Ohly2017-06-121-0/+40
| | | | | | | | | | | | | | | | | | | | | | The main intention is to provide easy-to-use and read helper functions for including files only when certain distro features are set. Functionally they are the same as bb.utils.contains and bb.utils.contains_any. Distro features are part of the base configuration and thus safe to use for conditional inclusion in recipes and bbappends, in contrast to recipe variables which might still change during parsing. Therefore the check is limited to DISTRO_FEATURES. This is the reason for having this in OE-core instead of bitbake. Default values are set so that no redundant parameters have to be passed for conditional includes. As a secondary usage, the functions can also be used in boolean checks. (From OE-Core rev: 13024ce5aae453769b546d5fbe533443aec3d6fd) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Add build_depends_string functionRichard Purdie2017-01-201-0/+4
| | | | | | | | | This is useful when manipulating depends strings for task [depends] flags and is slightly easier to parse than some inline python. (From OE-Core rev: 7b05ea65a8db8a27b2a5579675775ee34ceb63c2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie2017-01-091-1/+1
| | | | | | | | | | | | | | | | | There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). (From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/utils: Drop python2 compatibility codeRichard Purdie2016-12-221-7/+2
| | | | | | | | | We've moved to python3, we don't need this compatibility code which just makes the code less readable. (From OE-Core rev: 425afe2484707640ac71194885fdb263e95e9950) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Always use datastore's PATH for host_gcc_versionRoss Burton2016-12-161-4/+8
| | | | | | | | | | | BUILD_CC may reference something like ccache and expect this to come from ccache-native, we at least have some selftests which assume this. Modify the code to use PATH when runnig BUILD_CC to ensure the tests continue to work as expected. (From OE-Core rev: f3e753372baac43d0921186340cf260df056de20) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: remove True option to getVar callsJoshua Lock2016-12-161-15/+15
| | | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* uninative: rebuild uninative for gcc 4.8 and 4.9Ed Bartosh2016-12-161-0/+14
| | | | | | | | | | | | | | | | | | | Some c++ libraries fail to build if uninative is built with gcc 5.x and host gcc version is either 4.8 or 4.9. The issue should be solved by making separate uninative sstate directory structure sstate-cache/universal-<gcc version> for host gcc versions 4.8 and 4.9. This causes rebuilds of uninative if host gcc is either 4.8 or 4.9 and it doesn't match gcc version used to build uninative. [YOCTO #10441] (From OE-Core rev: d36f41e5658bbbb6080ee833027879c119edf3e0) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: cleanup d.getVar(var, 1)Robert Yang2016-09-141-2/+2
| | | | | | | | (From OE-Core rev: 79fe476be233015c1c90e9c3fb4572267b5551d1) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image: add do_image_qa task to run QA checks on the constructed imageJoshua Lock2016-07-201-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This task runs all functions in IMAGE_QA_COMMANDS after the image construction has completed in order to validate the resulting image. Image sanity checks should either be Python functions which raise bb.build.FuncFailed on failure or shell functions with return a non-zero exit code. Python functions may instead raise an oe.utils.ImageQAFailed Exception which takes an extra argument, a description of the failure. python image_check_python_ok () { if True: raise bb.build.FuncFailed('This check always fails') else: bb.note("Nothing to see here") } image_check_shell_ok () { if true exit 1 else exit 0 fi } [YOCTO #9448] (From OE-Core rev: c9bef2ecf1a30159d11781184829f41844a58c13) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Complete transition to python3Richard Purdie2016-06-021-3/+3
| | | | | | | | | This patch contains all the other misc pieces of the transition to python3 which didn't make sense to be broken into individual patches. (From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Update to explictly create lists where neededRichard Purdie2016-06-021-3/+3
| | | | | | | | | Iterators now return views, not lists in python3. Where we need lists, handle this explicitly. (From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Update to modern exception syntaxRichard Purdie2016-05-211-1/+1
| | | | | | | | | Update older exception syntax to modern one required by python 3. Compatible with python 2.7. (From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-11 21:39:52 +0000