summaryrefslogtreecommitdiffstats
path: root/meta
diff options
context:
space:
mode:
authorAndré Draszik <git@andred.net>2020-03-03 16:05:12 +0000
committerRichard Purdie <richard.purdie@linuxfoundation.org>2020-03-06 08:19:18 +0000
commitdec4e3f813533a87ac35ff0ac3b40c0e27083bfb (patch)
tree0f33681e5f81a3e86858786237b51ffb8dd0337b /meta
parente16134e3bb52c2fe5b754a3e36536935b476aaf2 (diff)
downloadpoky-dec4e3f813533a87ac35ff0ac3b40c0e27083bfb.tar.gz
bitbake.conf: omit XZ threads and RAM from sstate signatures
The number of threads used, and the amount of memory allowed to be used, should not affect sstate signatures, as they don't affect the outcome of the compression if xz operates in multi-threaded mode [1]. Otherwise, it becomes impossible to re-use sstate from automated builders on developer's machines (as the former might execute bitbake with certain constraints different compared to developer's machines). This is in particular a problem with the opkg package writing backend, as the OPKGBUILDCMD depends on XZ_DEFAULTS. Without the vardepexclude, there is no re-use possible of the package_write_ipk sstate. Whitelist the maximum number of threads and the memory limit given assumptions outlined in [2] below. (From OE-Core rev: 7927d5f6eaab48578b23db9a4c38e2f87f011353) Signed-off-by: André Draszik <git@andred.net> [1] When starting out in multi-threaded mode, the output is always deterministic, as even if xz scales down to single-threaded later, the archives are still split into blocks and size information is still added, thus keeping them compatible with multi-threaded mode. Also, when starting out in multi-threaded mode, xz never scales down the compression level to accomodate memory usage restrictions, it just scales down the number of threads and errors out if it can not accomodate the memory limit. [2] Assumptions * We only support multi-threaded mode (threads >= 2), builds should not try to use xz in single-threaded mode * The thread limit should be set via XZ_THREADS, not via modifying XZ_DEFAULTS or XZ_OPTS, or any other way * The thread limit should not be set to xz's magic value zero (0), as that will lead to single-threaded mode on single-core systems. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'meta')
-rw-r--r--meta/conf/bitbake.conf5
1 files changed, 4 insertions, 1 deletions
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 131ba296d3..4b544a22cd 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -795,7 +795,10 @@ BB_NUMBER_THREADS ?= "${@oe.utils.cpu_count()}"
795PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}" 795PARALLEL_MAKE ?= "-j ${@oe.utils.cpu_count()}"
796 796
797# Default parallelism and resource usage for xz 797# Default parallelism and resource usage for xz
798XZ_DEFAULTS ?= "--memlimit=50% --threads=${@oe.utils.cpu_count(at_least=2)}" 798XZ_MEMLIMIT ?= "50%"
799XZ_THREADS ?= "${@oe.utils.cpu_count(at_least=2)}"
800XZ_DEFAULTS ?= "--memlimit=${XZ_MEMLIMIT} --threads=${XZ_THREADS}"
801XZ_DEFAULTS[vardepsexclude] += "XZ_MEMLIMIT XZ_THREADS"
799 802
800################################################################## 803##################################################################
801# Magic Cookie for SANITY CHECK 804# Magic Cookie for SANITY CHECK