linux/poky.git/bitbake/lib/bb/runqueue.py, branch yocto-2.3.1

bitbake: runqueue: Allow recrdeptask not to exist for all recipes

2017-04-10T22:00:32+00:00

Currently if you specify a recrdeptask, it must exist for all recipes or you get a python traceback. This is a bug and it should be possible to have recipes which don't have the specified task. As well as preventing such a traceback (which shouldn't happen, it should be a user readable error), this allows us to fix issues in OE-Core which would otherwise trigger the traceback. (Bitbake rev: f93a77f2f188e18de9e3d812e86d77c2f3c71889) Signed-off-by: Richard Purdie

bitbake: runqueue: minor typo fix

2017-03-15T17:38:33+00:00

(Bitbake rev: f2c151cd345788a135452f63f622a2bc0b10c2d4) Signed-off-by: Andre McCurdy Signed-off-by: Richard Purdie

bitbake: runqueue: add option to run all tasks in specific build target

2017-03-13T09:43:40+00:00

For example: $ bitbake core-image-minimal --runall unpack ... NOTE: Tasks Summary: Attempted 326 tasks of which 88 didn't need to be rerun and all succeeded. $ bitbake core-image-minimal --runall patch NOTE: Tasks Summary: Attempted 453 tasks of which 332 didn't need to be rerun and all succeeded. This can replace fetchall as well: $ bitbake core-image-minimal --runall fetch NOTE: Tasks Summary: Attempted 135 tasks of which 119 didn't need to be rerun and all succeeded. (Bitbake rev: 7c0fa6ba66cdb956b37d94055307cde857633df9) Signed-off-by: Matthew McClintock Signed-off-by: Richard Purdie

bitbake: runqueue: Fix collapsed setscene dependency tree

2017-02-19T14:28:47+00:00

When we removed the postinst sstate dependency handling code from setscene_depvalid, we noticed things being installed into the old style sysroot for rootfs tasks which should not have been there, causing a performance regression. Analysis revealed that setscene dependencies were "bubbling" over sstate tasks when they should have been stopping there. The 'continue' added by this patch avoids this issue and eusures sstate tasks remain contained to their specific chains. There was another bug in the code this exposed where the acconting for tasks as they were removed from sq_revdeps was not correct. In fixing this, what looks like a workaround in another test can then be simplified. After this change, populate_sysroot tasks are no longer depending on package_write_rpm tasks for example, which would make no sense. A before/after analysis of image dependencies only revealed improved dependencies after this change. Recipe specific sysroots did highlight the issue here since the behaviour of the sysroot dependencies (and processing with depvalid) was not matching what bitbake itself was doing, with bitbake being incorrect. Failures were 'safe' in that too many dependencies would get installed. (Bitbake rev: 5ef2cb50041fa7106c8de170af73d2a54cb0b1f0) Signed-off-by: Richard Purdie

bitbake: runqueue.py: revised completion scheduler

2017-01-20T11:52:52+00:00

The idea is that tasks which complete building a recipe (like do_package_qa) are more important than tasks which start building new recipes (do_fetch) or those which increase disk usage (do_compile). Therefore tasks get ordered like this (most important first, do_rm_work before do_build because the enhanced rm_work.bbclass was used): 1. ID /work/poky/meta/recipes-support/popt/popt_1.16.bb:do_build 2. ID /work/poky/meta/recipes-core/readline/readline_6.3.bb:do_build 3. ID /work/poky/meta/recipes-connectivity/libnss-mdns/libnss-mdns_0.10.bb:do_build ... 464. ID /work/poky/meta/recipes-sato/images/core-image-sato.bb:do_build 465. ID /work/poky/meta/recipes-graphics/xorg-proto/inputproto_2.3.2.bb:do_rm_work 466. ID /work/poky/meta/recipes-devtools/python/python3_3.5.2.bb:do_rm_work 467. ID /work/poky/meta/recipes-core/packagegroups/packagegroup-base.bb:do_rm_work ... 3620. ID virtual:native:/work/poky/meta/recipes-extended/pbzip2/pbzip2_1.1.13.bb:do_install 3621. ID /work/poky/meta/recipes-devtools/qemu/qemu-helper-native_1.0.bb:do_install 3622. ID /work/poky/meta/recipes-core/zlib/zlib_1.2.8.bb:do_compile_ptest_base 3623. ID /work/poky/meta/recipes-extended/bzip2/bzip2_1.0.6.bb:do_compile_ptest_base ... 3645. ID /work/poky/meta/recipes-support/libevent/libevent_2.0.22.bb:do_compile_ptest_base 3646. ID /work/poky/meta/recipes-core/busybox/busybox_1.24.1.bb:do_compile_ptest_base 3647. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_uboot_mkimage 3648. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_sizecheck 3649. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_strip 3650. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_compile_kernelmodules 3651. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_shared_workdir 3652. ID /work/poky/meta/recipes-kernel/linux/linux-yocto_4.8.bb:do_kernel_link_images 3653. ID /work/poky/meta/recipes-devtools/quilt/quilt-native_0.64.bb:do_compile 3654. ID /work/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_compile ... The order of the same task between different recipes is the same as with the speed scheduler, i.e. more important recipes come first. (Bitbake rev: 70e297e5c285ce0a02e9efd3117ff62cdc77ec12) Signed-off-by: Patrick Ohly Signed-off-by: Richard Purdie

bitbake: runqueue: Fix traceback when using -b

2017-01-20T11:52:52+00:00

Without this, bitbake -b of image recipes cause tracebacks since the list of providers is empty. (Bitbake rev: c53e43f3c6a675c0934a7a4e358fd66b049ffca3) Signed-off-by: Richard Purdie

bitbake: use multiple processes to dump signatures.

2017-01-17T11:29:28+00:00

This change significantly shortens the time on reparsing stage of '-S' option. Each file is reparsed and then dumped within a dedicated process. The maximum number of the running processes is not greater than the value of BB_NUMBER_PARSE_THREADS if it is set. The dump_sigs() in class SignatureGeneratorBasic is _replaced_ by a new dump_sigfn() interface, so calls from the outside and subclasses are dispatched to the implementation in the base class of SignatureGeneratorBasic. Fixes [YOCTO #10352] (Bitbake rev: 99d3703edd77a21770b366c6ad65a3c0f5183493) Signed-off-by: Jianxun Zhang Signed-off-by: Richard Purdie

bitbake: runqueue: Only start fakeroot workers when needed

2016-12-22T12:36:40+00:00

Fakeroot workers usually have dependencies that need to be ready before they can be started. Starting them as a block therefore doesn't work as the dependencies may or may not have been built. Therefore start the multiconfig fakeworkers individually upon demand. [YOCTO #10344] (Bitbake rev: ab15486c636e451ed5a62e561980946887901c3b) Signed-off-by: Richard Purdie

bitbake: runqueue: Ensure setscene tasks with overlapping stamps don't parallel execute

2016-12-22T12:36:40+00:00

In multiconfig, mutliple tasks can execute which share the same stamp file. These must not execute in parallel, the idea is the first should execute, the subsequent ones should see a valid stamp and get skipped. The normal task execution code has stamps code to handle this, this adds similar code to the setscene execute() function to handle the issue there too. (Bitbake rev: 937acf267fa9e45f538695b2cf8aa83232a96240) Signed-off-by: Richard Purdie

bitbake: runqueue: Fix setscene issues with multiconfig

2016-12-22T12:36:40+00:00

setscene was being entirelu skipped for multiconfig variants as the tasks were simply not being spotted. If the default config was also being built it masked the problem. When this was fixed by using taskfn instead of fn in lookups against dataCache, several other instances of this problem were highlighted. This goes through and corrects the setscene code to correclty use taskfn instead of fn in the appropriate places meaning setscene tasks for multiconfig now work correctly. (Bitbake rev: cead85cf219c07b40aeb7c0d948c25f0429136db) Signed-off-by: Richard Purdie