linux/poky.git/meta/classes/externalsrc.bbclass, branch uninative-1.9

classes/externalsrc: ensure cleandirs code handles non-absolute paths

2018-04-05T13:49:07+00:00

It's possible that a trailing or extra slash somewhere in the external source path could result in the directory not being removed from cleandirs; it's also possible that a cleandirs entry is somewhere underneath the source tree and that tree should never have parts of it deleted by the build system. Use oe.path.is_path_parent() (which makes paths absolute before checking them) to find out if any path in cleandirs is anywhere underneath the external source path, and drop it if it is. (From OE-Core rev: 87ec0fb470274d980f8553a85f778809c269c5d7) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

classes/externalsrc: handle if cleandirs contains python expressions

2018-04-05T13:49:06+00:00

Use the existing oe.recipeutils.split_var_value() function to split the unexpanded value of the cleandirs varflag, in case it contains python expressions - we don't want to split the expression itself as the chunks will not expand properly individually and we can miss something that expands to the source tree (and thus it can get deleted, the avoidance of which is the whole point of this code). (From OE-Core rev: 611cee69d7300a4746edd752c9557af60e7beecc) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

externalsrc: do not call make clean for recipes with CLEANBROKEN = "1" set

2018-03-25T08:40:42+00:00

ERROR: distrho-ports-0.0.0+git999-r0 do_buildclean: oe_runmake failed ERROR: distrho-ports-0.0.0+git999-r0 do_buildclean: Function failed: do_buildclean ERROR: Logfile of failure stored in: <...>/temp/log.do_buildclean.17285 Log data follows: | DEBUG: Executing shell function do_buildclean | NOTE: make clean | make clean -C libs/drowaudio | make[1]: Entering directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio' | make clean -C build-drowaudio | make[2]: Entering directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio/build-drowaudio' | make[2]: *** No rule to make target 'clean'. Stop. | make[2]: Leaving directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio/build-drowaudio' | make[1]: *** [Makefile:7: clean] Error 2 | make[1]: Leaving directory '/home/a.mueller/data/oe-core/workspace/sources/distrho-ports/libs/drowaudio' | make: *** [Makefile:73: clean] Error 2 | ERROR: oe_runmake failed (From OE-Core rev: 21fa28d8faf33b2717e38886352238dd360ef346) Signed-off-by: Andreas Müller Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

externalsrc.bbclass: Suppress git errors

2018-02-16T18:05:40+00:00

Suppress any warnings git might generate when searching for a valid git directory, as there are use cases where the directory is expected to not exist and the warning is superfluous (From OE-Core rev: 66011996e1a8b738b31466fccad9973f8b48f71d) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

externalsrc: fix ExpansionError if the source dir does not exist yet

2017-11-08T22:24:03+00:00

The externalsrc class code assumes that the source directory (EXTERNALSRC) exists before bitbake is called. Otherwise do_configure will fail obviously since externalsrc does not fetch anything. Commit 3ca6085729d9 ("externalsrc: Handle .git not being a directory") changed this behaviour. Now on a missing EXTERNALSRC directory we get a bb.data_smart.ExpansionError during _parsing_, way before do_configure can be run. This new behaviour creates two problems: * First, there error message is very cryptic (and it's hard to provide a better message since no task is ever run): ERROR: ExpansionError during parsing /<...>/<...>.bb Traceback (most recent call last): bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception FileNotFoundError: [Errno 2] No such file or directory: '<...>' * Second, this prevents creating a class based on externalsrc that automatically fetches the code in EXTERNALSRC before do_compile runs. Fix both problems by simply calling git with '-C ${EXTERNALSRC}' instead of calling git inside the non-existing directory. This changes from a bb.data_smart.ExpansionError to a subprocess.CalledProcessError, which is in line with what's actually going on: git is telling us it can't find the git dir. Also remove a comment that does not apply anymore. (From OE-Core rev: 8e8251582e977d7c6ab525ac1b493fbbe3af38b2) Signed-off-by: Luca Ceresoli Cc: Joshua Watt Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

classes/externalsrc: delete symlinks at start of do_buildclean

2017-09-21T08:24:24+00:00

To help users find the work and log directories (especially within the eSDK) we create symlinks to these from the source tree. However during do_buildclean we call "make clean", and some project Makefiles which delete something like "*/*/lib" which will match files underneath the oe-workdir symlink and fail. do_buildclean is called from do_clean which is in turn called by devtool reset by default, and thus devtool reset is blocked. An example of a recipe where this is visible is the openssl-qat recipe in meta-intel. In order to fix this, delete the symlinks at the start of do_buildclean since we shouldn't need them at that point anyway, and thus make clean won't be able to trip over them. Fixes [YOCTO #11036]. (From OE-Core rev: 927b3b05a7f6b85967f7375b1942605fe8d37d0c) Signed-off-by: Paul Eggleton Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

externalsrc: Handle .git not being a directory

2017-08-15T23:03:14+00:00

Use git rev-parse to determine the location of the .git directory, in case it is not an immediate child of EXTERNALSRC (e.g. when using submodules). In the event git can't resolve the .git directory, fall back to the non-git method for hashing. (From OE-Core rev: 95e1341b49f7184d280a03f64f131a4468a06867) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

externalsrc: place copy of git index into /tmp and do not use copyfile2

2017-07-25T14:59:12+00:00

Using shutil.copy2() to copy .git/index to a temporary file tries to copy SELinux attributes which might fail for confined users in SELinux environments. E.g. our builders are running in docker containers and modification of sources (inclusive updated of .git/index) is done outside. Trying to copy .git/index fails with | $ python3 -c 'import shutil; shutil.copy2("index", "a")' | ... | PermissionError: [Errno 13] Permission denied: 'a' and an AVC like | denied { relabelto } for pid=18043 comm="python3" name="a" dev="dm-29" ino=1067553 scontext=system_u:system_r:container_t:s0:c39,c558 tcontext=unconfined_u:object_r:build_file_t:s0 tclass=file permissive=0 is created. This can not be solved by adapting the SELinux policy because this is a very deep constraint violation: | constrain file { create relabelfrom relabelto } ((u1 == u2 -Fail-) or (t1 == can_change_object_identity -Fail-) ); Constraint DENIED | | Possible cause is the source user (system_u) and target user (unconfined_u) are different. I do not see much sense in using 'shutil.copy2()' here; 'shutil.copyfile()' seems to be a better choice (target file is created in a secure way by tempfile.NamedTemporaryFile()). By placing the tempfile into /tmp we avoid potential problems related to git's 'core.sharedRepository'. As a (positive) side effect, the source tree will not be modified anymore (at least by this part of code) which prevented to mount it read-only from somewhere else. (From OE-Core rev: 3c3c8ecc61dfed68987750d79b5482ab2f6fa02f) Signed-off-by: Enrico Scholz Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

externalsrc: verify that EXTERNALSRC/EXTERNALSRC_BUILD are absolute paths

2017-06-14T13:53:58+00:00

If these are set to URLs then the errors produced are not helpful. (From OE-Core rev: 946b6623154e748a0d75ff498802a720aeec27a6) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

externalsrc: Use git add -A for compatibility with all git versions

2017-03-22T11:35:22+00:00

I've been debugging a selftest failure on Centos7. The problem turns out to be the elderly git version (1.8.3.1) on those systems. It means that the system doesn't correctly checksum changed files in the source tree, which in turn means do_compile fails to run and this leads to the following selftest failure: ====================================================================== FAIL [141.373s]: test_devtool_buildclean (oeqa.selftest.devtool.DevtoolTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/devtool.py", line 530, in test_devtool_buildclean assertFile(tempdir_mdadm, 'mdadm') File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/devtool.py", line 497, in assertFile self.assertTrue(os.path.exists(f), "%r does not exist" % f) AssertionError: False is not true : '/tmp/devtoolqag88s39z8/mdadm' does not exist The solution is to use -A on the git add commandline which matches the behaviour in git 2.0+ versions and resolves the problem. (From OE-Core rev: 964e8b8cae4b28e21ade12b5effb494e459b1f0f) Signed-off-by: Richard Purdie