linux/poky.git/meta/classes/uninative.bbclass, branch uninative-2.1

uninative: Set the dynamic linker to use at compile time

2018-04-18T17:57:06+00:00

Its possible some dynamic runtime library in the dependency chain may come from sstate and link to libraries which need the libc from uninative. If we don't do this and binaries are run at do_install time they would fail to find the symbols from the later libc. Examples: cmake-native do_install: bin/cmake: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by TOPDIR/tmp/work/x86_64-linux/cmake-native/3.10.3-r0/recipe-sysroot-native/usr/lib/libexpat.so.1) dbus-native do_install: tmp/work/x86_64-linux/dbus-native/1.12.2-r0/build/bus/.libs/lt-dbus-daemon: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x32/build/build/tmp/work/x86_64-linux/dbus-native/1.12.2-r0/recipe-sysroot-native/usr/lib/libexpat.so.1) This issue is resolved when the interpreter is changed at sstate unpack time but this isn't soon enough to avoid issues at compile/install time. By specifing which dynamic linker/loader to use at compile time, this race window is removed entirely. (From OE-Core rev: 35867ee035030ab76fc9ccdb0eb1c3f80126301c) Signed-off-by: Richard Purdie

uninative: Add allow-shlib-undefined to BUILD_LDFLAGS and drop other workarounds

2018-04-18T17:57:06+00:00

We have a problem when for example, a glibc 2.27 based system builds some library like libpopt-native and puts it into sstate then it is reused on a pre glibc-2.27 system to build something which depends on popt like rpm-native. This results in an error like: recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27' In the past we've had this problem with new symbols like getrandom and getentropy, here its with a more complex symbol where there is an old version and a newer version. We've looked into various options, basically we cannot link against our uninative libc/ld.so since we don't have the right headers or compiler link libraries. The compiler doesn't allow you to switch in a new set either, even if we did want to ship them. Shipping a complete compiler, dev headers and libs also isn't an option. On the other hand if we follow the ld man page, it does say: """ The reasons for allowing undefined symbol references in shared libraries specified at link time are that: - A shared library specified at link time may not be the same as the one that is available at load time, so the symbol might actually be resolvable at load time. """ which is exactly this case. By the time the binary runs, it will use our uninative loader and libc and the symbol will be available. Therefore we basically have a choice, we get weird intermittent bugs, we drop uninative entirely, or we pass this option. If we pass the option, we can drop the other workarounds too. (From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a) (From OE-Core rev: d18bf7fa8e80d6cfaf3fdbe1ab06eec84b954432) Signed-off-by: Richard Purdie

uninative: add variables to the whitelist so that it does not re-triger recipe parsing

2018-04-03T22:53:20+00:00

When uninative is activated (poky's default) internal datastore variables are modified (NATIVELSBSTRING and SSTATEPOSTUNPACKFUNCS) to enable uninative support. This is happening after parsing is done at the beginning of the build. On the next bitbake call the recipe would be parsed if the two variables above were not added to the parsing whitelist BB_HASHCONFIG_WHITELIST. The fix is to add these two variables to the recipe parsing whitelist BB_HASHCONFIG_WHITELIST, this is done at recipe parsing time, only when uninative.bbclass is used. (From OE-Core rev: 75bb95ada98ef129d2fa48568f27dddb078c852c) Signed-off-by: Cuero Bugot Signed-off-by: Richard Purdie

uninative: Add compatiblity version check

2018-03-15T13:27:19+00:00

If glibc is newer on the host than in uninative, the failure mode is pretty nasty for clusters where the sstate is shared, including the Yocto Project autobuilder. This check aborts the use of uninative in such scenarios where a newer glibc version appears and avoids corruption of sstate caches. We use ldd to check the glibc version since that is included in libc-bin (or equivalent) which locales use so it should always be present. (From OE-Core rev: d6f6101cd0ae92e8ad2dec0bcb6db5044726edf9) Signed-off-by: Richard Purdie

uninative.bbclass: Fix broken symlink issue

2017-12-10T22:45:20+00:00

If two builds are sharing the same DL_DIR, and the uninative file is local to a layer. When the first build gets to uninative it creates the link local to itself, and subsequent users can use the same link. However if that first build then is deleted from the disk, the symlink is no longer valid (broken). We need to update the system to detect this case, and use the model implemented by the bitbke fetch2 code. Look for a broken link, remove it, then try to create the link and ignore an exception if it already exists (since we just unlinked any bad one). (From OE-Core rev: bfd9664edad7044b5da53fc33b8d0f6508f00950) Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie

uninative: Allow a local own-mirror to work with uninative

2017-08-03T10:14:13+00:00

The games we play with path manipulation of DL_DIR in uninative mean standard PREMIRRORS don't work and we can't easily put 'chksum' into the url path from a url parameter with the current fetcher url handling to make a generic remapping. We therefore add to PREMIRRORS when we know the chksum to create a premirror mapping which can work. [YOCTO #9888] (From OE-Core rev: 3b0309dbb5d52e9d7abf6fd89cc999ac4cc5d2fb) Signed-off-by: Richard Purdie

classes/uninative: set SSTATEPOSTUNPACKFUNCS[vardepvalueexclude] properly

2017-04-11T17:10:17+00:00

Append to the value with appendVarFlag() instead of setting it outright, so that we can also append to it in other places. Accordingly, this varflag is pipe-separated (since we want to be able to exclude any string fragment, in this case including the leading space), thus put a leading pipe character to play nicely with any existing value. (From OE-Core rev: a147838c3dfd4c53084a19b052b8d4e183293412) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

uninative.bbclass: split long command

2017-03-17T16:53:06+00:00

Single long commands are difficult to read and maintain. Split it to make it more human-friendly. (From OE-Core rev: c93b3b18f8daa8b419cc65b52cb5ceccfb1c142d) Signed-off-by: Leonardo Sandoval Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

uninative: remove unneeded call to sparse copy

2017-03-11T16:09:14+00:00

As of this commit: 39f5a05152aa0c3503735e18dd3b4c066b284107 patchelf no longer inflates file sizes. Since the files are no longer inflated by patchelf, we can skip using cp with the --sparse option. More details as to how patchelf has changed are available in that commit log. (From OE-Core rev: a2519a2f96bfea53a527c704e620e3584c97c67d) Signed-off-by: Stephano Cetola Signed-off-by: Richard Purdie

uninative: Make patchelf modified files sparse

2017-02-02T17:58:24+00:00

When we switched to recipe specific sysroots (rss), performance took a nose dive. Its easy to blame rss but it turns out not to be entirely at fault. Three configurations are compared here: a) Pre-RSS (revision 45df694a9f472ac2f684aadac4d864c3dfdc48a7) b) Post-RSS (revision 226a508da955439b881b2f0a544a3aee76e59919) c) as b) with this change Overall build times: a) 22794.25user 2687.88system 30:32.84elapsed 1390%CPU (0avgtext+0avgdata 919056maxresident)k b) 22677.25user 3238.79system 36:16.68elapsed 1190%CPU (0avgtext+0avgdata 918896maxresident)k c) 23571.84user 3383.65system 31:36.83elapsed 1421%CPU (0avgtext+0avgdata 919068maxresident)k For the overall build and sstate directories, du -s shows: a) 3992588 build-pre-rss/sstate-cache 30804484 build-pre-rss/tmp b) 4013272 build-with-rss/sstate-cache 36519084 build-with-rss/tmp c) 4014744 build-with-rss2/sstate-cache 35336960 build-with-rss2/tmp However more worryingly: $ du -s build-pre-rss/tmp/sysroots/ 2506092 build-pre-rss/tmp/sysroots/ $ du -s build-with-rss/tmp/sysroots-components/ 3790712 build-with-rss/tmp/sysroots-components/ $ du -s build-with-rss2/tmp/sysroots-components/ 2467544 build-with-rss2/tmp/sysroots-components/ These numbers *should* be equivalent but as you can see, b) is ~1.2GB larger. The reason turned out to be patchelf. Taking a specific binary from a specific recipe, bc from bc-native, in a) its 82kb (stripped) yet in b) its 2.17MB. $ ./patchelf --set-interpreter /bin/rp bc warning: working around a Linux kernel bug by creating a hole of 2084864 bytes in ‘bc’ https://github.com/NixOS/patchelf/blob/master/src/patchelf.cc#L710 shows that this "hole" is just padded zeros using memset, its not a proper sparse hole. This patch copies files with cp --sparse=always after modifying them with patchelf, then replacing the original file. The better fix will be to fix this in patchself itself and seek() there when writing the new file but that means new uninative tarballs and will take a bit of work so I'm proposing this workaround in the meantime. Also, this patch drops error handling since subprocess check_output() tracebacks will print this information if the command fails so we can simplify the code. Signed-off-by: Richard Purdie