summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2: Fix os.errno referencesthudRichard Purdie2021-01-311-2/+2
| | | | | | | | | | os.errno used to happen to work but is invalid. Correct to use errno. [YOCTO #13068] (Bitbake rev: b3fc65289d33274cd5dace4d4ffe55be11c991f4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemu: Replace stime() API with clock_settimeKhem Raj2020-09-162-0/+63
| | | | | | | | | (From OE-Core rev: 2cca75155baec8358939e2aae822e256bed4cfe0) (From OE-Core rev: 1351f9be973cfbd043f9b10d218e3cecaa0ab372) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Adding back wrapper and using OEPYTHON3HOME variable for python3Jaewon Lee2020-09-162-1/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding back the python wrapper and adding a patch to use OEPYTHON3HOME instead of PYTHONHOME if set, for python3. If we add back the wrapper as is, we would see the following error that we also see in Thud: ImportError: No module named site OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python v3. Please upgrade your python v2 This is because python3 would've set PYTHONHOME to use nativesdk python3 libraries but when the oe-buildenv-internal script tries to call python2 for the py_v27_check, there will be no python2 libraries in the PYTHONHOME directory. In other words, bitbake needs host python2 and the env variable set from the wrapper contaminates the env and host python2 won't be able to find its libraries Creating another variable OEPYTHON3HOME and using this in the python3 wrapper to allow for a way to set a different paths for python3 and python2 [YOCTO #13208] (From OE-Core rev: 75d2a85e24ef9a2bf0e218521944523f0ff281e0) (From OE-Core rev: b29e87376fdd49ce07749b87c3000033fa96e43f) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* iso-codes: switch upstream branch master -> mainHongxu Jia2020-09-161-1/+1
| | | | | | | | | | | | (From OE-Core rev: a6e098e2e5932781b9c1012825bc86bc08382931) (From OE-Core rev: 54690c51765d3071406ef1bfd81c9d9db9552108) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6e16ef0c2e0ec2bbb862231cd84e7650bd5789af) Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update: handle baseMetricV2 as optionalKonrad Weihmann2020-09-161-4/+9
| | | | | | | | | | | | | Currently in NVD DB an item popped up, which hasn't set baseMetricV2. Let the parser handle it as an optional item. In case use baseMetricV2 before baseMetricV3 (From OE-Core rev: 77f119baf6f4b85194a9b26d8442ddc7fb3bb97c) (From OE-Core rev: 4cee5c4bc74edde48fe19ec11c78f6c598cf08b6) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/signing: Ensure build path relocation is safeRichard Purdie2020-09-161-1/+3
| | | | | | | | | | | Similarly to 04ee0e8b95cd8ed890374e0007f976684206b630, ensure only full build paths are replaced in the environment to avoid breaking buildtools. (From OE-Core rev: be07d93a4f59d4563f2d064be1997b39f05e9f0e) (From OE-Core rev: 7a46226288179df565b7c21c3316672d2e2a1ac0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* maintainers: Add entry for buildtools-extended-tarballRichard Purdie2020-09-161-0/+1
| | | | | | | | | | | | | (From OE-Core rev: 4281342a04078990bb0a110760ff2dc053eccc93) (From OE-Core rev: 665ef4274e0261bb8351c8d4fd2c8496a2dc27e7) (From OE-Core rev: 848af99b4e6afda0658db44128a6921217653e95) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 61d4d3d5a9f27e0fbf1d7ed6db818a779643b8f3) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* python3-testtools: Avoid traceback2 module requirementRichard Purdie2020-09-162-0/+25
| | | | | | | | | | | | | | | | | | | | traceback2 adds traceback for python2. Rather than depend on traceback2, we're python3 only so just use traceback. This caused breakage in oe-selftest -j which uses testtools on the autobuilder using buildtools-tarball. [YOCTO #13652] (From OE-Core rev: ee80a06c107375e3cf0d246ea17c09dda4536dab) (From OE-Core rev: ee82e3c24fe5727ce81e972cadedca431d6086c5) (From OE-Core rev: be4470c9590183b388d9ff176331d0c50984dec8) (From OE-Core rev: e15ff4775aef99a13acb98501454d1b99c923969) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* attr: Disable parallel make installRichard Purdie2020-09-161-0/+3
| | | | | | | | | | | | | | do_install fails on newer versions of make with interesting and hard to debug errors. Disablle parallle make install as a workaround. Later verisons of attr in newer releases don't have the issue. (From OE-Core rev: 6043b9a2ea879f8960897b11eb947801508a94da) (From OE-Core rev: f06861bbe402fff3f370687585e43c0270609d00) (From OE-Core rev: 77bfdb505c8483416fbd4e78cf42ad09923c401b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* uninative: Upgrade to 2.9Khem Raj2020-09-161-5/+5
| | | | | | | | | | | | | This supports glibc upto 2.32 which is now rolling into distributions (From OE-Core rev: 622371678ddb013fc456eaf75def26fc4e142d15) (From OE-Core rev: 4543eeacd65eebe74ff3a44182915a732ba26e47) (From OE-Core rev: ab3c7e09c347a2c57d894ba5e04f38fc9adfad59) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-extended-tarball: add nativesdk-libxcrypt-devJeremy Puhlman2020-09-161-0/+1
| | | | | | | | | | | | | | | | | | | virtual/crypt-native is assume provided in bitbake.conf, so buildtools-extended-tarball shoud provide crypt since it doesn't use the host's headers/libraries. [YOCTO #13714] (From OE-Core rev: da948b25d5ef452fb35275d108e18d2a2829f4fb) (From OE-Core rev: bc42406d83310398bc4d4db4244252411eff117d) (From OE-Core rev: 6f6d7278358b042aca3e911aefd0d6128480f32d) (From OE-Core rev: e1b5cab5cf65df4310b63826690a12ea7083e192) Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* glibc: Update nativesdk locale relocation patchRichard Purdie2020-09-161-0/+33
| | | | | | | | | | | | | | | | The locale binary reported incorrect locale lists in relocated toolchains as some path references were not relocated by this patch. Fix this missing relocations so the locale binary correctly reports the locales. (From OE-Core rev: f7a6a72880009380ae81bc7fc863921a26811c8c) (From OE-Core rev: e4c4337e642f565e9988a4a2c50a995090d1f49e) (From OE-Core rev: c9e8b7a40b2628331c7cb564aa3f3d9e1822fe36) (From OE-Core rev: a41c008eb12004ec8938c03dbc495e07c77d45a6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-tarball: add nativesdk-pythonJeremy Puhlman2020-09-161-0/+1
| | | | | | | | | | | (From OE-Core rev: 6467eb4461f3cab16cab2ba63154c92fc2adacef) (From OE-Core rev: 848c61a07f691638fa529bbe0f0ff1dfded4a967) (From OE-Core rev: afa4cacff186f28d6a4c4246d1e5caf0aa6938e9) Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-tarball: export OPENSSL_CONF in environment setupSteve Sakoman2020-09-161-0/+1
| | | | | | | | | | | | | | | | | The autobuilder has been experiencing SSL: CERTIFICATE_VERIFY_FAILED errors during error report uploads when using buildtools due to looking for certs in /opt/poky (From OE-Core rev: 197f1d5d14b8e57295f5a81c03c86abba5328614) (From OE-Core rev: 35c6ab2501672083cf8b974d8b9c3daa3202de36) (From OE-Core rev: 0cb479a5e99289b75e89b2ed5058f33605f15936) (From OE-Core rev: f96a3082a0822106dfed73d55117552ccff5734f) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-tarball: export OPENSSL_CONF for opensslLiwei Song2020-09-161-0/+1
| | | | | | | | | | | | | | | export OPENSSL_CONF to aviod SDK openssl can not find openssl.cnf. (From OE-Core rev: 0aaf3dd17dcde959e9c0d62543cb91c9b33551b4) (From OE-Core rev: 63d8569b2c9f66e8123e2672a7f8fb8e7cc1f0b4) (From OE-Core rev: e733a5f3b0e3c3b8a830db5ae99b3fc6b7e56921) (From OE-Core rev: 22dd23e3d6c4ee2066198fb91554bbe00a582db0) Signed-off-by: Liwei Song <liwei.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-extended-tarball: Add libstc++.aJeremy Puhlman2020-09-161-0/+1
| | | | | | | | | | | | | | | | | | Builds like native-openjdk, really wants a to link some tools against the static version. Since when using the extended tarball, its the only place to get it, add the library. (From OE-Core rev: 59c4a3fdbbfd5a6aaba7e0a1675dcd5866a7f3a4) (From OE-Core rev: 152709dec03bbac582ca63b65f2efb835e0b33fb) (From OE-Core rev: 5e3664e5f9a0dde07b0f8a56cdce1321456abaa5) (From OE-Core rev: 2cbc936110f1a5d9532b47439b6da1b12caa307b) Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* nativesdk-buildtools-perl-dummy: add dependencies for autoconf and automakeTim Orling2020-09-161-0/+3
| | | | | | | | | | | | | | | | * For buildtools-extended-tarball, where we are adding all of build-essentials to the nativesdk, we need additional perl modules for autoconf and automake. (From OE-Core rev: f0f766160663407ea7683d31bbf5f011accc9ba2) (From OE-Core rev: e7ade58a7da52ebb40120020dd86dd3ae9b2148e) (From OE-Core rev: ed9d60fb5d471b4ec472088cc9307fd8575b187a) (From OE-Core rev: 1276b895008919f510f609d8da4a157d47f09c48) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-extended-tarball: Add locale commandRichard Purdie2020-09-161-0/+1
| | | | | | | | | | | | | | | The eSDK installation code checks installed locales with the locale command which is from glibc-utils. Add this so that we find the correct locales from the buildtools. (From OE-Core rev: 7d35e4bc6ff94a2d03c48827d7d60a6855c9029d) (From OE-Core rev: d99b6432decec0964ac0e08698abc782c9b114f5) (From OE-Core rev: 3562a6848aa3e866ad8e2d3caed3211971817234) (From OE-Core rev: 76227185faedc0946f2b69a8cfe4286f6e5355d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* binutils: Install non-alternatives links for nativesdkRichard Purdie2020-09-161-0/+3
| | | | | | | | | | | | | | | In the SDK we need the plain symlinks and don't use alternative providers. When these are missing the toolchain can work incorrectly so fix this. (From OE-Core rev: 0c06cfaa016d06cc56d80dc1c244a938f3d38a3c) (From OE-Core rev: 0d299c5dc04407d2d54574157f4014f50f2d0468) (From OE-Core rev: aa37b5fe0620122e47f36165f5c7a07d3328dba3) (From OE-Core rev: 6540c5bb9241d5729a0e56f5cf24e1d1d1d4a4cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-tarball: Add an ld.so.conf for nativesdk-binutilsRichard Purdie2020-09-161-0/+4
| | | | | | | | | | | | | | | We need to search our own libdirs, then fall back to the system ones as our customised dynamic loader will. Have ld.so.conf reflect that. This ensures that binutils finds libraries here when linking too. (From OE-Core rev: ab729c362684474a8346e5256d636200826feb47) (From OE-Core rev: 8de0aee6befc0541fa40563f63dfe1cc36f064fe) (From OE-Core rev: d7894d3578d9e97185b4a326c346a3fbb6936ab6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildtools-extended-tarball: add recipe with build-essentialsTim Orling2020-09-161-0/+32
| | | | | | | | | | | | | | | | | | | | | | | * For some aging distros, such as CentOS 7, the native version of gcc is simply too ancient and is a constant source of headaches for moving forward. * Add an extended version of buildtools-tarball which adds all of build-essential, so that the host is now modernized and capable of compiling the latest versions of components. Fixes [YOCTO #13714] (From OE-Core rev: f0377af2325613b63716b0bb4db1ab253d79f388) (From OE-Core rev: bb4979f0e8367b475cc9a5274933a61bb0eb64b3) (From OE-Core rev: f492e172e133a4b52dbe818d806cab783204e575) (From OE-Core rev: 4b23c235bdf29cc45ab084e6fdce8cba3ce7fce2) Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* binutils: Fix relocation of ld.so.conf in nativesdk buildsRichard Purdie2020-09-162-0/+82
| | | | | | | | | | | | | | | | | | | | | | | We need binutils to look at our ld.so.conf file within the SDK to ensure we search the SDK's libdirs as well as those from the host system. There add a patch which passes in the directory to the code using a define, then add it to a section we relocate in a similar way to the way we relocate the gcc internal paths. This ensures that ld works correctly in our buildtools tarball. Standard sysroot relocation doesn't work since we're not in a sysroot, we want to use both the host system and SDK libs. (From OE-Core rev: f6c1089642934ad93056ef19a0888965486ee030) (From OE-Core rev: 09a2b16ac2bd1e3e415131e46315c851373aa7e0) (From OE-Core rev: d0b7811b0e8654cf83d1b0f8256c7941fc3d9c41) (From OE-Core rev: 669b73c9f469642085c6ad11b55a9065c889ddbd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: add lockfile for port used when slirp enabledChangqing Li2020-09-161-37/+91
| | | | | | | | | | | | | | | | There is race condition when multi qemu starting with slirp, add lockfile for each port to avoid problem like: runqemu - ERROR - Failed to run qemu: qemu-system-x86_64: Could not set up host forwarding rule 'tcp::2323-:23' [YOCTO #13364] (From OE-Core rev: ceb3555a40ba06e58914465376aaf41392c12a7c) (From OE-Core rev: 9f9657683df90c18c1dfc7e65715b134a44a9d5a) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* files/toolchain-shar-extract.sh: Rework PATH cleaningRichard Purdie2020-09-161-8/+3
| | | | | | | | | | | | | Trying to create a clean PATH breaks cases where we install a buildtools tarball on hosts to provide newer versions of gcc. Rework the fix for #8698 to clean up directories in PATH which don't exist isntead. Do it with python as the shell version was too fraught with corner cases. (From OE-Core rev: 7674b63819aa7ca95ca5ca5477a5cce32e9691eb) (From OE-Core rev: cd935db103312f6caec2832de80e49e3ed7d1ed8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* populate_sdk_ext: Fix to use python3, not pythonRichard Purdie2020-09-161-1/+1
| | | | | | | | | | | We should be using python3 here, it was missed in the conversion. Spotted on autobuilder tests failing on systems with python missing. (From OE-Core rev: db07b09196022078346aadd565760240b7da6a71) (From OE-Core rev: 2ce4dd53443e86c707280716bfe23572eff58abb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* populate_sdk_ext: We now require python3, not pythonRichard Purdie2020-09-161-2/+2
| | | | | | | | | | | | We no longer expect a "python" binary in PATH so update the eSDK's expectations to match. This was the only failure on autobuilder test systems with python missing. (From OE-Core rev: 946ce21b10dcad506edcaadb4e4242c049e4c316) (From OE-Core rev: 775336424bcc7c083e2ac6ccd3db0b16e87dc29a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/testsdk: Use original PATHRichard Purdie2020-09-161-5/+2
| | | | | | | | | | | | | | | | We want to test the SDK with PATH from the original host, not with our own tools injected via HOSTTOOLS. It even uses some tools which aren't in HOSTTOOLS. This is necessary after changing the SDK to not reset PATH to the system default which is bad for other reasons and brings the testing into sync with that change. (From OE-Core rev: 87c9602fd0dedc7bcf75b822aaf5f6ebfc17737c) (From OE-Core rev: 2cb99a44c650db7fd6fbd269f5788e4ebfd523fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Ensure buildtools in environment variables isn't replacedRichard Purdie2020-09-161-1/+1
| | | | | | | | | | | | | This avoids the seeing broken replacements like: oe-selftest-centos/build/build-st-926tools/sysroots/x86_64-pokysdk-linux/etc/ssl/certs/ca-certificates.crt which understandably break builds. (From OE-Core rev: 04ee0e8b95cd8ed890374e0007f976684206b630) (Cherry-picked from f930e2cadb9ee69759720b6c49aeeb6dd43a7edd but adjusted for thud) (From OE-Core rev: 611d3947054dad764aeded4c6a050415f7ca4991) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-uninative.inc: version 2.8 updates glibc to 2.31Michael Halstead2020-09-161-5/+5
| | | | | | | | | | Allow sstate use in Tumbleweed and other distros as they update glibc. (From OE-Core rev: ccb374c279b260b1fd3460f6bfd1567240816055) (From OE-Core rev: 0e12f41848fd2fdbc0f70f568ce13baeb3263d03) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: fix gcc 10 version detectionCharles-Antoine Couret2020-09-161-1/+1
| | | | | | | | | | | | | | | | | Utils can not detect GCC 10 correctly due to wrong regex. It generates this error "ERROR: Can't get compiler version from gcc --version output" Sub-version numbers should be 1 or more digits instead of 1 only. (From OE-Core rev: 1d6f50a5e58f46f8af6e83c4e288d93a717187ea) (From OE-Core rev: e73228e6b039bd972d36774bfb360a638a03d821) Signed-off-by: Charles-Antoine Couret <charles-antoine.couret@mind.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 186fe4a3d390a52b87282c3e694ce3251e45ee78) Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Allow wget upgrade tests to run against a local serverRichard Purdie2020-09-0940-20/+7691
| | | | | | | | | | | Currently these tests rely upon multiple uptream webservers which may change or be unavailable. Add local copies of the test data, copy the httpserver from OE-Core (used for testing there) and run these tests against a local server instead. (Bitbake rev: a21671e8a483ba8a6986d961987eda2d36ec61ca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Change git fetcher not to destroy old referencesRichard Purdie2020-09-081-1/+1
| | | | | | | | | | | | | | | | It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 12d8cc3fecd550c4aadf0519e80711d755ee75ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Switch from git.infradead.org to a YP mirrorRichard Purdie2020-07-101-1/+2
| | | | | | | | | | Upstream is unavailable, breaking tests. Switch to a YP mirror since if we can't reach that there are bigger problems. This should remove a source of intermittent failures on the autobuilder. (Bitbake rev: f4e60b29df88393302957c5bbdbe24ca38c4633c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Adding memoriam to scottrifJefro2020-01-311-0/+5
| | | | | | | | | | Added a few comment lines in Makefile to commemorate Scott's contributions (From yocto-docs rev: 421a80308c36c3da98d5fb6f6100ee3fab6abd0e) Signed-off-by: Jefro <jefro@jefro.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* linux-yocto/4.14: update Yocto Bsps to 4.14.154Armpit2020-01-021-10/+10
| | | | | | | (From meta-yocto rev: bf00cab7a55e2038e09a307378af5aec04c99380) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* linux-yocto/4.14: update to 4.14.154Armin Kuster2019-12-163-16/+16
| | | | | | | (From OE-Core rev: e68991ceb5933f7d03b96697e8a0ba0829feb320) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* glibc: finish incomplete fix for CVE-2016-10739Ross Burton2019-12-161-3/+907
| | | | | | | | | | Somehow the patch for this CVE only included one of the four required patches. (From OE-Core rev: e7ed139e48b683ebe3e6863886e712998aaa239c) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: fetch CVE data once at a time instead of in a single callRoss Burton2019-12-161-10/+10
| | | | | | | | | | | | | | | | | | | | | This code used to construct a single SQL statement that fetched the NVD data for every CVE requested. For recipes such as the kernel where there are over 2000 CVEs to report this can hit the variable count limit and the query fails with "sqlite3.OperationalError: too many SQL variables". The default limit is 999 variables, but some distributions such as Debian set the default to 250000. As the NVD table has an index on the ID column, whilst requesting the data CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time different is insignificant: 0.05s verses 0.01s on my machine. (From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99) (From OE-Core rev: b52d6340acdad27d41caf057b78f181297a9a75e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: neaten get_cve_infoRoss Burton2019-12-161-13/+5
| | | | | | | | | | | | | Remove obsolete Python 2 code, and use convenience methods for neatness. (From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff) (From OE-Core rev: 1f3863bc31e03207856f55591cbf17543e188587) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: rewrite look to fix false negativesRoss Burton2019-12-161-29/+34
| | | | | | | | | | | | | | | | | | | A previous optimisation was premature and resulted in false-negatives in the report. Rewrite the checking algorithm to first get the list of potential CVEs by vendor:product, then iterate through every matching CPE for that CVE to determine if the bounds match or not. By doing this in two stages we can know if we've checked every CPE, instead of accidentally breaking out of the scan too early. (From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69) (From OE-Core rev: 541dc24d974d3e22c45a650c34298eebc45121e8) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: clean up proxy handlingRoss Burton2019-12-161-26/+5
| | | | | | | | | | | | | | urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup. (From OE-Core rev: 6b73004668b3b71c9c38814b79fbb58c893ed434) (From OE-Core rev: aa197b91e1770925ae1a31ee7334b593bfcdc9e3) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: add an index on the CVE ID columnRoss Burton2019-12-161-0/+3
| | | | | | | | | | | | | | Create an index on the PRODUCTS table which contains a row for each CPE, drastically increasing the performance of lookups for a specific CVE. (From OE-Core rev: b4048b05b3a00d85c40d09961f846eadcebd812e) (From OE-Core rev: 27ee95bd1ec2076509cfc2230eadb876fb35d6c2) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: don't hardcode the database nameRoss Burton2019-12-161-2/+2
| | | | | | | | | | | | | | Don't hardcode the database filename, there's a variable for this in cve-check.bbclass. (From OE-Core rev: 0d188a9dc4ae64c64cd661e9d9c3841e86f226ab) (From OE-Core rev: 29cc2b5cd4bcce1c9e93395a1640014877486d7a) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: don't refresh more than once an hourRoss Burton2019-12-161-1/+9
| | | | | | | | | | | | | | | We already fetch the yearly CVE metadata and check that for updates before downloading the full data, but we can speed up CVE checking further by only checking the CVE metadata once an hour. (From OE-Core rev: 50d898fd360c58fe85460517d965f62b7654771a) (From OE-Core rev: 091a35cfbd2f3e82a7783ba9c8fd5586433ba59f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: we don't actually need to unpack to checkRoss Burton2019-12-161-2/+1
| | | | | | | | | | | | | | The patch scanner works with patch files in the layer, not in the workdir, so it doesn't need to unpack. (From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17) (From OE-Core rev: cbb5d26d88465c95a4a879f8635253259e8df0f0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: failure to parse versions should be more visibleRoss Burton2019-12-161-2/+2
| | | | | | | | (From OE-Core rev: 72f44bef3867295f73f8b91e17294b2876447c89) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: ensure all known CVEs are in the reportRoss Burton2019-12-161-2/+7
| | | | | | | | | | | | | | | CVEs that are whitelisted or were not vulnerable when there are version comparisons were not included in the report, so alter the logic to ensure that all relevant CVEs are in the report for completeness. (From OE-Core rev: 98256ff05fcfe9d5ccad360582c36eafb577c264) (From OE-Core rev: 9d01a64844998d98fcfcebbe8580422094cd2dde) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: backport rewrite from masterRoss Burton2019-12-1612-620/+292
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As detailed at [1] the XML feeds provided by NIST are being discontinued on October 9th 2019. As cve-check-tool uses these feeds, cve-check.bbclass will be inoperable after this date. To ensure that cve-check continues working, backport the following commits from master to move away from the unmaintained cve-check-tool to our own Python code that fetches the JSON: 546d14135c5 cve-update-db: New recipe to update CVE database bc144b028f6 cve-check: Remove dependency to cve-check-tool-native 7f62a20b32a cve-check: Manage CVE_PRODUCT with more than one name 3bf63bc6084 cve-check: Consider CVE that affects versions with less than operator c0eabd30d7b cve-update-db: Use std library instead of urllib3 27eb839ee65 cve-check: be idiomatic 09be21f4d17 cve-update-db: Manage proxy if needed. 975793e3825 cve-update-db: do_populate_cve_db depends on do_fetch 0325dd72714 cve-update-db: Catch request.urlopen errors. 4078da92b49 cve-check: Depends on cve-update-db-native f7676e9a38d cve-update-db: Use NVD CPE data to populate PRODUCTS table bc0195be1b1 cve-check: Update unpatched CVE matching c807c2a6409 cve-update-db-native: Skip recipe when cve-check class is not loaded. 07bb8b25e17 cve-check: remove redundant readline CVE whitelisting 5388ed6d137 cve-check-tool: remove 270ac00cb43 cve-check.bbclass: initialize to_append e6bf9000987 cve-check: allow comparison of Vendor as well as Product 91770338f76 cve-update-db-native: use SQL placeholders instead of format strings 7069302a4cc cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST 78de2cb39d7 cve-update-db-native: Remove hash column from database. 4b301030cf9 cve-update-db-native: use os.path.join instead of + f0d822fad2a cve-update-db: actually inherit native b309840b6aa cve-update-db-native: use executemany() to optimise CPE insertion bb4e53af33d cve-update-db-native: improve metadata parsing 94227459792 cve-update-db-native: clean up JSON fetching 95438d52b73 cve-update-db-native: fix https proxy issues 1f9a963b9ff glibc: exclude child recipes from CVE scanning [1] https://nvd.nist.gov/General/News/XML-Vulnerability-Feed-Retirement (From OE-Core rev: 8c87e78547c598cada1bce92e7b25d85b994e2eb) (From OE-Core rev: beeed02f9831e75c3f773e44d7efc726f1ff859c) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sudo: Fix CVE-2019-14287Dan Tran2019-12-163-0/+270
| | | | | | | | (From OE-Core rev: e21a8e3b2b2b035cf71883f72eeb665e3fa9c078) Signed-off-by: Dan Tran <dantran@microsoft.com> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tar: Fix CVE-2018-20482Dan Tran2019-12-162-0/+406
| | | | | | | | (From OE-Core rev: 95ab1519ea5f1a0ed73f6f484bcf15fde5de8140) Signed-off-by: Dan Tran <dantran@microsoft.com> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>