summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* cve-update-db-native: fix https proxy issuesChin Huat Ang2019-11-071-11/+30
| | | | | | | | | | | | | | | | When https_proxy is set, use proxy opener to open CVE metadata and database URLs, otherwise fallback to the urllib.request.urlopen. Also fix a minor issue where the json database which has been gzip decompressed as byte object should be decoded as utf-8 string as expected by update_db. (From OE-Core rev: 95438d52b732bec217301fbfc2fb019bbc3707c8) (From OE-Core rev: 6d3222fb7ecde524c4e033729318fb0fb80a444c) Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: clean up JSON fetchingRoss Burton2019-11-071-17/+12
| | | | | | | | | | | | | | | Currently the code fetches the compressed JSON, writes it to a temporary file, uncompresses that with gzip and passes the fake file object to update_db(). Instead, uncompress the gzip'd data in memory and pass the JSON directly to update_db(). (From OE-Core rev: 9422745979256c442f533770203f62ec071c18fb) (From OE-Core rev: 1d34aec479156a7dadf7867bbf0d53f12d21ef3e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: improve metadata parsingRoss Burton2019-11-071-8/+10
| | | | | | | | | | | | | | | | | | The metadata parser is fragile: first it coerces a bytes() to a str() (so the string is b'LastModifiedDate:2019...'), assumes the first line is the date, and then uses a regex to parse (which then includes the trailing quote as part of the date). Clean this up by parsing the bytes as UTF-8 (ASCII is probably fine, but this is safer), iterate through the lines and split on colons to find the right key/value pair. (From OE-Core rev: bb4e53af33d6ca1e9346464adbdc1b39c47530f3) (From OE-Core rev: c718e073e8e9cd5df9e19dd02fcac2139758b5b7) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: use executemany() to optimise CPE insertionRoss Burton2019-11-071-53/+32
| | | | | | | | | | | | Instead of calling execute() repeatedly, rewrite the function to be a generator and use executemany() for performance. (From OE-Core rev: b309840b6aa3423b909a43499356e929c8761318) (From OE-Core rev: d248ec9764d0439eb30fdb3605e9d05ee4219348) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: actually inherit nativeRoss Burton2019-11-071-2/+1
| | | | | | | | | | | The recipe was called -native but didn't inherit native. (From OE-Core rev: f0d822fad2a163d1ee32ed3b4c0359245140e19b) (From OE-Core rev: 5eeafcb492daf63602f0e2ed4a12f755701597d7) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: use os.path.join instead of +Ross Burton2019-11-071-4/+4
| | | | | | | | | (From OE-Core rev: 4b301030cf9cf7a981dcff85a50e915c045e3130) (From OE-Core rev: 7df7cd765e67535b72cd56eb679c6f5078c08460) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: Remove hash column from database.Pierre Le Magourou2019-11-072-20/+13
| | | | | | | | | | | | | | djb2 hash algorithm was found to do collisions, so the database was sometime missing data. Remove this hash mechanism, clear and populate elements from scratch in PRODUCTS table if the current year needs an update. (From OE-Core rev: 78de2cb39d74b030cd4ec811bf6f9a6daa003d19) (From OE-Core rev: e6541c6add1714938a81cca394886893cf24cdb0) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: Use NVD CPE data to populate PRODUCTS tablePierre Le Magourou2019-11-071-14/+74
| | | | | | | | | | | | | | | Instead of using expanded list of affected versions that is not reliable, use the 'cpe_match' node in the 'configurations' json node. For cve-check to correctly match affected CVE, the sqlite database need to contain operator_start, operator_end and the corresponding versions fields. (From OE-Core rev: f7676e9a38d595564922e5f59acbc69c2109a78f) (From OE-Core rev: 6977d15fbc3b78958768b21f6c501e7d63be9499) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db-native: use SQL placeholders instead of format stringsRoss Burton2019-11-071-1/+1
| | | | | | | | | (From OE-Core rev: 91770338f76ef35f3c4eeac216eb9d2b3188e575) (From OE-Core rev: 075683d23018760e8b2fa0b793ceacd9027e55c3) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELISTPierre Le Magourou2019-11-071-11/+11
| | | | | | | | | | | | CVE_CHECK_WHITELIST does not contain version anymore, as it was not used. This variable should be set per recipe. (From OE-Core rev: 7069302a4ccbb5b72e1902f284cf078516fd7294) (From OE-Core rev: 8dd899679fc881d02e081d1e0814252d604dd479) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: allow comparison of Vendor as well as ProductRoss Burton2019-11-071-4/+8
| | | | | | | | | | | | | | | | | | Some product names are too vague to be searched without also matching the vendor, for example Flex could be the parser compiler we ship, or Adobe Flex, or Apache Flex, or IBM Flex. If entries in CVE_PRODUCT contain a colon then split it as vendor:product to improve the search. Also don't use .format() to construct SQL as that can lead to security issues. Instead, use ? placeholders and lets sqlite3 handle the escaping. (From OE-Core rev: e6bf90009877d00243417898700d2320fd87b39c) (From OE-Core rev: 0851d68b4679a7035029d28091d9a6b21d266c99) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check.bbclass: initialize to_appendMikko Rapeli2019-11-071-0/+1
| | | | | | | | | | | | | Fixes build failure with core-image-minimal: Exception: UnboundLocalError: local variable 'to_append' referenced before assignment (From OE-Core rev: 270ac00cb43d0614dfe1c95f960c76e9e5fa20d4) (From OE-Core rev: 45758c900ff738e58fd37ff809960965867d79f8) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* glibc: exclude child recipes from CVE scanningRoss Burton2019-11-074-3/+10
| | | | | | | | | | | | | | | | As glibc will be scanned for CVEs, we don't need to scan glibc-locale, glibc-mtrace, and glibc-scripts which are all separate recipes for technical reasons. Exclude the recipes by setting CVE_PRODUCT in the recipe, instead of using the global whitelist. (From OE-Core rev: 1f9a963b9ff7ebe052ba54b9fcbdf7d09478dd17) (From OE-Core rev: 2b9f1b654c726e7c7b2fe8710d60ca10212295f5) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check-tool: removeRoss Burton2019-11-076-565/+0
| | | | | | | | | (From OE-Core rev: 5388ed6d1378d647a65912dbd537f9ef3cb5760a) (From OE-Core rev: eb227c8885580fc08dccc005056bb1fdb691ea1d) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: remove redundant readline CVE whitelistingRoss Burton2019-11-071-4/+9
| | | | | | | | | | | | | | CVE-2014-2524 is a readline CVE that was fixed in 6.3patch3 onwards, but the tooling wasn't able to detect this version. As we now ship readline 8 we don't need to manually whitelist it, and if we did then the whitelisting should be in the readline recipe. (From OE-Core rev: 07bb8b25e172aa5c8ae96b6e8eb4ac901b835219) (From OE-Core rev: c7f23d4e53d039838536f71996ad896c977cf138) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Update unpatched CVE matchingPierre Le Magourou2019-11-071-14/+40
| | | | | | | | | | | | Now that cve-update-db added CPE information to NVD database. We can check for unpatched versions with operators '<', '<=', '>', and '>='. (From OE-Core rev: bc0195be1b15bcffe60127bc5e8b7011a853c2ed) (From OE-Core rev: 48793a3b74bfaa5ffe6191d21f64aef3720433db) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Depends on cve-update-db-nativePierre Le Magourou2019-11-073-1/+3
| | | | | | | | | | | | | | do_populate_cve_db is a native task. (From OE-Core rev: 4078da92b49946848cddebe1735f301af161e162) (From OE-Core rev: 5d6cbab419770eb556b57445fd5509339d3142b4) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Conflicts: meta/conf/distro/include/maintainers.inc
* cve-update-db: Catch request.urlopen errors.Pierre Le Magourou2019-11-072-11/+24
| | | | | | | | | | | | | If the NVD url is not accessible, print a warning on top of the CVE report, and continue. The database will not be fully updated, but cve_check can still run on the previous database. (From OE-Core rev: 0325dd72714f0b447558084f481b77f0ec850eed) (From OE-Core rev: ae743789d893e950583014f38f0ad246aa4fe034) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: do_populate_cve_db depends on do_fetchPierre Le Magourou2019-11-071-8/+13
| | | | | | | | | | | | | | | | | To be able to populate NVD database on a fetchall (bitbake <image> --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE variable because do_populate_cve_db can be called in a context where cve-check class is not loaded. (From OE-Core rev: 975793e3825a2a9ca6dc0e43577f680214cb7993) (From OE-Core rev: 5d265e84ef47ec6545eaa0fa64b16ccbb9e8a4ea) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: Manage proxy if needed.Pierre Le Magourou2019-11-071-2/+9
| | | | | | | | | | | | If https_proxy environment variable is defined, manage proxy to be able to download meta and json data feeds from https://nvd.nist.gov (From OE-Core rev: 09be21f4d1793b1e26e78391f51bfc0a27b76deb) (From OE-Core rev: 3af4399ea35b5c4b87d656f09dd2afed11791f0a) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: be idiomaticRoss Burton2019-11-071-8/+9
| | | | | | | | | | | | Instead of generating a series of indexes via range(len(list)), just iterate the list. (From OE-Core rev: 27eb839ee651c2d584db42d23bcf5dd764eb33f1) (From OE-Core rev: 27ef8c40afc27ce0ae87d2fe9a973edc89133def) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: Use std library instead of urllib3Pierre Le Magourou2019-11-071-6/+4
| | | | | | | | | | | | | urllib3 was used in this recipe but it was not set as a dependency. As it is not specifically needed, rewrite the recipe with urllib from the standard library. (From OE-Core rev: c0eabd30d7b9c2517f4ec9229640be421ecc8a5e) (From OE-Core rev: bfaee04b8a7cb0fc6e149106619a01b848fd8a98) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* flac: also add flac to CVE_PRODUCTChen Qi2019-11-071-1/+1
| | | | | | | | | | | flac uses both 'flac' and 'libflac' as cve product. (From OE-Core rev: 3a043a078f6cc89bcc097823fa37cd1311805ae7) (From OE-Core rev: c130045aff7f51ddb6c7fbde590a79207dbb4ddf) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Consider CVE that affects versions with less than operatorPierre Le Magourou2019-11-071-2/+14
| | | | | | | | | | | | | | | | In the NVD json CVE feed, affected versions can be strictly matched to a version, but they can also be matched with the operator '<='. Add a new condition in the sqlite query to match affected versions that are defined with the operator '<='. Then use LooseVersion to discard all versions that are not relevant. (From OE-Core rev: 3bf63bc60848d91e90c23f6d854d22b78832aa2d) (From OE-Core rev: 70046288894184477dcf6f7eba25b1994b88c8de) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Manage CVE_PRODUCT with more than one namePierre Le Magourou2019-11-071-11/+14
| | | | | | | | | | | | In some rare cases (eg. curl recipe) the CVE_PRODUCT contains more than one name. (From OE-Core rev: 7f62a20b32a3d42f04ec58786a7d0db68ef1bb05) (From OE-Core rev: 4f96e9ba1f4f14f312b6024711fe8da0c3041e4c) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: Remove dependency to cve-check-tool-nativePierre Le Magourou2019-11-071-45/+26
| | | | | | | | | | | Use the new update-cve-db recipe to update database. (From OE-Core rev: bc144b028f6f51252f4359248f6921028bcb6780) (From OE-Core rev: 6556bb30998d9d37f2389492eb7c15667ba4a827) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-update-db: New recipe to update CVE databasePierre Le Magourou2019-11-072-1/+122
| | | | | | | | | | | | | | | | | | | | cve-check-tool-native do_populate_cve_db task was using deprecated NVD xml data feeds, cve-update-db uses NVD json data feeds. Sqlite database schema was updated to take into account CVSSv3 CVE scores and operator in affected product versions. A new META table was added to store the last modification date of the NVD json data feeds. (From OE-Core rev: 546d14135c50c6a571dfbf3baf6e9b22ce3d58e0) (From OE-Core rev: e344a27003cc9e39058b41c0e96463f231ebf245) Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Conflicts: meta/conf/distro/include/maintainers.inc
* uninative: Switch from bz2 to xzRichard Purdie2019-11-071-2/+2
| | | | | | | | | | (From OE-Core rev: 29fc9210b973be68de474e75068e4c72371afe5a) (From OE-Core rev: b6645596f2d2faf8f1fdfbedfe1edd004fbce6bc) (From OE-Core rev: 151f7fb11bb4c91dd6edaebcc63fa3c1a2cbfe8b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* yocto-uninative: Update to 2.5 releaseRichard Purdie2019-11-071-3/+3
| | | | | | | | | | | | This includes libstdc++ changes from gcc 9.X. It also switches uninative from bz2 to xz compression. (From OE-Core rev: 7ed16ec033366aea175ac4ecf7cd82656c4141bb) (From OE-Core rev: 0bc5136608f7e3cab31ea57a4c3dd8df7eca9a4b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker child process create group before registering ↵Ivan Efimov2019-11-071-3/+5
| | | | | | | | | | | | | | | | | | SIGTERM handler The bitbake-worker child on the SIGTERM signal handling send the SIGTERM to all processes in it's process group. In cases when the bitbake-worker child got SIGTERM after registering own SIGTERM handler and before the os.setsid() call it can send SIGTERM to unwanted processes. In the worst case during SIGTERM processing the bitbake-worker child can be in the group of the process that started BitBake itself. As a result it can kill processes that not related to BitBake at all. (Bitbake rev: 945719d852da6c787bc9115bd0aa90c429f5de07) Signed-off-by: Ivan Efimov <i.efimov@inango-systems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Resolve fetch error in bitbake-selftestArmin Kuster2019-11-071-2/+2
| | | | | | | | | | | | | | | | | | | | | FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 50d2166054338b7a63c4dc7116cd212a9ecfbfaf) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Split tests into multiple classesRichard Purdie2019-04-251-52/+61
| | | | | | | | | | | | | | | This allows better parallelism between the different tests as currently this block takes the longest time to execute. devtool tests are still all grouped into the "devtool" module for ease of exection. This also makes it easier to execute some subset of devtool tests for testing devtool changes. (From OE-Core rev: 84f19e78d9b1f3d634cf1d46ce48f24670199d0b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Default to buffer mode for testsRichard Purdie2019-04-256-13/+1
| | | | | | | | | | | | | | Currently some tests run in buffer mode and some don't. Those that don't can corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent. If there is useful output on stdout/stderr, it will be displayed if the test fails. (From OE-Core rev: 85c1b6fb516aae58240330a0aca659bfafcd3883) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/devtool: Ensure dbus is built befoe running testRichard Purdie2019-04-251-0/+2
| | | | | | | | | | | | | | | | | If dbus isn't build first the dbus dependency of dbus-wait can't be detected through pkgconfig and the test fails: AssertionError: {'DEPENDS': {'dbus'}} != {} - {'DEPENDS': {'dbus'}} + {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}} Ensure dbus is built and present in the sysroot. (From OE-Core rev: af7ba26a603a12de0aed35e786674c92049c2bee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* logging: use warning instead warnChen Qi2019-04-2516-48/+48
| | | | | | | | | | | | | | | | | The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ (From OE-Core rev: cc771aa4b74f222f1bea38b0b50196b2fbc97ab4) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe: Fix collections ABCs DeprecationWarning in Python 3.7+Khem Raj2019-04-251-2/+7
| | | | | | | | | | | | | | | | | | | - Prefer collections.abc (new in Python 3.3) over collections for abstract base classes - In Python 3.8, the abstract base classes in collections.abc will no longer be exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes." - https://docs.python.org/3.7/whatsnew/3.7.html#deprecated - see https://github.com/python/cpython/commit/c66f9f8d3909f588c251957d499599a1680e2320 (From OE-Core rev: b254ab6ce34da3d3241a51958b5770664d317fcc) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: use explicit errno importRoss Burton2019-04-251-4/+5
| | | | | | | | | | | | os.errno doesn't work in Python 3.7 and shouldn't have ever worked, so use import errno explicitly. (From OE-Core rev: 7b1dba1a464862e35295756ecf52b24d0962528b) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/engine: Fix missing parted autobuilder failuresRichard Purdie2019-04-251-3/+5
| | | | | | | | | | | | | | | | OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842 broke the ability to find parted as it may be in sbin which is not in PATH for some users on some distros. Iterate on the original patch to fix this and also fix the original problem. (From OE-Core rev: af3803e5189d7814f9dbd238fb6dab200f351e1a) (From OE-Core rev: 68f56a4967d3121940669ca9116e759081b0b73b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic/engine.py: Load paths from PATH environment variableWilliam Bourque2019-04-251-2/+7
| | | | | | | | | | | | | | | | | Load self.paths from environment variable and if it fails, fall back to hardcoded list. This is required for users that would need to load different e2fsprogs binaries if their system's ones are not recent enought (From OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842) (From OE-Core rev: c8331fa157b83a3ed77f77a6e884c7fbc4f1d91b) Signed-off-by: William Bourque <wbourque@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/lib/wic/engine: Fix cp's target path for ext* filesystemsLeonardo Augusto2019-04-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Python subprocess' shell=True defaults to /bin/sh[1][2], which often refers to a POSIX-compliant shell. As the -e flag is not defined in the POSIX standard[3], some shells may interpret "-e" as the first argument to the echo command, causing the rest of the command line to fail. In this particular case, "echo -e 'cd {}'" is interpreted as "-e cd {}", which causes the first line of the command to fail, and causing cp to always place the source file in the filesystem's root. Replacing "echo -e" for a printf command makes this more portable. This issue only affects "wic cp" for ext* filesystems. [1] https://docs.python.org/2/library/subprocess.html [2] https://docs.python.org/3/library/subprocess.html [3] http://pubs.opengroup.org/onlinepubs/9699919799 (From OE-Core rev: 9a5b703d4e60c3ccdf80b5289b8e6fc91133fcde) (From OE-Core rev: b9ccf44edccd31264890fa790d6120fc9e2c0238) Signed-off-by: Leonardo Augusto <leobsd@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: search nonarch STAGING_DATADIR for multilibKai Kang2019-04-251-1/+9
| | | | | | | | | | | | | | | | | | | | It fails to build multilib image such as lib32-core-image-minimal with wic by set 'IMAGE_FSTYPES = "wic"': | ERROR: Couldn't find correct bootimg_dir, exiting When multilib is enabled, STAGING_DATADIR is expanded with MLPREFIX. But dependencies of images such as syslinux is still populated to nonarch STAGING_DATADIR. Search nonarch STAGING_DATADIR to fix the error. (From OE-Core rev: 6f53c61dd989045a7075b678c7030421d715bc16) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: make engine.py:get_partitions() resilient to parted/dmidecode stderr outputGeoff Parker2019-04-251-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Running wic commands on Debian 10 systems fail in scripts/lib/wic/engine.py:get_partitions() due to new stderr output captured when trying to parse the output from /sbin/parted as a non-root user. The parted command calls the dmidecode utility, which produces this error as a non-root user: /sys/firmware/dmi/tables/smbios_entry_point: Permission denied /dev/mem: Permission denied scripts/lib/wic/engine.py:get_partitions() calls misc.py:exec_cmd(), a subprocess wrapper which returns a combined stderr and sdtdout. These messages to stderr confuse the partition table parser in get_partitions(). This patch has the partition table parser ignore lines before the expected "BYT;" header string. Running wic in Debian 9 does not have this issue. (From OE-Core rev: 6b5681533eed4e6c00e1cc7ee442c93076f1c976) Signed-off-by: Geoff Parker <geoffhp@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/wic: Be consistent about how we call bitbakeRichard Purdie2019-04-251-11/+5
| | | | | | | | | | | | | | Whilst its tempting to call bitbake_main(), the script also directly executes bitbake. By doing both this creates all kinds of races. Therefore change to use subprocess execution in all cases. (From OE-Core rev: 0507ec4c6069490c9a503e9e2b65f0e7b1962d34) (From OE-Core rev: 2f2d52a958d3e9d511a2f5b8b14ed46466dd0d23) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Support for changing the imager.Davis, Michael2019-04-252-1/+3
| | | | | | | | | | (From OE-Core rev: 30b33d787595b8c3120b73c755a432a0f0f30fa7) Signed-off-by: Michael Davis <michael.davis@essvote.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pkgdata.py: avoid target-sdk-dummy-provides to mess things upChen Qi2019-04-251-0/+1
| | | | | | | | | | | | | | | | | | | | | Sometimes we meet the following failure for the test_lookup_recipe test case. AssertionError: 'zlib\nbusybox is in the RPROVIDES of target-sdk-provid[32 chars]ummy' != 'zlib\nbusybox' zlib + busybox- busybox is in the RPROVIDES of target-sdk-provides-dummy: - target-sdk-provides-dummy This is because target-sdk-provides-dummy rprovides busybox. So clean things up to avoid failure. (From OE-Core rev: 5ae49260180adc0070287db01c01a0820ce99ad3) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Refactor and simplify codebaseYeoh Ee Peng2019-04-251-36/+20
| | | | | | | | | | | | Simplify and removed unnecessary codes. Refactor to allow pythonic loop. (From OE-Core rev: 77ede7cbb7da2ed1b01d11441edcbd7dedad9b9a) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Fixed step sorted by integerYeoh Ee Peng2019-04-251-1/+1
| | | | | | | | | | | | | | | Currently the manual execution display step by sorting the step as string, where steps were not being sorted correctly when there are more than 9 steps. Fixed the step sorting by sorting step as integer. (From OE-Core rev: 1e5fba51f022a9320e72b6df753a02bfc1732af8) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Enable display full steps without press enterYeoh Ee Peng2019-04-251-2/+3
| | | | | | | | | | | | | | Current manualexecution required pressing enter button to show each step information, where this was wasting execution time. Enable display full steps without needing to any press enter button. (From OE-Core rev: c833aba64f26546894a0c58d8a59db75d18e2ff7) Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool/manualexecution: Standardize input checkYeoh Ee Peng2019-04-251-2/+2
| | | | | | | | | | | | | Current input checking does not match the standard input practiced by QA team. Change the input checking to match the standard input practiced by the QA team. (From OE-Core rev: 6eb041fd930a3731743cc715eb8d73b045cf8932) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resulttool: Allow extraction of ptest dataRichard Purdie2019-04-252-9/+32
| | | | | | | | | | | | | | | Rather than simply discarding the ptest data, change the code to discard it when writing out the new testresult files, or optionally either preserve it, or write it as seperate discrete logs. This means the autobuilder should start writing out individual ptest log files as well as allowing ueers to extract these manually. (From OE-Core rev: 08c5471c5abd95616af8df862e96072d022a6f42) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-21 03:48:23 +0000