summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
* oe-build-perf-test: implement --globalres-file optionMarkus Lehtonen2016-07-011-0/+32
| | | | | | | | | | | | Using this option the script appends test results into a 'global results file'. A CSV-formatted output of the results. This option is to provide compatibility with the old build-perf-test.sh. (From OE-Core rev: e9f18e63220e452f2b0c878998e57d944ae83980) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add git revision and branch to result dataMarkus Lehtonen2016-07-011-0/+32
| | | | | | | | | | | | | | | BuildPerfTestRunner determines these from the Git repository under which it is being run (i.e. where the build directory exists). The branch and revision may be defined/overridden with OE_BUILDPERFTEST_GIT_BRANCH and OE_BUILDPERFTEST_GIT_BRANCH environment variables, if needed. This makes it possible to run the build performance test script even if the top directory is not a git repository clone, for example. (From OE-Core rev: e6004582454d8c6a18f617c12e6e408ded5be8df) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils: add git moduleMarkus Lehtonen2016-07-011-0/+38
| | | | | | | | | | | | | | A new helper module for easier interaction with Git repositories. Provides GitRepo class that represents one local Git repository clone. The GitRepo class currently only has one method, run_cmd(), for running arbitrary git commands in the repository. More specialized methods for commonly used git operations can be added later. (From OE-Core rev: 4eaf434f885afbda03fe67ab6e9ff291c7a9c77e) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: archive build/conf into test resultsMarkus Lehtonen2016-07-011-0/+8
| | | | | | | | | | Have the build/conf directory as part of test results. (From OE-Core rev: ea32a8a65f668450e0ec0e80483bd5e8c91b8a18) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test4Markus Lehtonen2016-07-011-0/+28
| | | | | | | | | | Re-implement "test4" from build-perf-test.sh which measures eSDK metrics. (From OE-Core rev: c449ca1b062426380a0572aed2da4ee4a83281f5) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test3Markus Lehtonen2016-07-011-0/+20
| | | | | | | | | | | Re-implement "test3" from build-perf-test.sh which measures bitbake parsing time. (From OE-Core rev: 388c7160b90247df9c78d8803083821402c13af5) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test2Markus Lehtonen2016-07-011-0/+14
| | | | | | | | | | | Re-implement "test2" from build-perf-test.sh which measures 'bitbake core-image-sato -c rootfs'. (From OE-Core rev: 8cb322d6a5be08f0bb2c9e5d3b73161b20d737a3) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test1P3Markus Lehtonen2016-07-011-0/+26
| | | | | | | | | | | Re-implement "test1_p3" from build-perf-test.sh which measures 'bitbake core-image-sato' with rm_work enabled. (From OE-Core rev: bfe48d343e8a281760da0b6744c08d47039f93e0) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test1P2Markus Lehtonen2016-07-011-0/+14
| | | | | | | | | | | Re-implement "test1_p2" from build-perf-test.sh which measures 'bitbake virtual/kernel'. (From OE-Core rev: 5ff82c6e2b0ff41a9bfdd49c99e84a982cf6e467) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add test Test1P1Markus Lehtonen2016-07-012-1/+33
| | | | | | | | | | | Re-implement "test1_p1" from build-perf-test.sh which measures 'bitbake core-image-sato'. (From OE-Core rev: 1ddd4a8ce4e2053def0e946d4e4b4b97bdc28e61) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: implement BuildPerfTestRunner classMarkus Lehtonen2016-07-012-1/+49
| | | | | | | | | | | | The new class is responsible for actually running the tests and processing their results. This commit also adds a decorator function for adding new tests. No automatic test discovery, at least yet. (From OE-Core rev: bf90aecb7e150d6bfac7240286c797b79d26528b) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add method for saving buildstatsMarkus Lehtonen2016-07-011-0/+5
| | | | | | | | (From OE-Core rev: 15b44a484ce408b0a1acfb907e2b12ebc6cce296) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add method for measuring file disk usageMarkus Lehtonen2016-07-011-0/+19
| | | | | | | | | | | Add a new method to BuildPerfTest class for measuring the disk usage of a file of directory. (From OE-Core rev: 85cdc240e75d481e93238fbf75f8b8431da05f19) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add method to log shell commandsMarkus Lehtonen2016-07-011-0/+5
| | | | | | | | | | | Add new methods to BuildPerfTest class for running a shell command and logging its output. (From OE-Core rev: 8f0b11ba1266f9c650cf34d9b394d72009ee7207) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: method for measuring system resource usageMarkus Lehtonen2016-07-011-2/+62
| | | | | | | | | | | | | | | | | | | | | | Extend BuildPerfTest class with a new method for measuring the system resource usage of a shell command to BuildPerfTest class. For now, easurement of the elapsed time is done with the Gnu time utility, similarly to the build-perf-test.sh shell script. And, it currently only records the elapsed (wall clock). The measured values (currently, only the elapsed time) is actually a dictionary, making it possible to extend it with additional resource values, e.g. cpu time or i/o usage, in the future. In addition to the actual values of the measurement each record contains a 'name' and 'legend' where name is supposed to function as a common key or id over test runs, making comparison and trending easier, for example. Legend is supposed to be a short human readable description. (From OE-Core rev: ced156bfea4a6649d201f41275641a633f218322) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add BuildPerfTest classMarkus Lehtonen2016-07-012-2/+78
| | | | | | | | | | | | | The new class will be used as an abstract base class for build performance tests. This implementation contains some common functionality used in multiple tests, "copied" from the build-perf-test.sh shell script. (From OE-Core rev: 35cd7363759a286e80ddca0d028db3d2bf524b17) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: functionality to drop kernel cachesMarkus Lehtonen2016-07-012-0/+47
| | | | | | | | | | | | | | | | | | Add a new utility class for dropping Linux kernel caches. It uses sudo and tee to write to the drop_caches file. Checking if the user has the permissions to drop caches (without a password) is done by trying to writing an invalid value to the drop_caches file. This way, we will find if writing (with tee) is possible but not really dropping caches, yet. User can avoid giving the password by adding something like: <user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches to the system sudoers file. (From OE-Core rev: c9cb248429ced50c96d11ba5361c272d4c9b9323) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: introduce oeqa.buildperf moduleMarkus Lehtonen2016-07-011-0/+12
| | | | | | | | | | | Wireframe of a new Python module for containing build performance tests and utility functions. (From OE-Core rev: b44af9051a9da8d2cddc7b071a896331967f15cc) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.commands: runCmd: return stderr output, tooMarkus Lehtonen2016-07-011-0/+1
| | | | | | | | | | Useful if one wants to separate stdout and stderr. (From OE-Core rev: de9744c91a997a5ab0e7a19dbe13d8def8d62800) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.commands: use get_bb_vars() in get_bb_var()Markus Lehtonen2016-07-011-15/+5
| | | | | | | | | | Get rid of duplicate code. (From OE-Core rev: cdd6b7386afd460337705d8117a4328d4993ecef) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.commands: Introduce get_bb_vars()Markus Lehtonen2016-07-011-0/+33
| | | | | | | | | | | A new function for getting values of multiple bitbake variables at the same time. (From OE-Core rev: fe3039322e2f846b336ac5af5177e9da27d79695) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: keep platform_extra and default_platform_extra lists ↵Bill Randle2016-07-011-6/+7
| | | | | | | | | | | | | | | | | | ordered In RpmPM:insert_feeds_uris, the paths are kept in sets, which are unordered, but they are later used to set the priority for the Smart channels, so unexpected results could occur. Change the sets to lists and use the same code as in create_configs() to add items to the list, rather than the set operators. [YOCTO #9717] (From OE-Core rev: ce4137f4bb955207fede0c4ef338835d9a461f59) Signed-off-by: Bill Randle <william.c.randle@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetbuild: use os.path.join instead of string concatenationRoss Burton2016-06-231-1/+1
| | | | | | | (From OE-Core rev: 1d40f6e3d85d0cd3be6dd784677686ed4dec89c4) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetbuild: add extra_cmds argument to run_configureRoss Burton2016-06-231-3/+2
| | | | | | | | | | | Some upstreams need more than just gnu-configize ran before ./configure works, such as ./autogen.sh or autoreconf. Add extra_args (defaulting to gnu-configize) so that this can be done in test cases. (From OE-Core rev: 7096f2889f1623ce97a6696f6f4c7217f0efb972) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/targetbuild: fix folder name estimate logicRoss Burton2016-06-231-1/+2
| | | | | | | | | | The re.sub() used to transform a tarball into a best guess folder name wasn't right, as there isn't enough escaping and tar.xz was missing. (From OE-Core rev: ac99135b009a1066486ed2afb2f298d0a5a3a854) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager.py: specify tmpdir for opkgChen Qi2016-06-231-1/+1
| | | | | | | | | | | | | | | Specify tmpdir for opkg via '-t' option so that opkg does not use the default 'TMPDIR' which usually is '/tmp' on build host. This would solve race problems like below. sh: /tmp/opkg-rOG6Tl/opkg-intercept-iPoEp5/depmod: Permission denied (From OE-Core rev: 5665e284b7e7ee071201664dc50a3412bd4ac8cd) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: fix hasPackage, add hasPackageMatchRoss Burton2016-06-235-12/+26
| | | | | | | | | | | | | | | hasPackage() was looking for the string provided as an RE substring in the manifest, which resulted in a large number of false positives (i.e. libgtkfoo would match "gtk+"). Rewrite the manifest loader to parse the files into a proper data structure, change hasPackage to do full string matches, and add hasPackageMatch which does RE substring matches. (From OE-Core rev: b9409863af71899e02275439949e3f4cdfaf2d0f) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager: Fix multilib package arch ordering issuesRichard Purdie2016-06-211-8/+4
| | | | | | | | | | | | | | | | | | | | | Order is not preserved in dict() and this code depends on the order of these lists of package architectures used when multilibs are enabled. This caused 'random' breakage where sometimes the correct order was present and sometimes it wasn't. Use collections.OrderedDict() to avoid this problem. Kudos to Bill Randle and Alejandro Hernandez who did most of the work debugging this, I simply took the problem they identified and wrote a patch to fix it. This unblocks the M1 build but this code needs auditing as there are clearly other ordering issues (e.g. the set() usage). [YOCTO #9717] (From OE-Core rev: 61a33582dfc964d612d20d34734a787d873e312c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/distro_check.py: Fixes for python3Aníbal Limón2016-06-121-48/+34
| | | | | | | | | | | | | | | | | | create_socket: Use urllib because urllib2 is now urllib in python3 and proxies as argument are deprecated so export them in the environ instead. get_links_from_url: Change usage of sgmllib for parsing HTML because is deprecated in python 3, use instead bs4 that is already imported in the bitbake tree. [YOCTO #9744] (From OE-Core rev: ee26ecf58277560459dd01992bb3f486f92c1531) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/terminal.py: decode bytes variable before rstrip/splitLeonardo Sandoval2016-06-121-1/+1
| | | | | | | | | | | | | On python 3, bytes variable types must be decoded if these are intended to be used as strings, otherwise we get the following error exception: TypeError: Type str doesn't support the buffer API (From OE-Core rev: b950539c911b7945d652b05616164828e711ac7f) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/testexport.py: Add test test_testexport_basicMariano Lopez2016-06-071-0/+44
| | | | | | | | | | | | | | | Exported tests is not been tested right now and it get broken very easily with import errors from bitbake. This adds a new test in order to verify in the CI if the testexport gets broken. [YOCTO #8384] (From OE-Core rev: b6e0a762f8a17ee323cef02934f9a624faf706ed) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testexport.bbclass: Add support for testexport-tarballMariano Lopez2016-06-071-1/+34
| | | | | | | | | | | | | | | | | | | | Add support to export the SDK tarball needed when a test system doesn't have the required software to perform runtime tests. The support is when exporting the test and when running the test on a remote system. The user of this feature just need to set TEST_EXPORT_SDK_ENABLED to "1" and declare the sdk packages in TEST_EXPORT_SDK_PACKAGES. [YOCTO #7850] (From OE-Core rev: a6041f81b81baa7564e4c712fc88de2b997e52e4) (From OE-Core rev: 05e6c89f0f71311f8bd32cdb86a2deb789c58035) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oetest.py: Add install/uninstall functionality for DUTsMariano Lopez2016-06-071-0/+74
| | | | | | | | | | | | | | | | Add the functionality to install/unistall packages in the DUTs without the use of the package manager. This is possible with the extraction introduced in package manager class. testimage and testexport bbclasses has been modified in order to support this new feature. [YOCTO #8694] (From OE-Core rev: b7111d9e9d64d21f57729d1ac1865aea6e54cc8b) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oeqa/otest.py: Fix import tests from other layers with python3Mariano Lopez2016-06-073-6/+2
| | | | | | | | | | | | | | In python3 the functionality to import modules has been changed and this broke the capability to add runtime tests from other layers. This commit returns this capability to testimage and testexport. [YOCTO #9705] (From OE-Core rev: a26f23d3ce8f7e9f59dbc9bf27516377fd7a0a6d) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/lib/pacakge_manager.py: Update missing pipeline decodingMariano Lopez2016-06-071-2/+2
| | | | | | | | | | | | Adds decoding needed by some commands output later used as strings. [YOCTO #9702] (From OE-Core rev: 0440b5ace411c61f802376d4e1c9eac93e72d65f) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: adapt for Python 3Ross Burton2016-06-051-2/+2
| | | | | | | | | | string.rstrip() doesn't exist in Python 3, so use the .rstrip method on the object itself instead. (From OE-Core rev: 554719e3f65e468f89a96d4869766d3ff7a8012e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/buildhistory_analysis: fix for Python 3Joshua Lock2016-06-031-10/+10
| | | | | | | | | | The read method of the data_stream File object now returns bytes, not a str, so we must decode it. (From OE-Core rev: cfae302c4996c49a8754497ea9f13f8331d6975d) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testexport.bbclass: Update to use python3Mariano Lopez2016-06-031-1/+1
| | | | | | | | | | | | | Remove deprecated attributes in order to use python3. runexported was changed to use python3. [YOCTO #9702] (From OE-Core rev: 9129af6dc421455c0253be313bf5781b913dc5fd) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* distro_check.py: Don't mix tabs and spacesJussi Kukkonen2016-06-021-2/+2
| | | | | | | (From OE-Core rev: 6c8d1876f5a2d53fa89531e4a5f414d5d5da77be) Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/recipetool.py: Tweak matching of warning lineLeonardo Sandoval2016-06-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | We are observing cases (see below) where the 'WARNING:' prefix is not at the beginning of a line, so instead of expecting it in the beginning, match it within the string. ====================================================================== FAIL: test_recipetool_appendfile_patch (oeqa.selftest.recipetool.RecipetoolTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 285, in test_recipetool_appendfile_patch self.fail('Patch warning not found in output:\n%s' % output) AssertionError: Patch warning not found in output: Parsing recipes..WARNING: File /etc/selftest-replaceme-patched is added by the patch /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta-selftest/recipes-test/recipetool/files/add-file.patch - you may need to remove or replace this patch in order to replace the file. NOTE: Writing append file /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile.bbappend NOTE: Copying /tmp/recipetoolqagci9tita/testfile to /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile/testfile done. (From OE-Core rev: 2289138bdaa17b764821f41a3b3fd0a01cda7440) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Use surrogateescape decodingRichard Purdie2016-06-021-1/+1
| | | | | | | | | | Since the stream can contain invalid binary characters (e.g. from ppc's bootloader) use surrogateescape decoding to ensure we do process the character stream, else it can hang/timeout. (From OE-Core rev: 28a0030430d4cfcaf5dfc3e71bda07cdbfbbf4a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool.py: Fix parsing of bitbake-layers' outputLeonardo Sandoval2016-06-021-5/+4
| | | | | | | | | | | Current parsing was picking wrong targets, leading to the following problem: AssertionError: Command 'bitbake Parsing recipes..done. -e' returned non-zero exit status 1: (From OE-Core rev: eaf83a58825d91c7445835b27d843da7532c208b) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory_analysis: Convert stream data (bytes) to stringsLeonardo Sandoval2016-06-021-1/+1
| | | | | | | | | | | | | The bytes type variables are threat different as strings variables in python3, so convert bytes to strings. This was found when using the script buildhistory-diff, where the interpreter was yielding this error TypeError: Type str doesn't support the buffer API (From OE-Core rev: 3064d36186b47954eb94095217f7bb37e3fce651) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/devtool: Compare sets instead of arrays on AssertEqualLeonardo Sandoval2016-06-021-16/+16
| | | | | | | | | | | | | Sets are safer when comparing internal elements and positions are not important. This commit avoid errors observed on python3 builds as reported on the below bugzilla entry. [YOCTO #9661] (From OE-Core rev: f6df164d09a4d4cf58977bf6cc0bc4f4bc71183a) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Complete transition to python3Richard Purdie2016-06-0229-91/+103
| | | | | | | | | This patch contains all the other misc pieces of the transition to python3 which didn't make sense to be broken into individual patches. (From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Fix encoding errors for python3Ed Bartosh2016-06-021-1/+4
| | | | | | | | | | | | | | Moved call of decode('utf-8') as close as possible to call of subprocess API to avoid calling it in a lot of other places. Decoded binary data to utf-8 where appropriate to fix devtool and recipetool tests in python 3 environment. (From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Fix deprecated dict methods for python3Ed Bartosh2016-06-021-2/+2
| | | | | | | | | | Replaced iteritems -> items, itervalues -> values, iterkeys -> keys or 'in' (From OE-Core rev: 25d4d8274bac696a484f83d7f3ada778cf95f4d0) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: convert data when working with socketEd Bartosh2016-06-021-2/+2
| | | | | | | | | | | | | Converted str to bytes before sending to socket. Converted bytes to str after receiving from socket. This should fix TypeError: 'str' does not support the buffer interface for qemurunner.run_serial method. (From OE-Core rev: 210e290c9251839dc74e3aabdcea3655dd707a50) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Update xrange -> range for python3Richard Purdie2016-06-021-2/+2
| | | | | | | | xrange() no longer exists in python 3, use range() (From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib: Update to explictly create lists where neededRichard Purdie2016-06-0215-57/+57
| | | | | | | | | Iterators now return views, not lists in python3. Where we need lists, handle this explicitly. (From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>