summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/parselogs: Add qemuarm64 warning from 4.8 kernel to whitelistRichard Purdie2016-09-051-0/+1
| | | | | | (From OE-Core rev: ae865fee26d2a32ae07236fc7aa1cf1b234a2156) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/parselogs/qemuarm: Whitelist amba and jitter for 4.8+ kernelsBruce Ashfield2016-09-051-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | With the update to the 4.8 kernel the versatile platform (and hence qemuarm) has switched to a device tree boot. We are using an ummodified mainline kernel versatilepb device tree, which includes definitions of multiple amba devices. These devices are not present in the qemu system emulation, hence throw warnings during boot. These warnings are not unique to oe-core, and rather than carry kernel patches to the device tree (for now), we whitelist the known warnings so qa testing will pass. We also can't turn amba off completely, since it is providing valid devices (like the serial port) and AMBA is force selected by other kconfig values. We also have a jitterentropy warning that shows up on some hosts. This warning is harmless, and like amba we can't turn it off in a fragment since it is force selected by crypto (and we'd rather not turn all crypto off). So we add it to the whitelist while investigations continue into what is needed in the host to support this fully. (From OE-Core rev: f5315b8c7998611da9984fd6bce2b48d6304ff6c) Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image: Deploy images to IMGDEPLOYDIREd Bartosh2016-09-041-2/+2
| | | | | | | | | | | | | | | | | Changed deployment directory from DEPLOY_DIR_IMAGE to IMGDEPLOYDIR to make sstate machinery to do final deployment and generate manifest. Renamed variable deploy_dir to deploy_dir_image in selftest code to avoid confusion with DEPLOYDIR variable. Updated the code of rootfs.py:Rootfs class to use IMGDEPLOYDIR variable as it's now used as a new deployment destination. (From OE-Core rev: 6d969bacc718e21a5246d4da9bf9639dcae29b02) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest Adds eSDK test cases to devtool verification.Francisco Pedraza2016-09-031-0/+103
| | | | | | | | | | The covered functions are, install libraries headers and image generation binary feeds. (From OE-Core rev: 994f8a41a16d0b82a1f7dfbcbbcc1df08225b14e) Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* /oeqa/sdkext Adds verification for devtool on eSDK.Francisco Pedraza2016-09-033-7/+103
| | | | | | | | | | | The covered funcions are, build make, build esdk package, build cmake extend autotools recipe creation, kernel module, node.js installation and recipe creation. (From OE-Core rev: 574a5d4cf3e79815aecc4d198545119d3bbfb023) Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: devtool: fix test after recent changePaul Eggleton2016-09-031-1/+3
| | | | | | | | | | | | OE-Core commit d3057cba0b01484712fcee3c52373c143608a436 fixed handling of wildcard bbappends, which means that this test's expectations about the bbappend file name are no longer met. devtool finish is meant to use wildcard bbappends so fix the test accordingly. (From OE-Core rev: 21603566e4a2e709dcb4a940b49d870c91c822be) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.selftest.liboe: add test for xattr in copytreeJoshua Lock2016-09-031-1/+32
| | | | | | | | | | Add a test to ensure that oe.path.copytree() preserves extended attributes on files. (From OE-Core rev: 2b047b8e3218f95978e41fee13635bff9af03dd6) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.selftest: add a test for oe.path.copytree()Joshua Lock2016-09-031-0/+33
| | | | | | | | | | | | One motivation for the use of cpio in oe.path.copytree() was to ensure that files with spaces in their names were copied. Add a new unittest module to test the OE module with a test case for copytree with a spaces in a filename. (From OE-Core rev: a408f8310d9426db4439cf8db0cf49f9bfe90b3b) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: include commands log file name in results.jsonMarkus Lehtonen2016-09-031-5/+9
| | | | | | | (From OE-Core rev: b22a71cf3a53a33763ff02608119d2c73cbde006) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: include buildstats file name in results.jsonMarkus Lehtonen2016-09-032-8/+17
| | | | | | | | | No need to do lsdir magic for finding buildstats when reading results. (From OE-Core rev: 4502f0979bf2e8698bb196345b89b170641fd43f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: show skipped tests in results, tooMarkus Lehtonen2016-09-031-1/+2
| | | | | | | (From OE-Core rev: 4112779f9f314148b475fc4b8e33146de8be6b27) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: convert buildstats into json formatMarkus Lehtonen2016-09-031-3/+66
| | | | | | | | | | | | Instead of archiving buildstats in raw text file format convert all buildstats into one json-formatted file. Some redundant information, i.e. 'Event:', 'utime:', 'stime:', 'cutime:' and 'cstime:' fields, are dropped. (From OE-Core rev: efcf74b194f2a40eb3e6359dd41386db3eb25287) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: measure io statMarkus Lehtonen2016-09-031-2/+9
| | | | | | | | | Add data from /proc/<pid>/io to system resource measurements. (From OE-Core rev: e69a46a77854fac1169a09e0c5b70fa4b972255a) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: don't use Gnu timeMarkus Lehtonen2016-09-031-33/+39
| | | | | | | | | | | | | Use Python standard library functionality instead of the time utility for measuring elapsed (wall clock) time of commands. The time.* log files are also ditched. However, the same detailed resource usage data, previously found in time.* logs is now provided in results.json file. This data is collected through the resource module of Python. (From OE-Core rev: d5ad818dd501b18379aa3540bffa9b920d7c3bab) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: rename buildstats directoriesMarkus Lehtonen2016-09-031-2/+3
| | | | | | | | | | | | Change directory name from 'buildstats-<test_name>' to just 'buildstats'. However, this patch adds the possibility to label buildstats directory name with a postfix which makes it possible to save multiple buildstats per test, for example. (From OE-Core rev: 8997556040b2e7bfcfa6a75d4d97eb2e32207217) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: separate output dir for each testMarkus Lehtonen2016-09-031-1/+2
| | | | | | | | | | Store the output data of each test in an individual subdirectory instead of storing everything in the root output directory. (From OE-Core rev: 64ff34df96aa9a74dd4303f76ec711aa5e9d5030) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: strip date from buildstats directory pathMarkus Lehtonen2016-09-031-1/+5
| | | | | | | | | | Archive buildstats in a directory like 'buildstats' instead of something like 'buildstats/20160513120000'. (From OE-Core rev: 95138cdc70bb7f9b7ab74e1d83305f009790dccc) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: enable json-formatted resultsMarkus Lehtonen2016-09-031-0/+40
| | | | | | | | | | Automatically create a json.formatted file (results.json) in the results directory that contains results from all tests. (From OE-Core rev: 6df3263531a41805b2280bb999cb4a73f9f91eae) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add 'product' to test result dataMarkus Lehtonen2016-09-031-0/+1
| | | | | | | | | | This defaults to 'oe-core' but can be defined using the OE_BUILDPERF_PRODUCT environment variable. (From OE-Core rev: a22cc3e04001be5d11bd85dbdceb7088cae7c735) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: treat failed measurements as errorsMarkus Lehtonen2016-09-031-23/+13
| | | | | | | | | | | Now failed measurements correctly cause a test failure (recorded as an error). There should be no need to continue the test if one step fails, especially now that the tests don't depend on each other. (From OE-Core rev: 446e32aadc775ca146d12173b1463f524d7fe6ef) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: make tests independentMarkus Lehtonen2016-09-032-3/+14
| | | | | | | | | | | | | | | | | Add test set-up functionality so that the individual tests do not depend on each other. This should make sure that a failure in one test does not affect the results of another test. The patch also makes it reasonable to run only a subset of the tests by using the --run-tests option. The increase in total execution time of the full suite - caused by the additional set-up steps - is insignificant because normally no additional tasks need to be run. The previous test has already done all set-up work. (From OE-Core rev: 69b3c63e32d09ea4a41b21daacdff6bf1fc447c1) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: fix checking of invalid resultsMarkus Lehtonen2016-09-031-1/+1
| | | | | | | | | The test status check done when writing globalres log was incorrect. (From OE-Core rev: 3efbd49fd80d2b349a8fd44dbcd509168dbc1061) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: add {git_commit_count} keyword for --commit-results-tagMarkus Lehtonen2016-08-251-0/+1
| | | | | | | | | | | Makes it possible to create easily sortable tags. Also, the default tag format is updated to use the new keyword. (From OE-Core rev: e3161654d75dfc3b059c519205b38b26e3ffb215) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add git commit count to result dataMarkus Lehtonen2016-08-251-7/+11
| | | | | | | | | | | This number represents the number of commits since the beginning of git history until the tested revision. This helps e.g. in ordering results. (From OE-Core rev: b52070dd057ff5b410cd193f9be2f25bc4c506cc) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: new {tag_num} keyword for --commit-results-tagMarkus Lehtonen2016-08-251-4/+15
| | | | | | | | | | | | | | | | | | | | | | This makes it possible to create numbered tags, where the "basename" of the tag is the same and the only difference is an (automatically) increasing index number. This is useful if you do multiple test runs on the same commit. For example, using: --commit-results-tag {tester_host}/{git_commit}/{tag_num} would give you tags something like: myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/0 myhost/decb3119dffd3fd38b800bebc1e510f9217a152e/1 ... The default tag format is updated to use this new keyword in order to prevent unintentional tag name clashes. (From OE-Core rev: cf2aba16338a147f81802f48d2e24a96c7133548) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: tag results committed to GitMarkus Lehtonen2016-08-251-1/+10
| | | | | | | | | | | | | | | | | | | | Create a Git tag when committing results to a Git repository. This patch also implements --commit-results-tag command line option for controlling the tag name. The value is a format string where the following fields may be used: - {git_branch} - target branch being tested - {git_commit} - target commit being tested - {tester_host} - hostname of the tester machine Tagging can be disabled by giving an empty string to --commit-results-tag. The option has no effect if --commit-results is not defined. (From OE-Core rev: 60059ff5b81d6ba9ba344161d51d1290559ac2df) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git: implement init() methodMarkus Lehtonen2016-08-251-0/+6
| | | | | | | | | | Method for doing 'git init'. (From OE-Core rev: c848e1dac68cd859a563a82286f8bc5ddabaa423) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: implement --commit-results-branchMarkus Lehtonen2016-08-251-0/+5
| | | | | | | | | | | | | | | | A new command line option for defining the branch where results are commited. The value is actually a format string accepting two field names: - {git_branch} expands to the name of the target branch being tested - {tester_host} expands to the hostname of the tester machine The option has no effect if --commit-results is not used. (From OE-Core rev: b54b63395ec632748a57a702812c8a9a07af35ab) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-test: support committing results data to GitMarkus Lehtonen2016-08-251-0/+35
| | | | | | | | | | | | Implement a new command line option '--commit-results' which commits the test results data into a Git repository. The given path must be an existing initialized local Git repository. (From OE-Core rev: b6f635513ca971402e7a970acc2168fb5d4a9476) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: use term commit instead of revisionMarkus Lehtonen2016-08-251-15/+15
| | | | | | | | | | | | This is basically a internal change, at this point. Term 'commit' better represents the data we actually have. Term 'revision' is more vague and could be understood to point to a tag object, for example. (From OE-Core rev: f49cf7959b8aaa52b79b22a5884c6aa580a50302) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git.GitRepo: new arg to require topdirMarkus Lehtonen2016-08-251-2/+7
| | | | | | | | | | | | | | | Add a new 'is_topdir' argument to the GitRepo init method which validates that the given path is the top directory of a Git repository. Without this argument GitRepo also accepts subdirectories of a Git repository (in which case GitRepo will point to the parent directory that is the top directory of this repository) which may have undesired in some cases. (From OE-Core rev: 044c81bd916fbe7140d184eb103f74786cfef604) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git: implement GitRepo.get_current_branch()Markus Lehtonen2016-08-252-5/+10
| | | | | | | | (From OE-Core rev: dcba2302adab47b398f1ce7d09c38828ea9ae426) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git: introduce GitRepo.rev_parse()Markus Lehtonen2016-08-252-1/+8
| | | | | | | | (From OE-Core rev: 55726e931536ed0cbd7b80588060b05a3145c934) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git: support git commands with updated envMarkus Lehtonen2016-08-251-2/+6
| | | | | | | | | | | | Extend GitRepo.run_cmd so that the caller may redefine and/or define additional environment variables that will be used when the git command is run. (From OE-Core rev: 9b3c7c47f5d0fa473fe1db81b59b26531414781c) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/rpm: use su instead of sudoRoss Burton2016-08-251-3/+2
| | | | | | | | | | This test works fine with su, which is more likely to be installed in images than sudo. (From OE-Core rev: 59d10be745a1f7d31c68e4d5da9e1c3461b7d390) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs: Ignore uvesafb timeoutsRichard Purdie2016-08-251-0/+5
| | | | | | | | | | | | | | | We're periodically seeing uvesafb timeouts on the autobuilder. Whitelist these errors as there is little it seems we can do about them and we therefore choose to ignore them rather than fail the builds. [YOCTO #8245] There is a better solution proposed in the bug with a -1 timeout however this avoids failed builds until such times as that is implemented. (From OE-Core rev: 8097f2da79b7862733494d2321e3dfdb0880804d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/buildiptables: Switch from netfilter.org to yoctoproject.org mirrorRichard Purdie2016-08-232-2/+2
| | | | | | | | | We've had some upstream mirror instability so use our own mirror for the iptables sources to ensure this doesn't affect the test results. (From OE-Core rev: 25f6af8895d5f5c6dcedde0a21285d63522769c8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: fix crash when creating globalres.logMarkus Lehtonen2016-08-201-1/+1
| | | | | | | | | Fix a bug that was introduced when converting to unittest framework. (From OE-Core rev: 3bdb7b2e512b2f160360e95ed5b2be3871ec0b4b) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/recipetool/meta: Adapt to bitbake API changes for ↵Richard Purdie2016-08-181-1/+1
| | | | | | | | | | | | | | multi-configuration builds Unfortunately to implenent multiconfig support in bitbake some APIs had to change. This updates code in OE to match the changes in bitbake. Its mostly periperhal changes around devtool/recipetool [Will need a bitbake version requirement bump which I'll make when merging] (From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs: Ignore amb_nb warning messages under qemux86*Richard Purdie2016-08-181-0/+1
| | | | | | (From OE-Core rev: 857f4ca134e4575e71993b4fa255ebafec612d1e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: Add failed to setup card detect gpio error on x86California Sullivan2016-08-181-0/+1
| | | | | | | | | | | This error has occurred on the MinnowBoard Max and Turbot since its inception. It supposedly indicates a non-working SD card reader, but ours works fine. Whitelist the error. (From OE-Core rev: d577028a1d756b70da056dee73df657cf8000baf) Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: Add dmi and ioremap errors to ignore list for core2California Sullivan2016-08-181-0/+2
| | | | | | | | | | These errors have been occuring since the introduction of the 4.4 kernel with no apparent functionality loss. Whitelist for now. (From OE-Core rev: 47b9058994f15507fc18ce0b08ac82a4c052966e) Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parselogs.py: Ignore Skylake graphics firmware load errors on genericx86-64California Sullivan2016-08-181-1/+7
| | | | | | | | | | | | These errors can't be fixed without adding the firmware to the initramfs and building it into the kernel, which we don't want to do for genericx86-64. Since graphics still work acceptably without the firmware blobs, just ignore the errors for that MACHINE. (From OE-Core rev: d73a26a71b2b16be06cd9a80a6ba42ffae8412c4) Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: allow using kvm when running qemux86* machinesBill Randle2016-08-182-2/+16
| | | | | | | | | | | | Using kvm can provide significant speedups when running qemux86* machines on an x86* host. Enabled by using the new QEMU_USE_KVM variable. [YOCTO #9298] (From OE-Core rev: ebac2c8d1fcd09ebce0659a4abb445e4f1c18571) Signed-off-by: Bill Randle <william.c.randle@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: use oe.path.remove()Markus Lehtonen2016-08-172-13/+7
| | | | | | | | | | Drop the self-baked force_rm() method. (From OE-Core rev: c86bf80abd87acb0da5860806822c64ec9dee089) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: be more verbose about failed commandsMarkus Lehtonen2016-08-171-3/+9
| | | | | | | | | | Log failures of commands whose output is stored. (From OE-Core rev: 240f6e7366c8a9ea830e531d307dd2e27a61a6bd) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: introduce runCmd2()Markus Lehtonen2016-08-172-6/+12
| | | | | | | | | | | | | | Special runCmd() for build perf tests which doesn't raise an AssertionError when the command fails. This causes command failures to be detected as test errors instead of test failures. This way "failed" state of tests is reserved for future making it possible to set e.g. thresholds for certain measurement results. (From OE-Core rev: 09590ac76a19ee1b1b4a9188f7fce5029f0de52a) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: convert test cases to unittestMarkus Lehtonen2016-08-171-31/+19
| | | | | | | | | | | This commit converts the actual tests to be compatible with the new Python unittest based framework. (From OE-Core rev: 4e81967131863df7ee6c8356cb41be51f1b8c260) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add BuildPerfTestResult classMarkus Lehtonen2016-08-172-74/+80
| | | | | | | | | | | | | The new class is derived from unittest.TextTestResult class. It is actually implemented by modifying the old BuildPerfTestRunner class which, in turn, is replaced by a totally new simple implementation derived from unittest.TestRunner. (From OE-Core rev: 89eb37ef1ef8d5deb87fd55c9ea7b2cfa2681b07) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.buildperf: add BuildPerfTestLoader classMarkus Lehtonen2016-08-172-0/+6
| | | | | | | | (From OE-Core rev: b281c4a49b0df1de9b3137efb8ff50744e06c48d) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-13 23:45:19 +0000