summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
Commit message (Collapse)AuthorAgeFilesLines
* classes/testsdk: Split implementation into classesJoshua Watt2018-11-212-0/+246
| | | | | | | | | | | | | | | | | | | | | | | | | Splits the SDK test implementation into configurable Python classes. The classes used for the normal and extensible SDKs are ${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively. This allows SDK machines to override the classes used to implement the tests. For the traditional SDK, a common "run()" function is provided by the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions that can be overridden in child classes, making it easier to have consistent behavior. The extensible SDK class (oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but no hooks have yet been added as there is not currently a known use case for create derived classes. These changes should be purely organizational; no functional changes have been made to either the standard SDK or extensible SDK tests. [YOCTO #13020] (From OE-Core rev: a06d53928b22d5f88276023c4d57b206db2f27f9) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/httpserver: Rework to avoid hangs and improve loggingRichard Purdie2018-11-164-11/+32
| | | | | | | | | | | | | | | | | | testimage.bbclass installs a SIGTERM handler which conflicts with the use of multiprocessing here. This is paritcularly problematic if the http service is terminated before its started and hence before its had a chance to reset the default signal handler (as the code was written). Instead, temporarily remove testimage's handler whilst forking the http process which means the correct handler is installed and won't deadlock. Also take the opportunity to add in some log messages about the server start and shutdown so that future debugging is easier and its clearer what the code is doing. (From OE-Core rev: 0762b6021b87ceb1f37952f3a6d64a36e99ae6a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation ↵Richard Purdie2018-11-161-1/+2
| | | | | | | | | | warnings Use our own lsb function instead as used elsewhere by the codebase. (From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/concurrencytest: Avoid unclosed file warningsRichard Purdie2018-11-161-0/+3
| | | | | | | | | Avoid an unclosed file per thread warning when running selftests concurrently by closing the result stream. (From OE-Core rev: 33a4a076e8aa72a872807332501e7f5ae1cee0e2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner.py: Fix python regex warningsRichard Purdie2018-11-161-5/+5
| | | | | | | | | | | | | | | | | | | Fix the warnings: meta/lib/oeqa/utils/qemurunner.py:250: DeprecationWarning: invalid escape sequence \. ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) meta/lib/oeqa/utils/qemurunner.py:343: DeprecationWarning: invalid escape sequence \- if re.search("root@[a-zA-Z0-9\-]+:~#", output): poky/meta/lib/oeqa/utils/qemurunner.py:350: DeprecationWarning: invalid escape sequence \- if re.search("root@[a-zA-Z0-9\-]+:~#", output): meta/lib/oeqa/utils/qemurunner.py:448: DeprecationWarning: invalid escape sequence \- if re.search("[a-zA-Z0-9]+@[a-zA-Z0-9\-]+:~#", data): by correctly marking the regexs. (From OE-Core rev: 8e6987735002560fca714f77ea8ece9d4b28f7fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: Replace deprecated imp module usageRichard Purdie2018-11-161-2/+2
| | | | | | | | | | | | | Avoid the warning: meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp In this case importlib is a direct replacement. (From OE-Core rev: db7a60c36a2d3eefc61ae6e1ede01680dc932035) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: Avoid unclosed file warningsRichard Purdie2018-11-141-0/+3
| | | | | | | | | | | Avoid warnings such as: meta/lib/oeqa/utils/commands.py:213: ResourceWarning: unclosed file <_io.BufferedReader name=4> return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options) (From OE-Core rev: 6a68c42de08cffbadb59ebda63fa5e19f6e5acef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/runner: Correctly markup regexsRichard Purdie2018-11-141-2/+2
| | | | | | | | | Avoid the warning "DeprecationWarning: invalid escape sequence \(" by marking the regexs correctly. (From OE-Core rev: cb49980fa4a158d5529902df731dec61a8c9b3d4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/loader: Fix deprecation warningRichard Purdie2018-11-141-1/+1
| | | | | | | | | | Clean up the warning: meta/lib/oeqa/core/loader.py:27: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec() _failed_test_args = inspect.getargspec(unittest.loader._make_failed_test).args (From OE-Core rev: d2deb66830be2d44532fea3d5db763b57778252a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runner: Sort the test result output by result classRichard Purdie2018-11-141-1/+10
| | | | | | | | | | | We want to see failures/errors listed last since this is the most easily visible part of the log on consoles or autobuilder output and makes human processing easier rather than having to scroll up and scan for a single failure. (From OE-Core rev: 7954b19020c28a4120bc1671aa81b9e1e2b05fa2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runner: Always show a summary of success/fail/error/skip countsRichard Purdie2018-11-141-3/+1
| | | | | | | | | Its useful to have the counts of success/failure/error/skipped at the end of the results to allow for easier human reading of what happened. (From OE-Core rev: 080d8900d470a8e7f929b0c5c2765ad461744fbb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: Improve dependencies of kernel/gcc/build testsRichard Purdie2018-11-146-30/+31
| | | | | | | | | Mark up these tests as needing a compiler, make and kernel source code as appropriate, the image feature requirements can then be retired. (From OE-Core rev: 9f64e6c25abdf494fb511e9cd401f8dcaa08be2a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/cases: Improve test dependency informationRichard Purdie2018-11-1410-25/+32
| | | | | | | | | | | | Add the OEHasPackage decorator to a variety of tests so they determine automatically if they should run against a given image. To ensure tests can do this we need to move target operations such as scp commands into the tests and out of the class startup/teardown. (From OE-Core rev: 60d6580b85714b8960a964e775d76a7f937f5e5a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testsdk: fix skipped testcase output "UNKNOWN" status while multiprocess ↵Hongxu Jia2018-11-146-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | execution Usually skipped testcase output "SKIPPED" [snip serial execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: SKIPPED (0.01s) |RESULTS - python.PythonTest.test_python3 - Testcase -1: SKIPPED (0.01s) [snip serial execution] But if enable multiprocess execution, skipped testcase output "UNKNOWN" status [snip enable multiprocess execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: UNKNOWN |RESULTS - python.PythonTest.test_python3 - Testcase -1: UNKNOWN [snip enable multiprocess execution] Here is my investigation: There is a class pairs TestProtocolClient and TestProtocolServer provided by python3-subunit. The TestProtocolClient generates a subunit stream of TestResult from a test run, and TestProtocolServer parses the stream of subunit TestResult. The class ProtocolTestCase is a unittest.TestCase adapter and it uses TestProtocolServer to parse the stream of subunit TestResult. In Yocto testsdk, it forks multiple processes to execute testcases and use TestProtocolClient to generate TestResult stream; and then it creates multiple threads to use ProtocolTestCase to parse stream of subunit TestResult through pipe; finally it passes multiple ProtocolTestCase as TestCase instance to main process and output status result. The problem point is TestProtocolServer parses `skip:' directive after reading a `test:' directive. Without `test:' directive, `skip:' directive will be ignored. All above requires SkipTest should be raised inside a test method rather than setUpClass method. Throwing SkipTest inside setUp works correctly (From OE-Core rev: 4828a88556d59e4d06933164c2ebeb9361b7450e) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/utils/targetbuildproject: Use a subdir within ~/Richard Purdie2018-11-141-1/+3
| | | | | | | | | Without doing this, the code can and sometimes does try and do "rm ~/" which I think we'd all prefer it didn't. (From OE-Core rev: 9dc1de59330c366783ea043c68c1b59b1b49e707) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/buildproject: Only clean files if we've done somethingRichard Purdie2018-11-141-0/+5
| | | | | | | | | We should only be wiping out things on target if the tests have actually run. (From OE-Core rev: d38c3eac0a5a1a9b0eb98385832e92f48145655e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual/build-appliance: Drop test which is already automatedRichard Purdie2018-11-141-26/+0
| | | | | | | | The nightly-build-appliance target already tests this particular build. (From OE-Core rev: 02c5f853f30568eb3f289c6b6fb5cacf712d15f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: Add tests for gstreamer CLIYeoh Ee Peng2018-11-141-0/+14
| | | | | | | | | | | | | | | | | | | We should not be finding GStreamer issues right around release time but that's what we seem to do. Port gstreamer tests from Intel Ref Kit. https://github.com/intel/intel-iot-refkit/blob/master/meta-iotqa/ lib/oeqa/runtime/multimedia/gstreamer/gstreamer_cli.py Add tests for gstreamer commandline tools (gst-inspect-1.0 & gst-launch-1.0). [YOCTO #11418] (From OE-Core rev: 3e77f845b517e3f4b3e7d09d088a9a80bf171653) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/recipetool: Fix problems from changing upstream sourceMohamad Noor Alim Hussin2018-11-091-2/+2
| | | | | | | | | | | | The upstream source tarball checksums changed. Use the copy from our source mirror to avoid failures. [YOCTO #12979] (From OE-Core rev: e97a31e6bbaec5cb56d4750bf5171dbba510ee33) Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* image-buildinfo,oeqa/selftest/containerimage: Ensure image-buildinfo doesn't ↵Richard Purdie2018-11-091-0/+1
| | | | | | | | | | | | | | | | | | | break tests Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files to fail due to the presence of an unexpected file: ['./', './etc/', - './etc/build', './etc/default/', './etc/default/postinst', Tweak the class to allow it to be disabled and disable it from the test just in case it was enabled. (From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Standardize json logging output directoryYeoh Ee Peng2018-11-091-1/+1
| | | | | | | | | | | | Currently sdk & sdkext will output json file to LOG_DIR, while selftest will output json file to TOPDIR/log. Standardize selftest json output file to LOG_DIR. (From OE-Core rev: 2012d6b076fc833864a0254d56d78536314a6799) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Add test for Yocto source mirror functionality/completenessRichard Purdie2018-11-071-0/+23
| | | | | | | | | | | | | | We've had a number of occasions where the Yocto Project source mirrors have not been complete or functioning correctly. This adds a test so that if this happens we find out out it sooner. It also only works over http meaning we should be able to test that anyone behind an http only proxy (no git protocol) also has functional fetches for OE-Core and layers built by the core of the project. (From OE-Core rev: 9a25b2eb9cc13dd77a774dad83067ea5f9bf9716) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/ptest: Inject results+logs into stored json results fileRichard Purdie2018-11-072-2/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows the ptest results from ptest-runner, run in an image to be transferred over to the resulting json results output. Each test is given a pass/skip/fail so individual results can be monitored and the raw log output from the ptest-runner is also dumped into the results json file as this means after the fact debugging becomes much easier. Currently the log output is not split up per test but that would make a good future enhancement. I attempted to implement this as python subTests however it failed as the output was too confusing, subTests don't support any kind of log output handling, subTest successes aren't logged and it was making things far more complex than they needed to be. We mark ptest-runner as "EXPECTEDFAILURE" since its unlikely every ptest will pass currently and we don't want that to fail the whole image test run. Its assumed there would be later analysis of the json output to determine regressions. We do have to change the test runner code so that 'unexpectedsuccess' is not a failure. Also, the test names are manipuated to remove spaces and brackets with "_" used as a replacement and any duplicate occurrences truncated. (From OE-Core rev: a13e088942e2a3c3521e98954a394e61a15234e8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/metadata: Allow to function without the git moduleRichard Purdie2018-10-291-2/+15
| | | | | | | | | | The python git module may or may not be enabled, allow this code to function without it, falling back to the same method as metadata_scm.bbclass uses. This will be cleaned up in the next round of feature development. (From OE-Core rev: 6350586ba9f4a4107a2d457590824cd4d662d5b9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/runner: Don't add empty log entriesRichard Purdie2018-10-291-1/+4
| | | | | | | | | There is no point in adding empty log entries to the json result files, only add them if there is log data. (From OE-Core rev: da6aef0b6bd4655805f7f743858d4e04341f75ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Improvements to the json loggingRichard Purdie2018-10-291-14/+11
| | | | | | | | | | | | | Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes (From OE-Core rev: 173f59acf9722e2ef27fdd49c20f7d3d664917eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/context: write testresult to json filesYeoh Ee Peng2018-10-291-3/+31
| | | | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the oe-selftest log directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. (From OE-Core rev: 10697165c832e3dbb2913b6215164ea75e23ec23) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/runner: write testresult to json filesYeoh Ee Peng2018-10-291-1/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia to store testresult, OEQA need to output testresult into single json file, where json testresult file will be stored in git repository by the future test-case-management tools. The json testresult file will store more than one set of results, where each set of results was uniquely identified by the result_id. The result_id would be like "runtime-qemux86-core-image-sato", where it was a runtime test with target machine equal to qemux86 and running on core-image-sato image. The json testresult file will only store the latest test content for a given result_id. The json testresult file contains the configuration (eg. COMMIT, BRANCH, MACHINE, IMAGE), result (eg. PASSED, FAILED, ERROR), test log, and result_id. Based on the destination json testresult file directory provided, it could have multiple instances of bitbake trying to write json testresult to a single testresult file, using locking a lockfile alongside the results file directory to prevent races. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. (From OE-Core rev: 00e03b5004f1eb6d59295544b3a8620504278f51) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: devtool: Support meta being a symbolic linkuninative-2.3Peter Kjellerstedt2018-10-211-1/+2
| | | | | | | | | | | oe-selftest's devtool tests have been broken since commit 2457cd57 (oe-selftest: devtool: avoid parallel races by using temporary copy of core) if meta is a symbolic link. (From OE-Core rev: daba6c5a991b370709d17e51305334f55a3858ec) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/runner: refactor for OEQA to write json testresultYeoh Ee Peng2018-10-181-41/+29
| | | | | | | | | | | | | | Refactor the original _getDetailsNotPassed method to return testresult details (test status and log), which will be reused by future OEQA code to write json testresult. Take the opportunity to consolidate and simplify the logic used to gather test status and log within the TestResult instance. (From OE-Core rev: 79ee7d1c371a86edeb61c99679985118da657e5d) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatetests.py: reset TCLIBCAPPENDKai Kang2018-10-161-0/+12
| | | | | | | | | | | | | | It appends TCLIBCAPPEND to TMPDIR in meta/conf/distro/defaultsetup.conf: TMPDIR .= "${TCLIBCAPPEND}" It affects some oe selftest cases in sstatetests.py. Reset TCLIBCAPPEND for these cases. (From OE-Core rev: 4549c9dd96be8db69d520f66f4507939df9e9587) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* ksample.py: fix error messageChen Qi2018-10-121-1/+1
| | | | | | | | | | | | | | | The current error message is like: kobject-example.kodoesn't exist Add a space so that it looks like: kobject-example.ko doesn't exist (From OE-Core rev: 30f2cdd0a3ee0874bebd135e35ab591f630e8adc) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual/oe-core: Drop image build during taskexp testRichard Purdie2018-10-111-4/+0
| | | | | | | | | Building an image takes time and isn't necessary to test taskexp, we can drop this step. (From OE-Core rev: 298eb27037172925a83b37541e5955bbf3b7d2fb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual: Remove bitbake-selftest executionRichard Purdie2018-10-111-30/+0
| | | | | | | | | The autobuilder runs bitbake-selftest already so we don't need to execute this manually. (From OE-Core rev: f223f49b7e3d3f6ef07915edea1a6233ae9869e2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/manual: transfer manual test cases from testopia to oeqaYeoh Ee Peng2018-10-119-0/+2750
| | | | | | | | | | | | | | | | | | | | | | As part of the solution to replace Testopia, manual test cases need to be migrated to OEQA. These manual test case json files will serve two use cases. Use case#1: as input to the future commandline-based manual test runner script, where this script will display actions and expected result information in user friendly text, then it will capture user input for test result and log, finally it will write test result and log into existing standardize test result json format from OEQA framework for automated tests. Use case#2: QA will open and read these json file manually for planning manual test execution. Any reader interested in understanding manual test cases will open and read these files. (From OE-Core rev: dcaf843878e9adcb3fab025d54cd5d9ceb6c2fce) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/opkg: Ensure the test works on multilibRichard Purdie2018-10-111-1/+4
| | | | | | | | | After allarch was disabled in multilib, this test broke. Fix the test to account for that change. (From OE-Core rev: 6f692d86c2c6742c67aba93fa24b930af02d8e85) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/runqemu: Handle SystemExitRichard Purdie2018-10-091-8/+11
| | | | | | | | | | | | | | | | | | | The sigchld handler in runqemu can raise a SystemExit when qemu shuts down. Rather than backtracing, accept this as a successful test result. ERROR: runqemu.QemuTest.test_qemu_can_shutdown (subunit.RemotedTestCase) testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 183, in test_qemu_can_shutdown qemu_shutdown_succeeded = self._start_qemu_shutdown_check_if_shutdown_succeeded(qemu, shutdown_timeout) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 175, in _start_qemu_shutdown_check_if_shutdown_succeeded time.sleep(1) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/qemurunner.py", line 100, in handleSIGCHLD raise SystemExit SystemExit (From OE-Core rev: 417245923c1c2c35a60d6db29cbe5a78548860d2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Remove the signal handler before stopping qemuRichard Purdie2018-10-091-2/+2
| | | | | | | | | | | The qemu shutdown can race with the signal handler removal leading to confusing tracebacks on slower/loaded systems. Remove the signal handler first before shutting down. (From OE-Core rev: 1b63ce7bb83134f84bf07d1075d5ca0d5466ed3d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/recipetool: fix non-determinism in cmake testRoss Burton2018-09-131-1/+3
| | | | | | | | | | | | | | The Navit recipe can depend on fribidi which is now part of oe-core. Since the pkgconfig lookup is based on pkgdata the results can change depending on what has been built, which leads to occasional failures. Build gtk+ before the test to ensure that the pkgdata is populated with more of the dependencies, and add fribidi to the generated DEPENDS checklist. (From OE-Core rev: d180dc758710c7259d45eeb9304e7284a8fd8825) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/runtime_test: Ensure we build/use gnupg-nativeRichard Purdie2018-09-131-1/+3
| | | | | | | | | Without this, we see errors if gpg is missing from the host system for "oe-selftest -r runtime_test.TestImage.test_testimage_dnf". (From OE-Core rev: e91838b63b506e2969582b2b8511fd3724d6aa3f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* insane: Recognise BPF as a valid EM_MACHINE typeKhem Raj2018-09-131-0/+1
| | | | | | | | | | | | | | | | BPF Linux ELF objects are generated with kernel-selftests with >= 4.18 kernel and when clang is enabled which packages BPF objects into packages, therefore recongnise this as a valid ELF target Add a selftest for BPF Do not flag BPF objects in target, since they pretty much will be ok for most of kernels architectures we care do support BPF (From OE-Core rev: 3667a8ec016bae3f8026ef7b4c895546804f6368) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* statetests.py: drop test_sstate_allarch_samesigs_multilibKai Kang2018-09-131-9/+2
| | | | | | | | | | | allarch is disabled when multilib is used, so sstate oeqa case test_sstate_allarch_samesigs_multilib is useless. Remove check for allarch part and rename to test_sstate_nativesdk_samesigs_multilib. (From OE-Core rev: 32fe47ea0aea791357d3045c202cdad86b16f2ff) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/runtime_test: Fix ↵Richard Purdie2018-09-041-0/+1
| | | | | | | | | | | | | | runtime_test.TestImage.test_testimage_install after ssl changes The test installs socat and expects it to run but assumes all its dependencies are already installed (which includes libssl). Recent changes mean this isn't the case so force libssl into the image in advance to work around this issue for now. (From OE-Core rev: 13b82380abb273e6575b1e74870d3fe8249ebbc1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/dnf: Make sure test_dnf_install does not skippedYeoh Ee Peng2018-09-041-1/+5
| | | | | | | | | | | | | | | | | | | | During debugging dnf issue, we found that the test_dnf_install PASSED the testing even though the environment does not allow dnf install to run successfully. Further debugging had identified that current test_dnf_install will execute dnf install even when the package to be installed already exist, thus dnf install will just skipped and this test will PASSED even though it was not. To solve this, added additional logic to check if the package to be installed already exist, if yes then remove the package before actually run dnf install. This will make sure dnf install was tested as expected. (From OE-Core rev: 4f662b253f7313c4e02bfafb527cdac076b6309a) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: fixes related to hasPackage semanticsChen Qi2018-09-044-13/+20
| | | | | | | | | | | | | | | | | | | | | | | | The current _hasPackage does a regex match when checking for the existence of packages. This will sometimes result in unexpected result. For example, the condition hasTargetPackage('gcc') is likely to be always true as it matches libgcc1. For most of the time, we should do exact match instead of regex match. So change _hasPackage function to do that. For the current sdk test cases, the only place that needs regex match is '^gcc-'. This is because there's no easy way to get multilib tune arch (e.g. i686) from testdata.json file. Besides, packagegroup-cross-canadian-xxx and gcc-xxx should be check in host manifest instead of the target one. So fix to use hasHostPackage. Also, as we are doing exact match, there's no need to use r'gtk\+3', just 'gtk+3' is enough. (From OE-Core rev: 595e9922cdbacf84cf35cc83f0d03cace042e302) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdk/buldgalculator.py: check against multilib for gtk+3Chen Qi2018-09-041-2/+2
| | | | | | | | | | | | | When determining whether to skip the test case, the check should be done with consideration of multilib. Otherwise, we will meet the following error when testing against lib32 environment. No package 'gtk+-3.0' found (From OE-Core rev: ce82ee46f4a7beb5663238b276e779e5c9657777) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sdk/context.py: add ability to check for multilib version of target packageChen Qi2018-09-041-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a named argument 'multilib' for the hasTargetPackage function. Its default value is False. When setting to True, it will try to get the correct multilib prefix from the sdk_env, the environment setup script. We need this because we don't want unexpected run of some sdk test cases. The following steps will generate error. 1. Enable multilib for qemux86-64 require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 ?= "core2-32" 2. bitbake core-image-sato -c populate_sdk 3. bitbake core-image-sato -c testsdk The error message is like below. No package 'gtk+-3.0' found RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: FAILED As we don't have lib32-gtk+3 installed, the test case should be skipped when testing against the lib32 environment setup script. (From OE-Core rev: 163764ad4760a5fabf65640df5f968be98ad13d2) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/oe/package_manager: turn postinst failure warnings into bitbake failuresAlexander Kanavin2018-09-041-2/+2
| | | | | | | | | | | | | Sumo release provides a transition period so that deferrals to first boot via 'exit 1' can be converted to pkg_postinst_ontarget(). For the next release however, postinst script failures should be treated as such. [YOCTO #12607] (From OE-Core rev: 42acb0ebde4e88bcdf34a541b700f19d8607abb0) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/package: Add package separated debug symbols hardlink testHongxu Jia2018-08-281-2/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tweak recipe selftest-hardlink - addition of libexecdir to simulate multiple directories - add gdb.sh to run gdb from script which is invoked at test time. - rename `hello' -> `hello1' to workaround name confliction with the one in lmbench Add test_gdb_hardlink_debug to selftest/package - run a qemu and invoke gdb.sh to gdb binaries of selftest-hardlink - check gdb to read symbols from separated debug hardlink file - check debug symbols works correctly [Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory'] 2018-08-26 01:27:30,195 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests) 2018-08-26 01:30:29,005 - oe-selftest - INFO - gdbtest /usr/bin/hello1 2018-08-26 01:30:36,539 - oe-selftest - INFO - gdbtest /usr/bin/hello2 2018-08-26 01:30:43,568 - oe-selftest - INFO - gdbtest /usr/libexec/hello3 2018-08-26 01:30:50,157 - oe-selftest - ERROR - No debugging symbols found. GDB result: Reading symbols from /usr/libexec/hello3...(no debugging symbols found)...done.^M (gdb) Function "main" not defined.^M Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]^M (gdb) Starting program: /usr/libexec/hello3 ^M Hello World!^M [Inferior 1 (process 320) exited normally]^M (gdb) The program is not being run.^M (gdb) 2018-08-26 01:30:51,180 - oe-selftest - INFO - ... FAIL 2018-08-26 01:30:51,181 - oe-selftest - INFO - Traceback (most recent call last): File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 148, in test_gdb_hardlink_debug self.fail('GDB %s failed' % binary) AssertionError: GDB /usr/libexec/hello3 failed [Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory'] [Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory'] 2018-08-26 12:40:30,976 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests) 2018-08-26 12:42:15,149 - oe-selftest - INFO - gdbtest /usr/bin/hello1 2018-08-26 12:42:24,064 - oe-selftest - INFO - gdbtest /usr/bin/hello2 2018-08-26 12:42:31,078 - oe-selftest - INFO - gdbtest /usr/libexec/hello3 2018-08-26 12:42:38,646 - oe-selftest - INFO - gdbtest /usr/libexec/hello4 2018-08-26 12:42:46,824 - oe-selftest - INFO - ... ok [Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory'] (From OE-Core rev: 104d07e57488f4a414fb5e1f60d0c8b0c02d6b4d) Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/core/decorator: add skipIfInDataVarRichard Leitner2018-08-231-0/+14
| | | | | | | | | skipIfInDataVar will skip a test if a value is in a certain variable. (From OE-Core rev: 10b935c713748346aea6c36c2f41e0ae6c320821) Signed-off-by: Richard Leitner <richard.leitner@skidata.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-22 07:52:14 +0000