summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/utils
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/utils/gitarchive: Handle case where parent is only on originRichard Purdie2019-03-271-2/+2
| | | | | | | | | | | | | | | The parent code currently assumed that any parent branch is locally checked out which may not be the case. Use the local branch by default but fall back to the origin. This also means removing the later saftey check as the branch may not exist locally. This fixes the autobuilder resulttool test pushing code. (From OE-Core rev: 633e024f5d492ba38debf745b59813f6a2f4a3f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-build-perf-report/gitarchive: Move common useful functions to libraryRichard Purdie2019-03-261-0/+71
| | | | | | | | | | | These functions can be reused by the resulttool code so move to the common function library for this purpose. (From OE-Core rev: c66f848938c04e133259c5b6903dc592866ab385) (From OE-Core rev: 94a3ca85d30fc957f8f01a216a75342be49f9143) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-git-archive: Separate out functionality to library functionRichard Purdie2019-03-261-0/+173
| | | | | | | | | | | | | This turns the core of the script into a library function. Ultimately this will let us call that code with custom 'keywords' rather than relying on the data parsed from bitbake metadata which can't be used when archiving historical results. (From OE-Core rev: 4820ca2b0850e29b04a4fd5659a6e9837d6714d0) (From OE-Core rev: ab5d49ba8bb02410fb9f3024f98ff19a04acb3ca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/httpserver: Rework to avoid hangs and improve loggingRichard Purdie2019-02-251-8/+29
| | | | | | | | | | | | | | | | | | | | | | testimage.bbclass installs a SIGTERM handler which conflicts with the use of multiprocessing here. This is paritcularly problematic if the http service is terminated before its started and hence before its had a chance to reset the default signal handler (as the code was written). Instead, temporarily remove testimage's handler whilst forking the http process which means the correct handler is installed and won't deadlock. Also take the opportunity to add in some log messages about the server start and shutdown so that future debugging is easier and its clearer what the code is doing. (From OE-Core rev: cc0471439aa0085ca87deccf061c5b676ef12388) (From OE-Core rev: 4eac9a5337d93b6cbd3916af97f62bb04881c9cd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/buildproject: Only clean files if we've done somethingRichard Purdie2019-02-251-0/+5
| | | | | | | | | | | | | We should only be wiping out things on target if the tests have actually run. (From OE-Core rev: d38c3eac0a5a1a9b0eb98385832e92f48145655e) (From OE-Core rev: dab22dc58eabaeb421afa3c7de1cc08c5ec34c61) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: make it work for multiple usersRobert Yang2019-02-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. (From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980) (From OE-Core rev: 872c6e5101f4f27dcfc63d141d1b6568f46d2b5f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: set timeout to 60s for run_serialRobert Yang2019-02-252-2/+2
| | | | | | | | | | | | | | The 5s timeout for non-kvm is too short, especially when the load is high, which leads to unexpected errors, so set timeout to 60s by default. (From OE-Core rev: 8197d0f638a760fc03062c7a9009117d083d7ead) (From OE-Core rev: 6c930a8f5c11947c3b916efe23f33af462701bee) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: Print output when failed to loginRobert Yang2019-02-251-0/+1
| | | | | | | | | | | | | This is useful for debugging. (From OE-Core rev: da527d73f5eeae0f29b5f99aab757491d3f87ec7) (From OE-Core rev: 7fb953573635afb1b6482dbd9f51eb0f05731eae) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/metadata: Allow to function without the git moduleRichard Purdie2018-12-161-2/+15
| | | | | | | | | | | | The python git module may or may not be enabled, allow this code to function without it, falling back to the same method as metadata_scm.bbclass uses. This will be cleaned up in the next round of feature development. (From OE-Core rev: 6350586ba9f4a4107a2d457590824cd4d662d5b9) (From OE-Core rev: 32c9169b76e13e53b6a9ab4a59932cea7863d992) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner.py: Fix python regex warningsRichard Purdie2018-12-161-5/+5
| | | | | | | | | | | | | | | | | | | | | Fix the warnings: meta/lib/oeqa/utils/qemurunner.py:250: DeprecationWarning: invalid escape sequence \. ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) meta/lib/oeqa/utils/qemurunner.py:343: DeprecationWarning: invalid escape sequence \- if re.search("root@[a-zA-Z0-9\-]+:~#", output): poky/meta/lib/oeqa/utils/qemurunner.py:350: DeprecationWarning: invalid escape sequence \- if re.search("root@[a-zA-Z0-9\-]+:~#", output): meta/lib/oeqa/utils/qemurunner.py:448: DeprecationWarning: invalid escape sequence \- if re.search("[a-zA-Z0-9]+@[a-zA-Z0-9\-]+:~#", data): by correctly marking the regexs. (From OE-Core rev: 8e6987735002560fca714f77ea8ece9d4b28f7fa) (From OE-Core rev: 7eb1f0be82d7ee12f893cdd40384da306fa597a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: Avoid unclosed file warningsRichard Purdie2018-12-161-0/+3
| | | | | | | | | | | | | Avoid warnings such as: meta/lib/oeqa/utils/commands.py:213: ResourceWarning: unclosed file <_io.BufferedReader name=4> return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options) (From OE-Core rev: 6a68c42de08cffbadb59ebda63fa5e19f6e5acef) (From OE-Core rev: 682d7b2810b235e86a28a8afe034e3853dbe8c45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: don't litter /tmp with temporary directoriesRoss Burton2018-12-162-3/+5
| | | | | | | | | | | | | | | | If we need to create a temporary directory in targetbuild or buildproject use tempfile.TemporaryDirectory so that when the test case is finished, the directory is deleted. Also synchronise the logic and don't possibly store the temporary directory in self.tmpdir as nothing uses that. (From OE-Core rev: db0e658097130d146752785d0d45f46a3e0bad71) (From OE-Core rev: d39252324a13580cc96f0694b88bc10515e030a0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: Avoid tracebacks on closed filesRichard Purdie2018-12-161-2/+2
| | | | | | | | | | | | | | | | Reorder the shutdown/teardown to avoid: File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 224, in launch op = self.getOutput(output) File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 90, in getOutput fl = fcntl.fcntl(o, fcntl.F_GETFL) ValueError: I/O operation on closed file (From OE-Core rev: 8e7d756862d2a8d62f3c87497d6d65ddb3c1b962) (From OE-Core rev: 48979ffbe25351f92179021a973207a71bbe7a4e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Remove resource python warningsRichard Purdie2018-12-161-2/+8
| | | | | | | | | | | If runqemu fails it would leak an unclosed socket and file. Ensure we close these in all cases to remove the resource warning. (From OE-Core rev: ed80e46ccbc8fe8e9148d80723152066fa00ba28) (From OE-Core rev: baa8313e492cff8c31633a558792500b10493e9e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: Avoid log message duplicationRichard Purdie2018-12-161-1/+1
| | | | | | | | | | | | | | Each time a runqemu() fails, the log handler would be left behind meaning messages from any subsequent run would be duplicated (or worse/more). This ensures we remove the handler regardless and means we no longer have the duplication. (From OE-Core rev: 532984708436bdfa3a8cac2c684a425eb249bad0) (From OE-Core rev: 600e71ede1977c45fed1958dd5356911e4fcf272) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/qemurunner: Fix python ResourceWarning for unclosed fileRichard Purdie2018-12-161-0/+2
| | | | | | | | | | | | | | Fixes: Stderr: /media/build1/poky/meta/lib/oeqa/utils/qemurunner.py:381: ResourceWarning: unclosed file <_io.BufferedWriter name=16> self.runqemu = None (From OE-Core rev: b9e0bf919e6fc1a58e02145a363ebe7066e5bf4f) (From OE-Core rev: b8a9eec2c9e56181350269acd6b42a14cc4abb3e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/commands: Add extra qemu failure loggingRichard Purdie2018-12-161-1/+5
| | | | | | | | | | | | Rather than just referring the user to the logs containing the failure, print them on the console. This aids debugging with oe-selftest with parallelisation as the logs may otherwise be lost. (From OE-Core rev: 36a018e245a232f520ff946f152cc875927a6fb4) (From OE-Core rev: 0ea441ab6a6fe752cc8820fb371e67eee92353e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/oeqa: Ensure added layers set LAYERSERIES_COMPATRichard Purdie2018-04-071-1/+1
| | | | | | | | | | Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool/oeqa needs to set this to avoid warnings which break various tests. (From OE-Core rev: 27568410ebb0d40db3428550704f35199df0e034) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager: Filter to only rpms we depend uponRichard Purdie2018-03-011-1/+2
| | | | | | | | | | | | | | | Currently do_rootfs gets to see all rpms in the deploy directory. This filters that view to only rpms which the image recipe has actual depends upon which potentially removes some sources of confusion in the image construction. This makes builds more reproducibile and also fixes contamination issues where dnf picks up packages it shouldn't be able to 'see'. [YOCTO #12039] (From OE-Core rev: 85e72e129362db896b0d368077033e4a2e373cf9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: qemurunner.py: cleanup grammar and ensure consistencyMark Asselstine2017-12-101-1/+1
| | | | | | | | | | | Minor grammar correction along with making the term 'login banner' consistent throughout to make searching logs easier. (From OE-Core rev: 54e739485faba8d80f5af15407905aaeda4767f4) Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: qemurunner.py: Log both 'failed to reach login banner" reasonsMark Asselstine2017-12-101-2/+6
| | | | | | | | | | | | | | The current logging always assumes the boot timeout has expired yet there is a second reason we might have ended up in a position where no login banner was found, that being a socket disconnect. Add logging for the disconnect case and make the timeout expiration conditional on the timeout being exhausted. (From OE-Core rev: 49403368ccf3e469ac111afa259a38cc11e0b688) Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: qemurunner.py: Add wall clock to 'timeout' loggingMark Asselstine2017-12-101-6/+14
| | | | | | | | | | | | | | | | | | | When debugging issues when timeouts are involved it is always best to have wall clock times included. This helps give confidence that the timeout is in fact run down at the right rate and that no unexpected events were the true cause of a premature running down of the timeout. Having these times in old logs also helps when debugging issues as we have a historic record as to what is a 'typical' time to complete an action. In addition to adding the wall clock times the time to 'login' is now printed making it consistent with the time to 'qemu pid'. (From OE-Core rev: 4a6364309547d77d1d7a94c48f7c51ceee2b5d1a) Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/commands: don't break if get_bb_vars is passed a tupleRoss Burton2017-12-021-1/+1
| | | | | | | | | | | | | get_bb_vars was using variables.copy() to duplicate the list of variables passed but this function only exists in lists [1,2] and not tuples (1,2). Instead of throwing an exception if the variables are in a tuple, simply construct a new list using the passed sequence-like object. (From OE-Core rev: b5837f62c8af94d134cf2160afdfb9e08b3418d1) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Simplify binary data handlingRichard Purdie2017-11-211-12/+8
| | | | | | | | | | | | | I have concerns that bad timing of the flow of data from the logger might corrupt the output due to the way binary strings are handled in qemurunner. This simplifies the code to do the same thing it did before but much more safely. (From OE-Core rev: 1e87283e92a2765bb5d54d17138b208bc395953b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: Clean up logger handlingRichard Purdie2017-11-093-23/+12
| | | | | | | | | | | | | | | | | | The logger handling in oeqa was confused at best. This patch: a) Passes in a logger through various qemu runner pieces b) Uses that logger consistently in the code c) Creates a logger for QemuRunner outside the bitbake namespace meaning we don't conflict with the tinfoil logging changes The result of this is more consistency. For runtime tests in testimage, the logs always contain the debug info, nothing is shwon on the console. For the oe-selftests, logs are intercepted and only shown if the test fails. (From OE-Core rev: 4ff678137a55b93c9ba2cbffda34335ba859f704) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Use logger.debug, not logger.infoRichard Purdie2017-11-091-37/+37
| | | | | | | | | | | Bitbake logs info messages to the console. These messages are really there as debugging information. At the debug level, they will be shown in failure logs and in the task logs but not on the console which is what we want in this case. (From OE-Core rev: e05b14ad5a3c5ac1656b6fffdcf7e58250638e96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Ensure logging handler is removedRichard Purdie2017-11-091-1/+3
| | | | | | | | | If we don't remove the handler we end up with duplicate log messages which is undesireable. (From OE-Core rev: 4155a58aebe9c730be9ff1f68150f34d1cd472e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sshcontrol.py: in copy_to() always use scpErik Botö2017-11-081-6/+3
| | | | | | | | | | | | | | | The current implementation is broken when the localpath is a link. Then only a symlink would be created on the target, instead of copying the actual file. [YOCTO #11524] (From OE-Core rev: 1eb2a9c2f48d3af13ce651f1adf024b3380299d1) Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: fix bad indentation in serial loginRoss Burton2017-11-051-3/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: print tail qemu log in case bootlog is emptyLeonardo Sandoval2017-09-251-1/+6
| | | | | | | | | | | | | | There are cases where the 'while loop' waiting for login prompt fails and the bootlog variable does not get populated, thus use the the new qemurunner member (self.msg) which stores all output coming from the qemu process. [YOCTO #12113] (From OE-Core rev: 39ffa0f3779305c5e8ef86fe4572e961c5912021) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: wait for PID to appear in procfsJuro Bystricky2017-09-131-3/+4
| | | | | | | | | | | | We need QEMU PID in order to access "/proc/<qemupid>/cmdline" Having a valid QEMU PID does not mean we can access the proc entry immediately, we need to wait for the /proc/<qemupid> to appear before we can access it. (From OE-Core rev: d2d069fa9910d1c7a94c898355a63fca03ec5ad8) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: refactor searching for QEMU PIDJuro Bystricky2017-09-111-167/+136
| | | | | | | | | | | | | Ask QEMU to store the QEMU process PID in a file, this way we don't have to parse running processes and analyze descendents. This is done via QEMU command line argument "-pidfile". [YOCTO #12001] (From OE-Core rev: 67612dcd2a8a1aa1d683dddb0bd2f592886ff020) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Improve loggingRichard Purdie2017-09-111-45/+45
| | | | | | | | | | | | | | | Python unittest intercepts stdout and stderr however qemurunner sets up a streamhandler before that interception occurs, hence the messages spam the unittest output. By moving the logging init to the class init time, we use the unittest stdout/stderr and this means unittest can only show the log output upon failure. This cleans up the selftest and testimage output whilst still showing logging upon failure. (From OE-Core rev: 9099cecc727fe0ae5f1559582426d30ba7a9f4d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Tweak qemu pid starting timeout codeRichard Purdie2017-08-301-4/+8
| | | | | | | | | | | We're seeing timeouts in the autobuilder testing code. Increase these timeouts to 120, print the length of time we're having to wait, change the error messages to really be errors and don't print empty logs, its not helpful, print a message about the empty log instead. (From OE-Core rev: b87a33d9c86cc1d2ea196259020e1d3dc712ccef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils/logparser.py: fix section checkRobert Yang2017-08-231-1/+1
| | | | | | | | | | | | | | | | The section might be None or '', so use "if section" for it. Fixed: File "/buildarea/lyang1/poky/meta/lib/oeqa/utils/logparser.py", line 113, in log_as_files prefix += section TypeError: Can't convert 'NoneType' object to str implicitly [YOCTO #11547] (From OE-Core rev: 320ea01f9eb33df462534bf08488ff6ada9bbe7b) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/logparser.py: add skip statusRobert Yang2017-08-231-2/+3
| | | | | | | | | | | Some test cases maybe skipped, let's parse it. [YOCTO #11547] (From OE-Core rev: f23c0f6a5aa93d38a5ea9d450f0accff0a5cc869) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* commands.py: live output logging + result.error encoding fixPatrick Ohly2017-06-281-22/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests that use bitbake("my-test-image") can run for a long time without any indication to the user of oe-selftest about what's going on. The test author has to log the bitbake output explicitly, otherwise it is lost in case of test failures. Now it is possible to use bitbake("my-test-image", output_log=self.logger) to get more output both on the console and in the XML output (when xmlrunner is installed). Example output: 2017-06-23 12:23:14,144 - oe-selftest - INFO - Running tests... 2017-06-23 12:23:14,145 - oe-selftest - INFO - ---------------------------------------------------------------------- 2017-06-23 12:23:14,151 - oe-selftest - INFO - Running: bitbake my-test-image 2017-06-23 12:23:16,363 - oe-selftest - INFO - Loading cache...done. 2017-06-23 12:23:17,575 - oe-selftest - INFO - Loaded 3529 entries from dependency cache. 2017-06-23 12:23:18,811 - oe-selftest - INFO - Parsing recipes...done. 2017-06-23 12:23:19,659 - oe-selftest - INFO - Parsing of 2617 .bb files complete (2612 cached, 5 parsed). 3533 targets, 460 skipped, 0 masked, 0 errors. 2017-06-23 12:23:19,659 - oe-selftest - INFO - NOTE: Resolving any missing task queue dependencies Because the implementation was already using threading, the same is done to decouple reading and writing the different pipes instead of trying to multiplex IO in a single thread. Previously the helper thread waited for command completion, now that is done in the main thread. The most common case (no input data, joined stdout/stderr) still uses one extra thread and a single read(), so performance should be roughly the same as before. Probably unintentionally, result.error was left as byte string when migrating to Python3. OE-core doesn't seem to use runCmd() with split output at the moment, so changing result.error to be treated the same as result.output (i.e. decoded to a normal strings) seems like a relatively safe API change (or rather, implementation fix). (From OE-Core rev: 00b8c7ff17cd8f1920728fdc2653068e63d71724) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> merge: wait() Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Fix return value checks from subprocess.call()'sMikko Rapeli2017-06-282-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Python function subprocess.call() returns the return value of the executed process. If return values are not checked, errors may go unnoticed and bad things can happen. Change all callers of subprocess.call() which do not check for the return value to use subprocess.check_call() which raises CalledProcessError if the subprocess returns with non-zero value. https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module All users of the function were found with: $ git grep "subprocess\.call" | \ egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call' Tested similar patch on top of yocto jethro. Only compile tested core-image-minimal on poky master branch. (From OE-Core rev: 578c8205fd14c48c6d30ef2889d86f1b4aee060a) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-test: Move load_test_components to oeqa.utilsAníbal Limón2017-06-061-1/+36
| | | | | | | | | | | | In order to maintain compatibility with oe-selftest, the load_test_components needs to be re-used, so the script executor needs to pass to only load components supported by certain script (oe-test, oe-selftest). (From OE-Core rev: d6b78ae711b93b4059690320cb8d821aaadd1684) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.git: use --verify in rev_parse()Markus Lehtonen2017-05-231-1/+1
| | | | | | | | | | | | We use rev-parse for turning git object names into SHA-1 and checking their existence. Using --verify option makes sure git-rev-parse does what we expect. (From OE-Core rev: f5b420c7f07a008c11d492b055c7cc8869f644d3) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* QemuRunner: avoid tainting os.environPatrick Ohly2017-05-181-7/+8
| | | | | | | | | | | | | | | | | | | | | | | That a utility function permanently changes the process environment is bad style and leads to subtle, hard to debug problems. For example, we had one oe-selftest which used runqemu() with an override for DEPLOY_DIR_IMAGE. Another test then just called runCmd() and ended up passing the wrong DEPLOY_DIR_IMAGE set earlier in os.environ. The approach used here is to pass the desired environment dict to the launch() method as a new, optional parameter, which then gets passed on to subproject.Popen(). The modified env variables do not get logged, as before. [YOCTO #11443] (From OE-Core rev: cab20f3b2fe668a63c58b44f2ad797fed74226fe) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe.lsb: add get_os_release()Markus Lehtonen2017-05-161-13/+3
| | | | | | | | | | | | Move get_os_release() from oeqa.utils.metadata to oe.lsb, merging the code with release_dict_osr() from oe.lsb. This removes some code duplication and makes get_os_release() more robust. (From OE-Core rev: 56b883f7765f6bd72e83dec26a5db8c7108c835d) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/targetbuild: tmp dir improvementsJoshua Lock2017-04-061-3/+7
| | | | | | | | | | Don't hard-code /tmp as the tmpdir, instead use WORKDIR as the tmpdir if the instantiater doesn't specify a value. (From OE-Core rev: c43c966e0ed4ed836bdf90b1d4c3f2f45426f1ec) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/buildproject: create a more unique tmp dirJoshua Lock2017-04-061-2/+5
| | | | | | | | | | Rather than hardcoding /tmp as the default tmpdir make a more unique tmpdir with tempfile.mkdtemp() when the caller doesn't specify a tmpdir value. (From OE-Core rev: 9425c2658fea0b45468a04574cd77bffc6668a8d) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/commands: extend variable regex to include A_B variable notationLeonardo Sandoval2017-04-051-1/+1
| | | | | | | | | | This change allows quering for variables with the format A_B, i.e. PREFERRED_PROVIDER_virtual/kernel instead of just A. (From OE-Core rev: 3810738eff6bdcf27c7e291dbeaedc699ab14bfc) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: allow persistent image writes in runqemu()Patrick Ohly2017-03-283-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | By default, QemuRunner avoids modifying the image files that it boots into by enabling the qemu snapshot mode. However, some tests may want to test changes that must persists across reboots, so this mode should be optional. This can be combined by copying the image file to a temporary location first and then booting with that copy. It's also useful when testing with additional drives attached to a virtual machine. QemuTinyRunner doesn't use the snapshot parameter and therefore ignores the new parameter. Long term, a better way of passing these various configuration parameters should be used, and perhaps QemuRunner and QemuTinyRunner can be merged into one again to avoid code duplication. But for now the patch follows the exiting style. Also beware that QemuTarget.start() now acts in two different modes (with or without explicit launch command), and depending on that mode parameters like discard_writes must be ignored, i.e. not get passed to launch(). (From OE-Core rev: 969d079a33a57f5a8f7af86d7bab04d35ab07584) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: allow customizing image used by runqemu()Patrick Ohly2017-03-281-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | runqemu() takes all parameters for the virtual machine from the variables of the given recipe. By allowing the caller to provide a hash with variables that get applied locally, the caller gets more control. Here's the intended usage: <prepare internal-image in self.resultdir> overrides = { 'DEPLOY_DIR_IMAGE': self.resultdir, 'IMAGE_LINK_NAME': 'internal-image-%s' % self.image_arch, } with runqemu('refkit-installer-image', ssh=False, overrides=overrides) as qemu: .... This can be used to replace the image completely with something else or to copy it before allowing runqemu() to write into it. (From OE-Core rev: 4c6ba32abd6b9de33f4b8a0b87e8a56432ed7825) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: allow extending qemuparams="..."Patrick Ohly2017-03-281-2/+2
| | | | | | | | | | | | | | | Sometimes it is useful to reconfigure the qemu virtual machine directly. runqemu has the "qemuparams" parameter for that, and the underlying start() methods also supported modifying that via their "params" parameter. Only the runqemu() wrapper function lacked a way to specify additional parameters. One potential usage is to attach additional disks. (From OE-Core rev: b2a1cf09ea65ed6f561886b1b4241f9adf3e5709) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: tolerate interrupted select() while waiting for qemuPatrick Ohly2017-03-282-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes, the OEQA utility code aborts with: ... File ".../meta/lib/oeqa/utils/qemurunner.py", line 131, in start return self.launch(launch_cmd, qemuparams=qemuparams, get_ip=get_ip, extra_bootparams=extra_bootparams) File ".../meta/lib/oeqa/utils/qemurunner.py", line 259, in launch sread, swrite, serror = select.select(socklist, [], [], 5) InterruptedError: [Errno 4] Interrupted system call strace shows that this is because of a SIGWINCH: Connection from 127.0.0.1:52668 select(21, [20], [], [], {5, 0}) = ? ERESTARTNOHAND (To be restarted if no handler) --- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} --- This is related to some special conditions: * whether qemu opens a graphical console window (enabled in Poky by default) * where that window gets opened * whether the window manager changes the size of the shell window (mine is a tiling window manager and reorders and resizes windows automatically) Ignoring the interrupted system calls avoids the problem. Code elsewhere (for example, run() in ssh.py) already does the same thing. (From OE-Core rev: 44fe106baf5fd5aebe26c5f28004e2b18d839b7c) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa.utils.commands: limit runCmd exception outputMarkus Lehtonen2017-03-271-3/+10
| | | | | | | | | | | | Make it possible to limit the length of output lines shown in runCmd exceptions. E.g when running bitbake we easily get thousands of lines of log output, where only the last few (tens) are interesting or relevant when an error occurs. (From OE-Core rev: 403dd205828002d6ef4e8b474aedb6082289e22f) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-24 02:01:09 +0000