linux/poky.git/meta/lib/oeqa/core/target, branch hardknott

oeqa/runqemu: Support RUNQEMU_TMPFS_DIR as a location to copy snapshot images to

2021-04-06T08:30:10+00:00

We have a working theory that IO queues on the autobuilder are impacting runtime testing under qemu, particularly async writes which inice does not influence. We already pass the snapshot option to qemu which copies the image and runs out of the copy. Add in the ability to copy the image to a specificed location which can be a tmpfs. This means that writes to the image would no longer be blocked by other writes to disk in the system. Preliminary tests show that this does improve the qemu errors at the expense of sometimes showing qemu startup timeouts as on a loaded system with a large test image, it can take longer than 120s to copy the image to tmpfs. Having a most consistent failure mode for loaded tests is probably desireable though. (From OE-Core rev: fd1c26ab426c3699ffd8082b83d65a84c8eb8bff) Signed-off-by: Richard Purdie

target/ssh.py: Add dump_target support

2020-09-30T14:01:51+00:00

This adds the dump_target support when the ssh command fails with a 'No route to host'. This is will provide additional data when a Qemu target fails to respond during autobuilder testing. This does not fix 14002 [0], but may help track down why qemu looses networking [0] https://bugzilla.yoctoproject.org/show_bug.cgi?id=14002 (From OE-Core rev: cef1a2b03b359c018911abc29db1895d0f46814c) Signed-off-by: Saul Wold Signed-off-by: Richard Purdie

testimage: Add testimage_dump_target to kwargs

2020-09-30T14:01:51+00:00

This passes the list of commands to run on the OEQemuTarget when the TargetDumper needs to run in a test context due to a failure on the target. This is added here as a kwargs because the 'd' dictionary is not available in the staticmethod getTarget in the OERuntimeTestContextExecutor class. The OEQemuTarget is different from the QemuTarget which already uses the list of commands from testimage_dump_target from 'd'. The create_dir() is needed to initialize the TargetDumper's dump_dir variable. (From OE-Core rev: a63675fab4d9f638570912b15a07932f549cc4d1) Signed-off-by: Saul Wold Signed-off-by: Richard Purdie

ltp: make copyFrom scp command non-fatal

2020-08-06T14:12:39+00:00

[YOCTO #13802] Make the scp failure non-fatal so the ltp tests continue to run and the rest of the logs will be available to see afterwards. (From OE-Core rev: 0f7d093038274f4f21f6cca39a96aac4f6c32ee3) Signed-off-by: Mingde (Matthew) Zeng Signed-off-by: Richard Purdie

testimage: enable ovmf support

2020-04-24T13:10:08+00:00

Add support for running wic images with EFI as testimage. Introduces a variable called QEMU_USE_OVMF for configuration. (From OE-Core rev: 3af8aaff68ed332d812ea7dc184d392700ad7882) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

testimage: Extend runtime testing infrastructure to allow unconventional booting processes to be tested

2020-02-13T12:19:14+00:00

The current runtime infrastructure contains hardcoded values which Ill refer to as patterns, these patterns are either searched through or sent via the serial terminal to communicate between HOST and TARGET. These patterns are required since they allow us to check when a device has finished booting, to log in, and to check whether a command sent from our tests has returned, this way we are able to check both the status of the commands that were sent along with its output. The testing process goes somewhat as follows: 1. Launch QEMU and start booting. 2. Check when the device has booted by looking for the pattern login:. 3. Log in as the root user (default for our images). 4. Check that we were able to log in succesfully. 5. Start running the runtime test cases defined by TEST_SUITES. 6. One of such test cases could send a command to the QEMU target. 7. Check whether that command returned. 8. Check its output and status, return whether the test case passed or failed. This patch allows this set of patterns to be defined instead of being hardcoded, but it also automatically sets the defaults that we have been using in the past if they have not been manually defined, for this reason, the patch is less invasive and should not affect in any way how tests are currently being run. Cases that can be enabled with this patch: - A customized image that does not use the root user (or maybe we want to check what happens if we dont use the root user). - An image where the PS1 env variable has been modified, and the prompt pattern wouldnt match the default. - Baremetal applications, which do not follow the conventional way of booting Linux and would probably not show a prompt for a user to log in, same applies for testing bootloaders. - poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny from meta-intel, which boots directly to RAM, and does not show a log in prompt since it does not contain a conventional init process. The code itself contains comments that should be self explanatory but here is an example on how these patterns can be defined in a hypothetical case where we want to run test cases as the webserver user instead: TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded" TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n" TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#" The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when used to communicate with the target when booting, anyone familiar with the PACKAGECONFIG syntax should have no trouble setting these. Other patterns would still be set up as default, e.g. search_reached_prompt would still be login: The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following: search_reached_prompt, send_login_user, search_login_succeeded, search_cmd_finished. They are prefixed with either search/send, to differentiate if the pattern is meant to be sent or searched to/from the target terminal. A working example of this code that falls under the baremetal case mentioned above along with a test case is present on the meta-freertos layer, which tests an RTOS image built with OpenEmbedded and automatically runs a test case on it after booting such image: As usual, INHERIT += "testimage" needs to be present on local.conf $ bitbake freertos-demo -c testimage RESULTS: RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s) SUMMARY: freertos-demo () - Ran 1 test in 2.006s freertos-demo - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) (From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831) Signed-off-by: Alejandro Hernandez Samaniego Signed-off-by: Alejandro Hernandez Samaniego Signed-off-by: Richard Purdie

testimage: Allow testing on QEMU machines with a single serial port

2020-02-13T12:19:14+00:00

commmit 6cde67d0a84 enables the use of qemurunner on machines that only have a single serial port, but still sets the default value as serial_ports=2 if not provided. The testimage class does not call qemurunner with a serial_ports argument, hence always defaulting to two. Pass the serial_ports argument from the testimage class to allow tests to run on QEMU machines with a single serial port. (From OE-Core rev: a50d0163770f0b405a8de8a8a9cccd48c1de4112) Signed-off-by: Alejandro Hernandez Samaniego Signed-off-by: Richard Purdie

oeqa: modernise subprocess: preexec_fn=os.setsid -> start_new_session=True

2020-02-04T15:56:29+00:00

start_new_session was added to python3 subprocess in v3.2 and is meant to take the place of the common use of preexec_fn to call os.setsid() in the child - as done here. Update to use the new equivalent. (From OE-Core rev: c3bd502f31a621394a2f6ea3ae90443ff4ac98a4) Signed-off-by: André Draszik Signed-off-by: Richard Purdie

oeqa/target/ssh oeqa/target/qemu: expose server listening port to tests

2019-12-16T23:25:49+00:00

Allow tests to access the listening port as just introduced. Note that when using qemu this infrastructure shouldn't be needed, but we still need to set the port to 0 so that a listening port is determined automatically (e.g. by the python http server). (From OE-Core rev: ca58559fc30842c123907fb5cc68b3356c530862) Signed-off-by: André Draszik Signed-off-by: Richard Purdie

oeqa/target/ssh: Replace suggogatepass with ignoring errors

2019-05-13T19:50:05+00:00

We continued to see encoding problems with ssh commands run in oeqa. After much research the conclusion was we should use ignore the errors since some occasional bad locale encoding is better than the unicode decoding issues we were seeing which crashed large parts of tests. (From OE-Core rev: b3bae8aba5b897c1afcc8bf4ad8929251812d2b5) Signed-off-by: Richard Purdie