linux/poky.git/meta/classes-recipe/testimage.bbclass, branch yocto-5.1

testexport: support for executing tests over serial

2024-09-01T11:28:10+00:00

Uses TEST_SERIALCONTROL_CMD to open a serial connection to the target and execute commands. This is a drop in replacement for the ssh target, fully supporting the same API. Supported with testexport. To use, set the following in local.conf: - TEST_TARGET to "serial" - TEST_SERIALCONTROL_CMD to a shell command or script which connects to the serial console of the target and forwards that connection to standard input/output. - TEST_SERIALCONTROL_EXTRA_ARGS (optional) any parameters that must be passed to the serial control command. - TEST_SERIALCONTROL_PS1 (optional) A regex string representing an empty prompt on the target terminal. Example: "root@target:.*# ". This is used to find an empty shell after each command is run. This field is optional and will default to "root@{MACHINE}:.*# " if no other value is given. - TEST_SERIALCONTROL_CONNECT_TIMEOUT (optional) Specifies the timeout in seconds for the initial connection to the target. Defaults to 10 if no other value is given. The serial target does have some additional limitations over the ssh target. 1. Only supports one "run" command at a time. If two threads attempt to call "run", one will block until it finishes. This is a limitation of the serial link, since two connections cannot be opened at once. 2. For file transfer, the target needs a shell and the base32 program. The file transfer implementation was chosen to be as generic as possible, so it could support as many targets as possible. 3. Transferring files is significantly slower. On a 115200 baud serial connection, the fastest observed speed was 30kbps. This is due to overhead in the implementation due to decisions documented in #2 above. (From OE-Core rev: d817b27d73d29ba2beffa2e0a4e31a14dbe0f1bf) Signed-off-by: Andrew Oppelt Signed-off-by: Matthew Weber Signed-off-by: Chuck Wolber -- Tested with core-image-sato on real hardware. TEST_SERIALCONTROL_CMD was set to a bash script which connected with telnet to the target. Additionally tested with QEMU by setting TEST_SERIALCONTROL_CMD to "ssh -o StrictHostKeyChecking=no root@192.168.7.2". This imitates a serial connection to the QEMU instance. Steps: 1) Set the following in local.conf: - IMAGE_CLASSES += "testexport" - TEST_TARGET = "serial" - TEST_SERIALCONTROL_CMD="ssh -o StrictHostKeyChecking=no root@192.168.7.2" 2) Build an image - bitbake core-image-sato 3) Run the test export - bitbake -c testexport core-image-sato 4) Run the image in qemu - runqemu nographic core-image-sato 5) Navigate to the test export directory 6) Run the exported tests with target-type set to serial - ./oe-test runtime --test-data-file ./data/testdata.json --packages-manifest ./data/manifest --debug --target-type serial Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests

2024-07-26T11:28:42+00:00

Currently, "tarball" sdk based recipes don't generate SPDX manifests as they don't include the rootfs generation classes. Split the SPDX 3.0 image class into two so the SDK components can be included where needed. To do this, introduce an SDK_CLASSES variable similar to IMAGE_CLASSES which the SDK code can use. Migrate testsdk usage to this. Also move the image/sdk spdx classes to classes-recipe rather than the general classes directory since they'd never be included on a global level. For buildtools-tarball, it has its own testsdk functions so disable the class there as a deferred inherit would overwrite it. (From OE-Core rev: 662396533177b72cc1d83e95841b27f7e42dcb20) Signed-off-by: Richard Purdie

testimage: Fix TESTIMAGE_FAILED_QA_ARTIFACTS setting

2024-07-18T16:21:39+00:00

MCNAME isn't defined outside our mcextend ptest images so use a wildcard in TESTIMAGE_FAILED_QA_ARTIFACTS instead. This unbreaks the value in other images. (From OE-Core rev: e7af85a7b7b966685a9eeaba11628dc10c1ea44b) Signed-off-by: Richard Purdie

testimage: fallback for empty IMAGE_LINK_NAME

2024-07-02T21:29:13+00:00

if IMAGE_LINK_NAME is set empty to disable the symlinking for image artifacts in deploy, testimage fails, as the path assembly is incorrect. In that case fallback to IMAGE_NAME (From OE-Core rev: c7a4e7e294992acc589c62adcaf6cd32659f2f9b) Signed-off-by: Konrad Weihmann Signed-off-by: Richard Purdie

oeqa/runtime/login: Various code improvements and fixes

2024-03-07T17:27:51+00:00

* Allow tools to be found from the host PATH so that imagemagick from a buildtools tarball/sdk can work * Reformat the code to have imports at the start of the file and have more standard formatting and whitespace * Always save copies of the images, the space imapct is negligle compared to the debug win * Write the images to ${T} * Use bb.utils.mkdirhier() instead of more complex code * Restrict the tests to images containing matchbox-desktop (From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a) Signed-off-by: Richard Purdie

testimage: create a list of failed test post actions

2024-02-27T11:35:43+00:00

testimage is able to detect whenever a test run leads to some tests failing, and execute some actions in this case. The only action currently defined in such case is to retrieve artifacts from the target under test, as listed in TESTIMAGE_FAILED_QA_ARTIFACTS In order to be able to add multiple actions, define a central function to gather all "post actions" to run whenever a test has failed (run_failed_tests_post_actions). This function contains a table listing all functions to be called whenever a test fails. Any function in this table will be provided with bitbake internal data dictionary ("d") and the current runtime testing context ("tc"). Isolate all this feature in a dedicated postactions.py file inherited by testimage. This patch does not bring any functional change. (From OE-Core rev: c01aa8df0613a103859b4431d3cc5056b2fef1b8) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

lib/oeqa: share get_json_result_dir helper

2024-02-27T11:35:43+00:00

Multiple places in oeqa need to get the log output path, and redefine a small helper to accomplish this Define this helper in lib/oeqa/utils/__init__.py and import it wherever needed to allow using it. (From OE-Core rev: 01b1a6a5a4e7cede4d23a981b5144ae9c8306274) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

testimage: log exception when failing to retrieve artifacts

2024-02-21T22:20:10+00:00

Despite managing to retrieve the failed ptests artifacts, testimage seems to dump some retrieval errors like the following one: WARNING: core-image-ptest-valgrind-1.0-r0 do_testimage: Can not retrieve /usr/lib/valgrind/ptest from test target Log the corresponding exception to help analyzing such issue (From OE-Core rev: 12873e5b1620414a76e4a0e87cc2c806a0513cfe) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

testimage: retrieve ptests directory when ptests fail

2024-02-08T10:59:06+00:00

TESTIMAGE_FAILED_QA_ARTIFACTS is set with a default, minimal list of files to retrieve whenever a runtime test fails. Add ptests directory to the list so we can get ptests artifacts (eg: logs) whenever a ptest fails. By appending the ptest directory with the multiconfig component in the path, only failing ptests will lead to corresponding ptest artifacts retrieval, instead of all ptests artifacts retrieval. While doing this addition, reinforce default value using "=" operator to make sure to get the default list in any case. (From OE-Core rev: 9357ab6c47f0a0a7000cb18358bc9775fd54e1f7) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

testimage: Drop target_dumper and most of monitor_dumper

2023-12-20T07:47:00+00:00

The target_dumper code is basically broken. It has been reading binary files over the text base serial communication and runs at every command failure which makes no sense. Each run might overwrite files from the previous run and the output appears corrupted due to confusion from the binary data. For now, remove the commands and the target dumper code as the command and execution point are problematic. Also remove the same pieces of the monitor code but leave the command list since in theory this can be moved to a more useful place in the code. (From OE-Core rev: a24d787987dccc95fdd95b7e85bf525a1c55b285) Signed-off-by: Richard Purdie