linux/poky.git/meta/classes-recipe/testimage.bbclass, branch 5.0_M2

testimage: Drop target_dumper and most of monitor_dumper

2023-12-20T07:47:00+00:00

The target_dumper code is basically broken. It has been reading binary files over the text base serial communication and runs at every command failure which makes no sense. Each run might overwrite files from the previous run and the output appears corrupted due to confusion from the binary data. For now, remove the commands and the target dumper code as the command and execution point are problematic. Also remove the same pieces of the monitor code but leave the command list since in theory this can be moved to a more useful place in the code. (From OE-Core rev: a24d787987dccc95fdd95b7e85bf525a1c55b285) Signed-off-by: Richard Purdie

testimage: Exclude wtmp from target-dumper commands

2023-12-20T07:47:00+00:00

wtmp is filled with binary data which the run_serial command can't cope with. Catting this results in confusion of the serial interface and potentially large backlogs of data in the buffers which can hang qemu. Exclude the problematic files from the command. (From OE-Core rev: 599ac08a6f6fb3f6a89a897c8e06367c63c2f979) Signed-off-by: Richard Purdie

testimage.bbclass: remove QEMU_USE_SLIRP variable

2023-09-14T20:57:16+00:00

QEMU_USE_SLIRP is replaced by TEST_RUNQEMUPARAMS with "slirp" and possibly other arguments to runqemu script. (From OE-Core rev: 99fd24f0d9ff79fed389ae5a01c3031d7e7167d0) Signed-off-by: Mikko Rapeli Signed-off-by: Richard Purdie

testimage: respect target/server IPs when using slirp

2023-09-14T08:19:38+00:00

We can't hardcode these IPs when using slirp. The target IP will need a port to be specified as this controls what port the SSH connection uses, and when slirp is used it can't bind to port 22. The qemu runner (OEQemuTarget) assumes that the first port forward is the SSH forward, but this may be wrong or a different target may be used. The server IP depends on how the virtual networking is configured. runqemu defaults to 10.0.2.x for the guests so that is a wise default, but that may be configured differently. (From OE-Core rev: 81b304e2558730de285f2773371340fc636a8ed1) Signed-off-by: Ross Burton Signed-off-by: Richard Purdie

testimage.bbclass: detect slirp from TEST_RUNQEMUPARAMS

2023-09-12T11:23:09+00:00

Also set target and server IP addresses correctly to 127.0.0.1 so that TEST_TARGET_IP and TEST_SERVER_IP don't need to be set manually with slirp. Users run qemu images with slirp networking like this: $ bitbake core-image-minimal $ ../poky/scripts/runqemu slirp slirp networking means that only one TCP port is forwarded from host system to the qemu machine so that ssh into the machine works. ping and other low level networking from host to target machine do not work, but networking from target via host does work. This is much more easy to setup than the bridge networking used by default, which I've never managed to get running on various host machines. To use slirp networking wita qemu and core-image-minimal to run testimage.bbclass tests, add these to local.conf: IMAGE_CLASSES += "testimage" IMAGE_FEATURES += "ssh-server-dropbear" TEST_RUNQEMUPARAMS += "slirp" Then image can be compiled and tested with: $ bitbake core-image-minimal $ bitbake -c testimage core-image-minimal Note that only few of the current oeqa runtime tests work against core-image-minimal, but test results look like: RESULTS: RESULTS - date.DateTest.test_date: PASSED (3.64s) RESULTS - df.DfTest.test_df: PASSED (0.55s) RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (0.56s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (1.88s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (0.93s) RESULTS - pam.PamBasicTest.test_pam: PASSED (2.20s) RESULTS - parselogs.ParseLogsTest.test_parselogs: PASSED (4.98s) RESULTS - ping.PingTest.test_ping: PASSED (0.05s) RESULTS - ssh.SSHTest.test_ssh: PASSED (1.28s) RESULTS - systemd.SystemdBasicTests.test_systemd_basic: PASSED (0.56s) RESULTS - systemd.SystemdBasicTests.test_systemd_failed: PASSED (1.10s) RESULTS - systemd.SystemdBasicTests.test_systemd_list: PASSED (0.92s) RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: PASSED (0.56s) RESULTS - systemd.SystemdJournalTests.test_systemd_journal: PASSED (0.54s) RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.00s) RESULTS - buildcpio.BuildCpioTest.test_cpio: SKIPPED (0.00s) RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s) RESULTS - buildlzip.BuildLzipTest.test_lzip: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand_help: SKIPPED (0.00s) RESULTS - connman.ConnmanTest.test_connmand_running: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s) RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s) RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gcc_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp2_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_gpp_compile: SKIPPED (0.00s) RESULTS - gcc.GccCompileTest.test_make: SKIPPED (0.00s) RESULTS - gi.GObjectIntrospectionTest.test_python: SKIPPED (0.00s) RESULTS - go.GoHelloworldTest.test_gohelloworld: SKIPPED (0.00s) RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: SKIPPED (0.00s) RESULTS - ldd.LddTest.test_ldd: SKIPPED (0.00s) RESULTS - logrotate.LogrotateTest.test_logrotate_newlog: SKIPPED (0.00s) RESULTS - logrotate.LogrotateTest.test_logrotate_wtmp: SKIPPED (0.00s) RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s) RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: SKIPPED (0.00s) RESULTS - perl.PerlTest.test_perl_works: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: SKIPPED (0.00s) RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s) RESULTS - python.PythonTest.test_python3: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s) RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s) RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s) RESULTS - rust.RustCompileTest.test_cargo_compile: SKIPPED (0.00s) RESULTS - rust.RustCompileTest.test_rust_compile: SKIPPED (0.00s) RESULTS - scp.ScpTest.test_scp_file: SKIPPED (0.00s) RESULTS - stap.StapTest.test_stap: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_status: SKIPPED (0.00s) RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: SKIPPED (0.00s) RESULTS - weston.WestonTest.test_wayland_info: SKIPPED (0.00s) RESULTS - weston.WestonTest.test_weston_can_initialize_new_wayland_compositor: SKIPPED (0.00s) RESULTS - weston.WestonTest.test_weston_running: SKIPPED (0.00s) RESULTS - weston.WestonTest.test_weston_supports_xwayland: SKIPPED (0.00s) RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s) SUMMARY: core-image-minimal () - Ran 70 tests in 22.173s core-image-minimal - OK - All required tests passed (successes=14, skipped=56, failures=0, errors=0) NOTE: recipe core-image-minimal-1.0-r0: task do_testimage: Succeeded NOTE: Tasks Summary: Attempted 1305 tasks of which 1304 didn't need to be rerun and all succeeded. (From OE-Core rev: f4e865062cec06586d8c38c05d86a5b0d727625c) Signed-off-by: Mikko Rapeli Signed-off-by: Richard Purdie

testimage/oeqa: Drop testimage_dump_host functionality

2023-06-29T09:57:27+00:00

The intent behind these functions was to dump the system state when issues occured but it has never really worked as we'd planned. Regular monitoring as the build runs has largely replaced this as that allows a trend to be seen rather than a spot value which was never really useful. The code is bitrotting and not functioning correctly so drop it. [YOCTO #13872] RP: Reword commit message (From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508) Signed-off-by: Thomas Roos Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie

testimage: Only note missing target directories, don't warn

2023-06-17T10:50:57+00:00

We don't need to see warnings for missing target debug directories. Just show a note in the logs instead. (From OE-Core rev: 52db25c58069c4f440da33daf0474255c9fa870b) Signed-off-by: Richard Purdie

testimage: implement test artifacts retriever for failing tests

2023-06-17T10:50:57+00:00

Add a basic artifacts retrievers in testimage class which: - triggers when at least one runtime test fails but tests execution encountered no major issue - reads a list of paths to retrieve from TESTIMAGE_FAILED_QA_ARTIFACTS - checks for artifacts presence on target - retrieve those files over scp thanks to existing ssh class - store those files in an "artifacts" directory in "tmp/log/oeqa/" This implementation assumes that the SSH or Qemu target has run and finished gracefully. If tests do not finish because of an exception, artifacts will not be retrieved Bring partial solution to [YOCTO #14901] (From OE-Core rev: 36ef582b8c1c99e6af1ce79ea79f5b059d2a1aad) Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie

runqemu: respect IMAGE_LINK_NAME

2023-04-04T12:11:49+00:00

* when searching for qemuboot.conf * don't assume that IMAGE_LINK_NAME is always - (with -.qemuboot.conf) * runqemu: use IMAGE_LINK_NAME set by testimage.bbclass or query with bitbake -e * testimage.bbclass was setting DEPLOY_DIR which I don't see used anywhere else, so I assume it was supposed to be DEPLOY_DIR_IMAGE as mentioned in corresponding runqemu code, do the same with IMAGE_LINK_NAME variable * add virtual/kernel as bitbake -e target in run_bitbake_env to make sure IMAGE_LINK_NAME is defined (kernel-artifact-names.bbclass inherits image-artifact-names.bbclass as well) * improve .qemuboot.conf search 1st search for file matching the rootfs and only when not found try again with .rootfs suffix removed [YOCTO #12937] (From OE-Core rev: 716eb55bb963db7b02d985849cb025898aabc855) Signed-off-by: Martin Jansa Signed-off-by: Richard Purdie

testimage: Fix error message to reflect new syntax

2023-02-02T10:27:30+00:00

(From OE-Core rev: ec73d19d78e8f30ff9b817490c23bcdf8ea47c86) Signed-off-by: Alejandro Enedino Hernandez Samaniego Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie