summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
* oeqa/selftest/debuginfod: improve selftestRoss Burton2023-11-241-36/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | This test was occasionally failing for no obvious reason, so refactor and improve: - While waiting for the daemon, check that it is still running and explicitly timeout after 10s when making the HTTP call. - While waiting for the daemon to be ready, log the current state of the daemon so we can tell if we're timing out as it is still scanning. - This was in fact the cause of the intermittant failures, because the TMPDIR is reused between tests and may contain a large number of packages. Do the tests in an isolated TMPDIR to hopefully mitigate this issue and increase the timeout to two minutes. - Decorate the test using runqemu as such so that can be skipped in environments without runqemu - Add a second test that doesn't use runqemu or images, which is faster but less realistic. (From OE-Core rev: 99590fac1bfb5474f5bf0e02d3888b518af9fb3e) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 88b660aaae2527736b6eccec4c952eee969e20a2) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: rework license checksum testsTrevor Gamblin2023-11-241-32/+8
| | | | | | | | | | | | | | | Remove the pretest_lic_files_chksum_modified_not_mentioned test entirely and use pyparsing in test_lic_files_chksum_modified_not_mentioned to scan the patches for lines starting with either "+LIC_FILES_CHKSUM" or "-LIC_FILES_CHKSUM". If either is found but no "License-Update" tag is present in the commit, fail the test. (From OE-Core rev: d1871afcec769b6503852adf6217460897ecf301) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 8e1bda0eb225ada22fdf5990edfec512be1d6629) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: remove test for CVE tag in mboxTrevor Gamblin2023-11-243-170/+0
| | | | | | | | | | | | | | | After patchtest went live it was determined that testing for a CVE tag in the mbox commit message is unnecessary, since it will already be in the shortlog and in any carried patches. Remove the test and the associated selftest files so that its absence isn't flagged in future test results. (From OE-Core rev: bf9671896eb60880b5dad36c2706855932ce091f) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 54690f18f04a2ab993a85d551ce4f8d0fa56618a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: make pylint tests compatible with 3.xTrevor Gamblin2023-11-241-5/+12
| | | | | | | | | | | | | pylint 3.x has removed epylint, which is now a separate module. To avoid adding another recipe or using outdated modules, modify the test_python_pylint tests so that they use the standard pylint API. (From OE-Core rev: 8b3c6837fe2367fa7aa20b2ee5be554be98f2acd) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 72be3d6a116febf46130cccbe12afe5ad93779b5) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: shorten test result outputsTrevor Gamblin2023-11-242-9/+9
| | | | | | | | | | | | | | Some test result lines in TestMbox and TestPatch are still too long to avoid being flagged by the mailer script. Clean them up by removing redundant information, so that they are all under the length limit of 220 characters. (From OE-Core rev: c543469e2da32a474a60a497b5d51fd9fc43dbb4) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c10d0bb542b23fbdc14d76dfa8e5885aa4d33083) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: reduce checksum test output lengthTrevor Gamblin2023-11-241-1/+1
| | | | | | | | | | | | | | | The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is outputting very long lines that fail the maximum length check when sending email results, preventing the actual errors from being displayed. Reduce the length of the failure message by rewording and removing redundant information. (From OE-Core rev: e3c680bab99f7e5f0cb7874ada0297b5fac66702) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2e2625735181160e9760a6f3af4955bda2ea6d4d) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: simplify test directory structureTrevor Gamblin2023-11-2453-603/+436
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Consolidate the various mbox tests into a new TestMbox class, metadata tests into TestMetadata, and patch tests into TestPatch. Also update the selftest filenames to match the changes. The test contents are not significantly changed (other than to reference the new class names). While this doesn't improve overall readability, it does result in more obvious categorization, and more importantly reduces the number of calls to setup tinfoil in the tests, resulting in a roughly 25% reduction in runtime. Before: [tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass) XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail) XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail) XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass) XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail) XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail) XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass) XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail) XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass) XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass) XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail) XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail) XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail) XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip) XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass) XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail) XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass) XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail) XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail) XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail) XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass) XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip) XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass) XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail) XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass) XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass) XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass) XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass) XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass) XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail) XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail) XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail) XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass) XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail) ============================================================================ Testsuite summary for patchtest ============================================================================ ============================================================================ real 24m14.386s user 1m13.599s sys 0m21.477s After: [tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail) XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass) XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail) XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail) XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip) XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass) XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail) XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass) XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail) XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail) XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail) XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass) XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail) XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail) XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail) XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass) XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip) XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass) XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass) XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass) XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass) XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass) XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass) XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail) XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail) XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail) XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass) XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass) XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass) XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail) XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass) XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail) XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail) XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail) ============================================================================ Testsuite summary for patchtest ============================================================================ ============================================================================ real 18m39.749s user 0m41.857s sys 0m14.708s (From OE-Core rev: 497e128546b37d6cf5fb86188fff4c7a22526ec8) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f788592da2fd0e21638ce2c3326675a060ba51cf) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest/selftest: add XSKIP, update test filesTrevor Gamblin2023-11-243-3/+8
| | | | | | | | | | | | | | Since we are skipping the merge test, two of the selftests now report SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have the right endings for XSKIP, and add the category so that it can be used for more extensive testing in the future. (From OE-Core rev: a354265065516ec634042ea8210f97aaba7ff43c) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3331f53c0be2575784a042bb2401eeba4f2a5a3e) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: disable merge testTrevor Gamblin2023-11-241-0/+1
| | | | | | | | | | | | | Disable the merge-on-head test until patchtest properly handles merging of series subsets and accounts for patches that are rapidly merged (i.e. before patchtest is run). (From OE-Core rev: 97c1d8aa7318e36a037aa6c8b721fb75608a92a7) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e561c614dc72b7f8bf5e09a09bbe6ebc3cf500bb) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: fix lic_files_chksum test regexTrevor Gamblin2023-11-241-3/+2
| | | | | | | | | | | | | the test_lic_files_chksum_modified_not_mentioned test in patchtest wasn't picking up on 'License-Update:' tags correctly. Use pyparsing's AtLineStart class to simplify the regex setup and search. (From OE-Core rev: 978c819c33c0552750f5383f0ed761d505b4bdf5) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dc9126e45e74b915faaf296037e7ece41785bf4a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: skip merge test if not targeting masterTrevor Gamblin2023-11-241-0/+2
| | | | | | | | | | | | | Avoid testing mergeability of a patch when not targeting master, so that patches tested via other means (e.g. maintainer branches and AB runs) don't get unnecessarily reviewed an extra time. (From OE-Core rev: 127fa20bd1d628548f7f4ba087b3ca105e705098) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e6cf23e353f48c57249681bd0b12bd8494d4959a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: test regardless of mergeabilityTrevor Gamblin2023-11-244-17/+0
| | | | | | | | | (From OE-Core rev: 06d2066a5061d23a316f65cfc731ce44b576b2bf) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dc089073eb120de76c8907e476c341ed3e97c164) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* patchtest: remove unused importsRoss Burton2023-11-248-8/+0
| | | | | | | | | (From OE-Core rev: 673d1b51b5183a81d1daaa7a48250aa938679883) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit be8429d986335aae65c2426862b97836ba46e42a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/selftest/devtool: abort if a local workspace already existJulien Stephan2023-11-241-0/+3
| | | | | | | | | | | | | | | | | | | | | if user run devtool selftests with a local workspacelayer the tests fail with various error such as: - devtool.DevtoolAddTests.test_devtool_add just hangs - devtool.DevtoolModifyTests.* fail with the following error: ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it. Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it. Check if a workspacelayer exists, warn the user and abort the tests (From OE-Core rev: b8756f6e20d15f1cc724784d8ceb45a969ec7f81) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit a74962cfb0485f6f2b9e2b751c33c8eafca8705a) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* oeqa/ssh: Handle SSHCall timeout error codeluca fancellu2023-11-242-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The current code in ssh.py is terminating the ssh process that does not finish its computation in a given timeout (when timeout is passed), the SSHCall function is returning the process error code. The Openssl ssh before version 8.6_p1 is returning 0 when it is terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda onwards (version >= 8.6_p1) ssh is returning 255 instead. So for version of ssh older than 8.6_p1 when the SSHCall time out, the return code will be 0, meaning success, which is wrong. Fix this issue checking if the process has timeout (hence it's been terminated) and checking if the returned code is 0, in that case set it to 255 to advertise that an error occurred. Add a test case excercising the timeout in the SSHTest, test_ssh test function. (From OE-Core rev: 82215c855ee39b4e39f24113241a7fb3f20f9531) Signed-off-by: Luca Fancellu <luca.fancellu@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 948fecca1db4c7a30fcca5fcf5eef95cd12efb00) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* goarch: Move Go architecture mapping to a libraryJoshua Watt2023-11-242-1/+35
| | | | | | | | | | | | | | Other spaces uses the Go architecture definitions as their own (for example, container arches are defined to be Go arches). To make it easier for other places to use this mapping, move the code that does the translation of OpenEmbedded arches to Go arches to a library. (From OE-Core rev: 5e0267aeb7d9f575f270f6856a67ac62ce8a0f71) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3e86f72fc2e1cc2e5ea4b4499722d736941167ce) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* cve-check: don't warn if a patch is remoteRoss Burton2023-11-031-5/+6
| | | | | | | | | | | | | | | | We don't make do_cve_check depend on do_unpack because that would be a waste of time 99% of the time. The compromise here is that we can't scan remote patches for issues, but this isn't a problem so downgrade the warning to a note. Also move the check for CVEs in the filename before the local file check so that even with remote patches, we still check for CVE references in the name. (From OE-Core rev: 201f0e1d55ca2fa6ab948a82d94e52c6a77ca7d2) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cve-check: slightly more verbose warning when adding the same package twiceRoss Burton2023-11-031-1/+1
| | | | | | | | | | | Occasionally the cve-check tool will warn that it is adding the same package twice. Knowing what this package is might be the first step towards understanding where this message comes from. (From OE-Core rev: 699863be46fab91d5729fce1dc5b795761247f98) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest/selftest: fix command argumentsTrevor Gamblin2023-10-171-1/+1
| | | | | | | | | | | | Changes to patchtest's command-line arguments to work with oe-core by default do not match the selftest script's argument list. Explicitly use the --testdir and --repodir flags in selftest so that it is compatible them. (From OE-Core rev: 6cd547b24896596d4e0fe57f26f553842c5560b5) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: update SPDX identifiersTrevor Gamblin2023-10-1729-97/+29
| | | | | | | | | | Replace full license headers with SPDX identifiers and adjust all patchtest-related code to use GPL-2.0-only. (From OE-Core rev: 9bea6b39074296bb8d8719a3300636e316f19d1b) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: set default repo and testdir targetsTrevor Gamblin2023-10-171-2/+7
| | | | | | | | | | | | | | | | | Since patchtest is in oe-core, the Python os module's methods can be used to retrieve the repository path and tests directory by default. This reduces the number of mandatory arguments for invocation of patchtest unless the user wants to use a custom test suite or test patches against a different repo. The REPO and TESTDIR arguments are likewise adjusted so that they are optional. Also, make it more obvious what the --startdir flag is meant for on the command line by renaming it to --testdir, and update the scripts/patchtest.README file to be consistent with the new usage. (From OE-Core rev: bae7421ece4806f5148f164293810b9fe75e0756) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: add supporting modulesTrevor Gamblin2023-10-174-0/+532
| | | | | | | | | | | | Add modules that support core patchtest functionality to meta/lib/patchtest. These include classes and functions for handling repository and patch objects, parsing the patchtest CLI arguments, and other utilities. (From OE-Core rev: 499cdad7a16f6cc256837069c7add294132127a4) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest/requirements.txt: updateTrevor Gamblin2023-10-171-0/+2
| | | | | | | | | Add git-pw and boto3 to the list of requirements for patchtest. (From OE-Core rev: f88e295cb5034950e9a0899c1dc3ca685a30a176) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Drop newlines serial workaroundRichard Purdie2023-10-151-12/+0
| | | | | | | | | Drop the newlines serial workaround, it doesn't seem to fix things enough of the time to be useful. (From OE-Core rev: 6cd57b2d148c5de7839d52d8cec359e4a10e0cd6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* reproducible: Exclude rust for now againRichard Purdie2023-10-131-0/+2
| | | | | | | | | | | | Unfortunately there is still an issue with rustdoc not being reproducible so part of the problem has been fixed but not everything. Add the exclusion back until this has been addressed to avoid autobuilder failures and long diffoscope analysis time. (From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: clean up test suiteTrevor Gamblin2023-10-1312-44/+45
| | | | | | | | | | | | | | | | Various tweaks to make the test suite cleaner and more efficient: - Replace use of "re" module with "pyparsing" in tests (but not base.py) - Make test_mbox_cve only check for CVE tags in the commit if the added patch has them - Make test_mbox_cve SKIP instead of PASS if there's no CVE tag - Simplify the bugzilla tag checking test now that pyparsing is used - Modify the selftest script to correctly parse the new result output (From OE-Core rev: 7a187c2475aa762e2bc830950f608143f2535a72) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patchtest: improve test issue messagesTrevor Gamblin2023-10-1318-52/+29
| | | | | | | | | | | | | | The patchtest tests provide vague feedback to the user, and many of them also provide redundant 'fix' strings that could easily be incorporated into the issue messages themselves. Simplify them so that it is more clear what the errors are and how they can be addressed. No recommendation is given when the issue string adequately conveys the issue, e.g. with a missing "LICENSE" entry in a newly-created recipe. (From OE-Core rev: 0bfb3614244ec7aa79b6424bc63f9f2bccdabe98) Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/_qemutiny: rewrite test to be functionalRoss Burton2023-10-111-3/+8
| | | | | | | | | | | | The _qemutiny is a small test case that was explicitly designed to do a minimal level of testing for poky-tiny images. These typically don't have SSH servers so we need to assume that qemu is being used and access the serial console directly. (From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: don't skip test_read_only_image on qemuarm64Ross Burton2023-10-111-2/+0
| | | | | | | | | | | | There's no need to skip this test anymore, as skipping on qemuarm64 was a proxy for "uses SERIAL_CONSOLES_CHECK" which no longer exists. [ YOCTO #14962 ] (From OE-Core rev: 1b8b1cb74c8f5f387a14d86f03f37f68c07dd9aa) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: ensure tag matches regex before getting its fieldsAlexis Lothoré2023-10-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Whenever we ask gitarchive to retrieve test results for specific revisions, we first do a "large" search in get_tags, which uses glob patterns with git ls-remote, and then we filter received tags with a regex to parse the tags fields. Currently gitarchive assumes that all tags returned by get_tags will match the regex. This assumption is wrong (for example searching "master-next" in get_tags may return some tags like "abelloni/master-next), and leads then to exception when we try to retrieve tags fields: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module> sys.exit(main()) ^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main ret = args.func(args, logger) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs fields, runs = get_test_runs(log, repo, tag_name, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs groups = m.groupdict() Fix this exception by merely skipping those additionals tags which won't match the regex (From OE-Core rev: 8b5ace47372e958db9e4abb23378947fb02f6fc2) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: fix tag pattern searchingAlexis Lothoré2023-10-111-1/+1
| | | | | | | | | | | | | | | | Whenever we ask gitarchive to search for tags, we can provide it with a pattern (containing glob patterns). However, when searching for example for tags matching branch master-next, it can find more tags which does not correspond exactly to branch master-next (e.g. abelloni/master-next tags will match). Prevent those additional tags from being fetched by gitarchive by using a more specific pattern: prefix user-provided pattern with "refs/tags" (From OE-Core rev: c24b7ea28021da48aa8f3498a9b899f595efde56) Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Ensure we retry after BrokenPipeErrorRichard Purdie2023-10-111-1/+1
| | | | | | | | | | | | If the BrokenPipeError occurs when writing to the serial port to wake it up, defer the write and try again (which will happen on the 5s timeout of the select call). Why it should return ESHUTDOWN and then work later I'm not sure but it does appear to make it work. For now we need 'working' QA tests whilst the issue is debugged. (From OE-Core rev: ea9e6ba0ab31a0b20012c283aa768496a50b527a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* rust: reproducibility issue fixSundeep KOKKONDA2023-10-111-2/+0
| | | | | | | | | | | | | | | | | | | | The '--remap-path-prefix' option removes all references to build directory structure in the debug information within the compiled output for Cargo dependencies and the project's binary. However, some references to build directories remains in the final binary in .rustc section in the form of compressed metadata and this makes the build output dependent on the folder structure of the computer it's compiled on. So, for reproducible builds, use the configuration option 'remap-debuginfo = true' along with the '--remap-path-prefix'. [YOCTO# 14875] (From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a) Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Add extra logging when console doesn't appearRichard Purdie2023-10-101-0/+10
| | | | | | | | | | | | | If the console doesn't appear, breifly sleep and try an extra read. This is a useful debugging trick which we may as well preserve in the code. We're already failing at this point so extra data is useful. This means if we perturb things on the other port, we may gain useful logging insight. (From OE-Core rev: be3724854947ded160789d0353b7efb8c593040f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu/qemurunner: Use nodelay with tcp serial connectionsRichard Purdie2023-10-101-0/+1
| | | | | | | | | This disables Nagle's algorithm for our tcp serial connections which may be causing data transfer issues. (From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemurunner: Add newlines serial workaroundRichard Purdie2023-10-101-0/+12
| | | | | | | | | | | We're struggling with the 6.5 kernel as the serial port getty doesn't appears sometimes leading to failures in CI. Add a workaround of sending some newlines as a way of unblocking the kernel/release issues whilst we try and work out how to get to the bottom of the issue. (From OE-Core rev: 0a65f0d272895ba13c8c133ee71f3605d765a8a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/bblock: add self test for bblock toolJulien Stephan2023-10-091-0/+203
| | | | | | | | | | | | it implements various combination of locking single/multiple recipe(s)/task(s) it also tests that locked sig are architecture dependant (From OE-Core rev: 7047a067548acd592a834d4315579fb606b4b9d7) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: add a new info level for SIGGEN_LOCKEDSIGS_TASKSIG_CHECKJulien Stephan2023-10-091-1/+17
| | | | | | | | | | | | | | | | | | | | | | | as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and "error", displaying respectively a warning or a fatal error message only when a task is locked and the task signature is different from the locked one. The "info" level is introduced to add a "note" message to remind the user that a recipe is locked even if the signature is equivalent to the locked one. The "warn" and "error" level display the warn/error message for each task having a mismatch of the signature. Doing this with the "info" level would result in very verbose output if there are several tasks locked, so the info level will only print once the list of recipes that have locked signature. (From OE-Core rev: 840402181d36ca3f60119984478979afb5bb3bbf) Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner.py: detect login prompt without without utf-8 conversionMikko Rapeli2023-10-081-2/+1
| | | | | | | | | | | In case utf-8 conversion of the serial console data is flaky due to bad characters in the stream. (From OE-Core rev: 3f4921ced1e2aae546ff7cef232eec3e214c28be) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/concurrencytest: Remove invalid buffering optionRichard Purdie2023-10-051-2/+2
| | | | | | | | | | | | | Fix warnings from oe-selftest -j: /usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used return io.open(fd, mode, buffering, encoding, *args, **kwargs) Remove the option since it clearly doesn't do much. (From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Show both the login console log and all logging upon failureRichard Purdie2023-10-041-3/+2
| | | | | | | | | It is unclear when things fail which output was on which serial port. Improve the output to show the last lines of both data to improve debugging. (From OE-Core rev: 605938b3bb19dcf6c7218648b4d16df9eaa675fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Log the second serial console as well as the firstRichard Purdie2023-10-041-6/+3
| | | | | | | | | | To aid debugging, always log the second serial console as well as the first to a seperate log file. This should make it clearer what happened when we see test failures. (From OE-Core rev: 372cd58fe672726900af80f3e65a108984da2750) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Use backslashreplace with utf8 to make invalid characters clearRichard Purdie2023-10-041-2/+2
| | | | | | | | | Switch to the backslashreplace error handling when decoding strings so that invalid characters are clear in the stream to improve debugging. (From OE-Core rev: 8ec86d46f736ec3e625b741e97545377c79414ee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qemurunner: Skip conversion from/to binary data for logfileRichard Purdie2023-10-041-4/+3
| | | | | | | | | | | | | There is no point in decoding binary data only to encode it again risking conversion issues. Write the raw data to the log file as binary and skip the conversion. Also always update self.msg even if a logfile isn't specified to improve logging/debug. (From OE-Core rev: b5d67471dddd198723c4f711747783a33e8e5987) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/gitarchive: Handle broken commit counts in results repoRichard Purdie2023-09-281-1/+9
| | | | | | | | | | | | | | | | | | | | | The test results repository contains tags like: master/64501-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0 master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/3 master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/2 master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/1 master/1-g65c94ca3196e5ef3344a469fea8e30444f2e967a/0 where the commit count is correct in one case and not in the others. This causes assertion errors in the current code. Add in some code to work around these historical issues where the commit counts are low. (From OE-Core rev: d51fc5c8c469730885af7bbde7122032de411d89) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* qa: Add selftest for python3-spdx-toolsSamantha Jalabert2023-09-271-0/+54
| | | | | | | | | (From OE-Core rev: e766fe7a9391cc9348d92ab704ec58c4806e38c6) Signed-off-by: Marta Rybczynska <mrybczynska@syslinbit.com> Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: parse the logs with Python, not grepRoss Burton2023-09-261-50/+41
| | | | | | | | | | | | | | | | | Instead of constructing huge grep statements, we can simply open the logs in Python and do the relevant string operations directly. The trick is to remember to casefold() all of the strings, so that the "in" operator can be used. Just one of the ignores needs to be adjusted because it uses a regular expression and the new logic doesn't support that. This is handled by simply reducing the size of the ignore match. (From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: select the correct machine-specific ignores earlyRoss Burton2023-09-261-7/+6
| | | | | | | | | | This has no impact to the execution, but makes the following changes neater. (From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: move some variables out of global scopeRoss Burton2023-09-261-10/+9
| | | | | | | | | | errors and log_locations can be trivially set in the class directly, instead of being defined in the module and then copied into the class. (From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/parselogs: don't pass around membersRoss Burton2023-09-261-10/+14
| | | | | | | | | | There's no point in passing around member fields, just access them directly. (From OE-Core rev: a24d6eda061363cdcfa95980cd2698a674737d23) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2026-09-09 21:55:53 +0000