patchtest: add test_commit_message_user_tags

2024-10-23T16:35:23+00:00

This test makes patchtest check to ensure that there aren't any GitHub-style user account names being tagged in the commit message, e.g. it should catch lines like: "fix added by @threexc" This is desired so that if (for example) we add upstream changelogs in recipe upgrade commit messages verbatim, we don't end up subscribing any associated maintainers to our repo mirrors' updates by accident. There is a small possibility of a false positive with this test, where if someone is mentioning Python decorators in their commit message (or similar syntax from other languages), it will fail when it should pass. However, having this test in place to guard against username inclusion is more important that the occasional false positive for that reason. With this addition, a failure will look like: |FAIL: test commit message user tags: Mbox includes one or more GitHub-style username tags. Ensure that any "@" symbols are stripped out of usernames (test_mbox.TestMbox.test_commit_message_user_tags) (From OE-Core rev: 8278d82d8203212bb159eb3805d4a5617c5370df) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest: selftest: fix patch files

2024-06-05T15:47:58+00:00

Recent changes to the selftest-hello recipe caused the patchtest selftest patches to not apply cleanly, which in turn meant several tests weren't being fully executed or were reporting unexpected PASS/SKIP. Update all of the test patch files to use a newer baseline and apply cleanly again. (From OE-Core rev: d73373d8a590d68e448af2e9ac600adc85777604) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest: tests: update bugzilla_entry_format.fail testfile

2024-05-13T15:28:35+00:00

Update the test file to target a more stable README file. (From OE-Core rev: 3d0649ea8ca9dde982ee67aa866170ea4297ef4c) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest: Add selftest for test cve_check_ignore

2024-02-19T11:39:08+00:00

Add a selftest for the recently introduced check for CVE_CHECK_IGNORE. (From OE-Core rev: dc6ebbbb94b162d8aa68e46d5a166606ace5e39c) Signed-off-by: Simone Weiß Signed-off-by: Richard Purdie

patchtest: remove test for CVE tag in mbox

2023-11-01T19:19:40+00:00

After patchtest went live it was determined that testing for a CVE tag in the mbox commit message is unnecessary, since it will already be in the shortlog and in any carried patches. Remove the test and the associated selftest files so that its absence isn't flagged in future test results. (From OE-Core rev: 54690f18f04a2ab993a85d551ce4f8d0fa56618a) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest: simplify test directory structure

2023-10-26T20:11:28+00:00

Consolidate the various mbox tests into a new TestMbox class, metadata tests into TestMetadata, and patch tests into TestPatch. Also update the selftest filenames to match the changes. The test contents are not significantly changed (other than to reference the new class names). While this doesn't improve overall readability, it does result in more obvious categorization, and more importantly reduces the number of calls to setup tinfoil in the tests, resulting in a roughly 25% reduction in runtime. Before: [tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass) XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail) XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail) XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass) XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail) XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail) XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass) XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail) XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass) XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass) XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail) XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail) XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail) XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip) XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass) XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail) XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass) XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail) XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail) XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail) XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass) XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip) XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass) XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail) XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass) XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass) XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass) XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass) XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass) XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail) XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail) XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail) XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass) XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail) ============================================================================ Testsuite summary for patchtest ============================================================================ ============================================================================ real 24m14.386s user 1m13.599s sys 0m21.477s After: [tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail) XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass) XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail) XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail) XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip) XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass) XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail) XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass) XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail) XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail) XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail) XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass) XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail) XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail) XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail) XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass) XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip) XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass) XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass) XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass) XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass) XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass) XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass) XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail) XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail) XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail) XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass) XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass) XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass) XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail) XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass) XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail) XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail) XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail) ============================================================================ Testsuite summary for patchtest ============================================================================ ============================================================================ real 18m39.749s user 0m41.857s sys 0m14.708s (From OE-Core rev: f788592da2fd0e21638ce2c3326675a060ba51cf) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest/selftest: add XSKIP, update test files

2023-10-25T22:10:19+00:00

Since we are skipping the merge test, two of the selftests now report SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have the right endings for XSKIP, and add the category so that it can be used for more extensive testing in the future. (From OE-Core rev: 3331f53c0be2575784a042bb2401eeba4f2a5a3e) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

patchtest: Add tests from patchtest oe repo

2023-09-14T14:20:08+00:00

Copy the core components of the patchtest-oe repo into meta/lib/patchtest in oe-core. (From OE-Core rev: 257f64f4e4414b78981104aec132b067beb5a92a) Signed-off-by: Trevor Gamblin Signed-off-by: Richard Purdie

linux/poky.git/meta/lib/patchtest/selftest/files, branch 5.2_M2