linux/poky.git/meta/lib, branch thud-20.0.2

oeqa/utils/gitarchive: Handle case where parent is only on origin

2019-03-26T15:38:23+00:00

The parent code currently assumed that any parent branch is locally checked out which may not be the case. Use the local branch by default but fall back to the origin. This also means removing the later saftey check as the branch may not exist locally. This fixes the autobuilder resulttool test pushing code. (From OE-Core rev: 36624a17d382d84647c5811134fe76251fd64dc3) Signed-off-by: Richard Purdie

resulttool: Improvements to allow integration to the autobuilder

2019-03-26T15:38:22+00:00

This is a combined patch of the various tweaks and improvements I made to resulttool: * Avoid subprocess.run() as its a python 3.6 feature and we have autobuilder workers with 3.5. * Avoid python keywords as variable names * Simplify dict accesses using .get() * Rename resultsutils -> resultutils to match the resultstool -> resulttool rename * Formalised the handling of "file_name" to "TESTSERIES" which the code will now add into the json configuration data if its not present, based on the directory name. * When we don't have failed test cases, print something saying so instead of an empty table * Tweak the table headers in the report to be more readable (reference "Test Series" instead if file_id and ID instead of results_id) * Improve/simplify the max string length handling * Merge the counts and percentage data into one table in the report since printing two reports of the same data confuses the user * Removed the confusing header in the regression report * Show matches, then regressions, then unmatched runs in the regression report, also remove chatting unneeded output * Try harder to "pair" up matching configurations to reduce noise in the regressions report * Abstracted the "mapping" table concept used to pairing in the regression code to general code in resultutils * Created multiple mappings for results analysis, results storage and 'flattening' results data in a merge * Simplify the merge command to take a source and a destination, letting the destination be a directory or a file, removing the need for an output directory parameter * Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression mappings * Have the store command place the testresults files in a layout from the mapping, making commits into the git repo for results storage more useful for simple comparison purposes * Set the oe-git-archive tag format appropriately for oeqa results storage (and simplify the commit messages closer to their defaults) * Fix oe-git-archive to use the commit/branch data from the results file * Cleaned up the command option help to match other changes * Follow the model of git branch/tag processing used by oe-build-perf-report and use that to read the data using git show to avoid branch change * Add ptest summary to the report command * Update the tests to match the above changes (From OE-Core rev: b4513e75f746a0989b09ee53cb85e489d41e5783) Signed-off-by: Richard Purdie

resulttool: enable merge, store, report and regression analysis

2019-03-26T15:38:22+00:00

OEQA outputs test results into json files and these files were archived by Autobuilder during QA releases. Example: each oe-selftest run by Autobuilder for different host distro generate a testresults.json file. These scripts were developed as a test result tools to manage these testresults.json file. Using the "store" operation, user can store multiple testresults.json files as well as the pre-configured directories used to hold those files. Using the "merge" operation, user can merge multiple testresults.json files to a target file. Using the "report" operation, user can view the test result summary for all available testresults.json files inside a ordinary directory or a git repository. Using the "regression-file" operation, user can perform regression analysis on testresults.json files specified. Using the "regression-dir" and "regression-git" operations, user can perform regression analysis on directory and git accordingly. These resulttool operations expect the testresults.json file to use the json format below. { "": { "configuration": { "": "", "": "", ... "": "", }, "result": { "": { "status": "", "log": "" }, "": { "status": "", "log": "" }, ... "": { "status": "", "log": "" }, } }, ... "": { "configuration": { "": "", "": "", ... "": "", }, "result": { "": { "status": "", "log": "" }, "": { "status": "", "log": "" }, ... "": { "status": "", "log": "" }, } }, } To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To store test result from oeqa automated tests, execute the below $ resulttool store To merge multiple testresults.json files, execute the below $ resulttool merge To report test report, execute the below $ resulttool report To perform regression file analysis, execute the below $ resulttool regression-file To perform regression dir analysis, execute the below $ resulttool regression-dir To perform regression git analysis, execute the below $ resulttool regression-git [YOCTO# 13012] [YOCTO# 12654] (From OE-Core rev: 39cda3498ec68b91a671ead256231324d74f7d4c) Signed-off-by: Yeoh Ee Peng Signed-off-by: Richard Purdie

oe-build-perf-report/gitarchive: Move common useful functions to library

2019-02-25T22:27:40+00:00

These functions can be reused by the resulttool code so move to the common function library for this purpose. (From OE-Core rev: c66f848938c04e133259c5b6903dc592866ab385) (From OE-Core rev: 4b1bd35030c5502873106782a35c4f5a9446e20c) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

scripts/oe-git-archive: Separate out functionality to library function

2019-02-25T22:27:40+00:00

This turns the core of the script into a library function. Ultimately this will let us call that code with custom 'keywords' rather than relying on the data parsed from bitbake metadata which can't be used when archiving historical results. (From OE-Core rev: 4820ca2b0850e29b04a4fd5659a6e9837d6714d0) (From OE-Core rev: e47c48768a01191ec36b9732288e05c9f5e80d52) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

logrotate.py: restore /etc/logrotate.d/wtmp

2019-02-25T22:27:39+00:00

During the test logrotate.LogrotateTest.test_1_logrotate_setup, there is below logic: # mkdir $HOME/logrotate_dir # sed -i "s#wtmp {#wtmp {\n olddir $HOME/logrotate_dir#" /etc/logrotate.d/wtmp After all logrotate.LogrotateTest finished, only cleanup $HOME/logrotate_dir as below, but don't restore the config file /etc/logrotate.d/wtmp. [snip] def tearDownClass(cls): cls.tc.target.run('rm -rf $HOME/logrotate_dir') [snip] That's to say, there is one additional line added to /etc/logrotate.d/wtmp and will make the logrotate service start failed when run systemd.SystemdBasicTests.test_systemd_failed Take an example as below when run test as root: # cat /etc/logrotate.d/wtmp # no packages own wtmp -- we'll rotate it here /var/log/wtmp { olddir /root/logrotate_dir missingok monthly create 0664 root utmp minsize 1M rotate 1 } # ls /root/logrotate_dir ls: cannot access '/root/logrotate_dir': No such file or directory # systemctl start logrotate Job for logrotate.service failed because the control process exited with error code. See "systemctl status logrotate.service" and "journalctl -xe" for details. # systemctl status logrotate logrotate.service - Rotate log files Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset> Active: failed (Result: exit-code) since Wed 2019-02-13 03:35:19 UTC; 7s ago Docs: man:logrotate(8) man:logrotate.conf(5) Process: 540 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE) Main PID: 540 (code=exited, status=1/FAILURE) Feb 13 03:35:18 qemumips systemd[1]: Starting Rotate log files... Feb 13 03:35:19 qemumips logrotate[540]: error: wtmp:9 error verifying olddir path /root/logrotate_dir: No such file or directory Feb 13 03:35:19 qemumips logrotate[540]: error: found error in file wtmp, skipping Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Failed with result 'exit-code'. Feb 13 03:35:19 qemumips systemd[1]: Failed to start Rotate log files. Add the logic to restore /etc/logrotate.d/wtmp to make the cleanup complete to fix the above issue. (From OE-Core rev: a2db9320d97d12d87524ff16a329f9c38a8da33f) (From OE-Core rev: 9036f78bdfde1d79af6ce57db78e2cdff9fb5677) Signed-off-by: Mingli Yu Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

oeqa/runtime/ptest: Ensure OOM errors are logged

2019-02-25T22:27:39+00:00

Currently processed being killed by the OOM killer may not be spotted by ptest-runner. After we complete the tests, check the logs and report if there were any. This ensures the user is aware of OOM conditions affecting the ptest results. (From OE-Core rev: 20a441d53817f80e0ce1597e77f6e794422ac49a) (From OE-Core rev: f922970b119849bcdd7f7df74bd2029816ff637c) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

oeqa/logparser: Various misc cleanups

2019-02-25T22:27:39+00:00

Get rid of further unneeded code complications: * value mappings we could just direct use * ftools when we can write files easily ourself * test result status filtering we don't use * variable overwriting module imports (From OE-Core rev: d6065f136f6d353c3054cc3f440a4e259509f876) (From OE-Core rev: ba944a72302fa088c31c7b1eee4ad9f64f9769e4) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

oeqa/logparser: Improve results handling

2019-02-25T22:27:39+00:00

Merge the results handling into the ptest log parser as a seperate method. Drop the weird "pass.skip.fail." prefix to the results filename, its just bizarre. Drop the code turning a list into a regex then searching the regex for an item, "x in y" is perfectly capable. Use a dict, sort the keys as needed and drop the list sorting code. (From OE-Core rev: f317800e950b4a37b4034133bc52e0c47f04dc29) (From OE-Core rev: 966ffaada3e9f43a25b1361c53d4b16a521aa517) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

oeqa/utils/logparser: Add in support for duration, exitcode and logs by section

2019-02-25T22:27:39+00:00

Allow parsing of the ptest duration, exit code and timeout keywords from the logs, returning data on each section. Also include the logs broken out per section. (From OE-Core rev: a9a67dccaa5be0f06eedcab46dcff7cbf9202850) (From OE-Core rev: dc49021f75ed7e82713d1c9a04e045718bb9a548) Signed-off-by: Richard Purdie Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie