summaryrefslogtreecommitdiffstats
path: root/documentation/test-manual/test-process.rst
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/test-manual/test-process.rst')
-rw-r--r--documentation/test-manual/test-process.rst50
1 files changed, 24 insertions, 26 deletions
diff --git a/documentation/test-manual/test-process.rst b/documentation/test-manual/test-process.rst
index 8a5e29d922..7bec5ba828 100644
--- a/documentation/test-manual/test-process.rst
+++ b/documentation/test-manual/test-process.rst
@@ -20,8 +20,8 @@ helps review and test patches and this is his testing tree).
20We have two broad categories of test builds, including "full" and 20We have two broad categories of test builds, including "full" and
21"quick". On the Autobuilder, these can be seen as "a-quick" and 21"quick". On the Autobuilder, these can be seen as "a-quick" and
22"a-full", simply for ease of sorting in the UI. Use our Autobuilder 22"a-full", simply for ease of sorting in the UI. Use our Autobuilder
23console view to see where me manage most test-related items, available 23:yocto_ab:`console view </typhoon/#/console>` to see where we manage most
24at: :yocto_ab:`/typhoon/#/console`. 24test-related items.
25 25
26Builds are triggered manually when the test branches are ready. The 26Builds are triggered manually when the test branches are ready. The
27builds are monitored by the SWAT team. For additional information, see 27builds are monitored by the SWAT team. For additional information, see
@@ -34,24 +34,21 @@ which the result was required.
34 34
35The Autobuilder does build the ``master`` branch once daily for several 35The Autobuilder does build the ``master`` branch once daily for several
36reasons, in particular, to ensure the current ``master`` branch does 36reasons, in particular, to ensure the current ``master`` branch does
37build, but also to keep ``yocto-testresults`` 37build, but also to keep (:yocto_git:`yocto-testresults </yocto-testresults/>`),
38(:yocto_git:`/yocto-testresults/`), 38(:yocto_git:`buildhistory </poky-buildhistory/>`), and
39buildhistory 39our sstate up to date. On the weekend, there is a ``master-next`` build
40(:yocto_git:`/poky-buildhistory/`), and
41our sstate up to date. On the weekend, there is a master-next build
42instead to ensure the test results are updated for the less frequently 40instead to ensure the test results are updated for the less frequently
43run targets. 41run targets.
44 42
45Performance builds (buildperf-\* targets in the console) are triggered 43Performance builds (``buildperf-\*`` targets in the console) are triggered
46separately every six hours and automatically push their results to the 44separately every six hours and automatically push their results to the
47buildstats repository at: 45:yocto_git:`buildstats </yocto-buildstats/>` repository.
48:yocto_git:`/yocto-buildstats/`.
49 46
50The 'quick' targets have been selected to be the ones which catch the 47The "quick" targets have been selected to be the ones which catch the
51most failures or give the most valuable data. We run 'fast' ptests in 48most failures or give the most valuable data. We run "fast" ptests in
52this case for example but not the ones which take a long time. The quick 49this case for example but not the ones which take a long time. The quick
53target doesn't include \*-lsb builds for all architectures, some world 50target doesn't include ``\*-lsb`` builds for all architectures, some ``world``
54builds and doesn't trigger performance tests or ltp testing. The full 51builds and doesn't trigger performance tests or ``ltp`` testing. The full
55build includes all these things and is slower but more comprehensive. 52build includes all these things and is slower but more comprehensive.
56 53
57Release Builds 54Release Builds
@@ -59,20 +56,20 @@ Release Builds
59 56
60The project typically has two major releases a year with a six month 57The project typically has two major releases a year with a six month
61cadence in April and October. Between these there would be a number of 58cadence in April and October. Between these there would be a number of
62milestone releases (usually four) with the final one being stablization 59milestone releases (usually four) with the final one being stabilization
63only along with point releases of our stable branches. 60only along with point releases of our stable branches.
64 61
65The build and release process for these project releases is similar to 62The build and release process for these project releases is similar to
66that in `Day to Day Development <#test-daily-devel>`__, in that the 63that in :ref:`test-manual/test-process:day to day development`, in that the
67a-full target of the Autobuilder is used but in addition the form is 64a-full target of the Autobuilder is used but in addition the form is
68configured to generate and publish artefacts and the milestone number, 65configured to generate and publish artifacts and the milestone number,
69version, release candidate number and other information is entered. The 66version, release candidate number and other information is entered. The
70box to "generate an email to QA"is also checked. 67box to "generate an email to QA" is also checked.
71 68
72When the build completes, an email is sent out using the send-qa-email 69When the build completes, an email is sent out using the ``send-qa-email``
73script in the ``yocto-autobuilder-helper`` repository to the list of 70script in the :yocto_git:`yocto-autobuilder-helper </yocto-autobuilder-helper>`
74people configured for that release. Release builds are placed into a 71repository to the list of people configured for that release. Release builds
75directory in https://autobuilder.yocto.io/pub/releases on the 72are placed into a directory in https://autobuilder.yocto.io/pub/releases on the
76Autobuilder which is included in the email. The process from here is 73Autobuilder which is included in the email. The process from here is
77more manual and control is effectively passed to release engineering. 74more manual and control is effectively passed to release engineering.
78The next steps include: 75The next steps include:
@@ -80,14 +77,15 @@ The next steps include:
80- QA teams respond to the email saying which tests they plan to run and 77- QA teams respond to the email saying which tests they plan to run and
81 when the results will be available. 78 when the results will be available.
82 79
83- QA teams run their tests and share their results in the yocto- 80- QA teams run their tests and share their results in the
84 testresults-contrib repository, along with a summary of their 81 :yocto_git:`yocto-testresults-contrib </yocto-testresults-contrib>`
85 findings. 82 repository, along with a summary of their findings.
86 83
87- Release engineering prepare the release as per their process. 84- Release engineering prepare the release as per their process.
88 85
89- Test results from the QA teams are included into the release in 86- Test results from the QA teams are included into the release in
90 separate directories and also uploaded to the yocto-testresults 87 separate directories and also uploaded to the
88 :yocto_git:`yocto-testresults </yocto-testresults>`
91 repository alongside the other test results for the given revision. 89 repository alongside the other test results for the given revision.
92 90
93- The QA report in the final release is regenerated using resulttool to 91- The QA report in the final release is regenerated using resulttool to