diff options
Diffstat (limited to 'documentation/test-manual/test-process.rst')
-rw-r--r-- | documentation/test-manual/test-process.rst | 50 |
1 files changed, 24 insertions, 26 deletions
diff --git a/documentation/test-manual/test-process.rst b/documentation/test-manual/test-process.rst index 8a5e29d922..7bec5ba828 100644 --- a/documentation/test-manual/test-process.rst +++ b/documentation/test-manual/test-process.rst | |||
@@ -20,8 +20,8 @@ helps review and test patches and this is his testing tree). | |||
20 | We have two broad categories of test builds, including "full" and | 20 | We have two broad categories of test builds, including "full" and |
21 | "quick". On the Autobuilder, these can be seen as "a-quick" and | 21 | "quick". On the Autobuilder, these can be seen as "a-quick" and |
22 | "a-full", simply for ease of sorting in the UI. Use our Autobuilder | 22 | "a-full", simply for ease of sorting in the UI. Use our Autobuilder |
23 | console view to see where me manage most test-related items, available | 23 | :yocto_ab:`console view </typhoon/#/console>` to see where we manage most |
24 | at: :yocto_ab:`/typhoon/#/console`. | 24 | test-related items. |
25 | 25 | ||
26 | Builds are triggered manually when the test branches are ready. The | 26 | Builds are triggered manually when the test branches are ready. The |
27 | builds are monitored by the SWAT team. For additional information, see | 27 | builds are monitored by the SWAT team. For additional information, see |
@@ -34,24 +34,21 @@ which the result was required. | |||
34 | 34 | ||
35 | The Autobuilder does build the ``master`` branch once daily for several | 35 | The Autobuilder does build the ``master`` branch once daily for several |
36 | reasons, in particular, to ensure the current ``master`` branch does | 36 | reasons, in particular, to ensure the current ``master`` branch does |
37 | build, but also to keep ``yocto-testresults`` | 37 | build, but also to keep (:yocto_git:`yocto-testresults </yocto-testresults/>`), |
38 | (:yocto_git:`/yocto-testresults/`), | 38 | (:yocto_git:`buildhistory </poky-buildhistory/>`), and |
39 | buildhistory | 39 | our sstate up to date. On the weekend, there is a ``master-next`` build |
40 | (:yocto_git:`/poky-buildhistory/`), and | ||
41 | our sstate up to date. On the weekend, there is a master-next build | ||
42 | instead to ensure the test results are updated for the less frequently | 40 | instead to ensure the test results are updated for the less frequently |
43 | run targets. | 41 | run targets. |
44 | 42 | ||
45 | Performance builds (buildperf-\* targets in the console) are triggered | 43 | Performance builds (``buildperf-\*`` targets in the console) are triggered |
46 | separately every six hours and automatically push their results to the | 44 | separately every six hours and automatically push their results to the |
47 | buildstats repository at: | 45 | :yocto_git:`buildstats </yocto-buildstats/>` repository. |
48 | :yocto_git:`/yocto-buildstats/`. | ||
49 | 46 | ||
50 | The 'quick' targets have been selected to be the ones which catch the | 47 | The "quick" targets have been selected to be the ones which catch the |
51 | most failures or give the most valuable data. We run 'fast' ptests in | 48 | most failures or give the most valuable data. We run "fast" ptests in |
52 | this case for example but not the ones which take a long time. The quick | 49 | this case for example but not the ones which take a long time. The quick |
53 | target doesn't include \*-lsb builds for all architectures, some world | 50 | target doesn't include ``\*-lsb`` builds for all architectures, some ``world`` |
54 | builds and doesn't trigger performance tests or ltp testing. The full | 51 | builds and doesn't trigger performance tests or ``ltp`` testing. The full |
55 | build includes all these things and is slower but more comprehensive. | 52 | build includes all these things and is slower but more comprehensive. |
56 | 53 | ||
57 | Release Builds | 54 | Release Builds |
@@ -59,20 +56,20 @@ Release Builds | |||
59 | 56 | ||
60 | The project typically has two major releases a year with a six month | 57 | The project typically has two major releases a year with a six month |
61 | cadence in April and October. Between these there would be a number of | 58 | cadence in April and October. Between these there would be a number of |
62 | milestone releases (usually four) with the final one being stablization | 59 | milestone releases (usually four) with the final one being stabilization |
63 | only along with point releases of our stable branches. | 60 | only along with point releases of our stable branches. |
64 | 61 | ||
65 | The build and release process for these project releases is similar to | 62 | The build and release process for these project releases is similar to |
66 | that in `Day to Day Development <#test-daily-devel>`__, in that the | 63 | that in :ref:`test-manual/test-process:day to day development`, in that the |
67 | a-full target of the Autobuilder is used but in addition the form is | 64 | a-full target of the Autobuilder is used but in addition the form is |
68 | configured to generate and publish artefacts and the milestone number, | 65 | configured to generate and publish artifacts and the milestone number, |
69 | version, release candidate number and other information is entered. The | 66 | version, release candidate number and other information is entered. The |
70 | box to "generate an email to QA"is also checked. | 67 | box to "generate an email to QA" is also checked. |
71 | 68 | ||
72 | When the build completes, an email is sent out using the send-qa-email | 69 | When the build completes, an email is sent out using the ``send-qa-email`` |
73 | script in the ``yocto-autobuilder-helper`` repository to the list of | 70 | script in the :yocto_git:`yocto-autobuilder-helper </yocto-autobuilder-helper>` |
74 | people configured for that release. Release builds are placed into a | 71 | repository to the list of people configured for that release. Release builds |
75 | directory in https://autobuilder.yocto.io/pub/releases on the | 72 | are placed into a directory in https://autobuilder.yocto.io/pub/releases on the |
76 | Autobuilder which is included in the email. The process from here is | 73 | Autobuilder which is included in the email. The process from here is |
77 | more manual and control is effectively passed to release engineering. | 74 | more manual and control is effectively passed to release engineering. |
78 | The next steps include: | 75 | The next steps include: |
@@ -80,14 +77,15 @@ The next steps include: | |||
80 | - QA teams respond to the email saying which tests they plan to run and | 77 | - QA teams respond to the email saying which tests they plan to run and |
81 | when the results will be available. | 78 | when the results will be available. |
82 | 79 | ||
83 | - QA teams run their tests and share their results in the yocto- | 80 | - QA teams run their tests and share their results in the |
84 | testresults-contrib repository, along with a summary of their | 81 | :yocto_git:`yocto-testresults-contrib </yocto-testresults-contrib>` |
85 | findings. | 82 | repository, along with a summary of their findings. |
86 | 83 | ||
87 | - Release engineering prepare the release as per their process. | 84 | - Release engineering prepare the release as per their process. |
88 | 85 | ||
89 | - Test results from the QA teams are included into the release in | 86 | - Test results from the QA teams are included into the release in |
90 | separate directories and also uploaded to the yocto-testresults | 87 | separate directories and also uploaded to the |
88 | :yocto_git:`yocto-testresults </yocto-testresults>` | ||
91 | repository alongside the other test results for the given revision. | 89 | repository alongside the other test results for the given revision. |
92 | 90 | ||
93 | - The QA report in the final release is regenerated using resulttool to | 91 | - The QA report in the final release is regenerated using resulttool to |