1 files changed, 109 insertions, 0 deletions
diff --git a/documentation/test-manual/test-manual-test-process.xml b/documentation/test-manual/test-manual-test-process.xml
new file mode 100644
index 0000000000..0b5036cd2c
--- /dev/null
+++ b/documentation/test-manual/test-manual-test-process.xml
@@ -0,0 +1,109 @@
+<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
+[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
+<chapter id='test-manual-test-process'>
+<title>Project Testing and Release Process</title>
+    <section id='test-daily-devel'>
+        <title>Day to Day Development</title>
+        <para>This section details how the project tests changes, through automation on the
+            Autobuilder or with the assistance of QA teams, through to making releases.</para>
+        <para>The project aims to test changes against our test matrix before those changes are
+            merged into the master branch. As such, changes are queued up in batches either in the
+                <filename>master-next</filename> branch in the main trees, or in user trees such as
+                <filename>ross/mut</filename> in <filename>poky-contrib</filename> (Ross Burton
+            helps review and test patches and this is his testing tree).</para>
+        <para>We have two broad categories of test builds, including "full" and "quick". On the
+            Autobuilder, these can be seen as "a-quick" and "a-full", simply for ease of sorting in
+            the UI. Use our Autobuilder console view to see where me manage most test-related items,
+            available at: <link linkend=""
+                >https://autobuilder.yoctoproject.org/typhoon/#/console</link>.</para>
+        <para>Builds are triggered manually when the test branches are ready. The builds are
+            monitored by the SWAT team. For additional information, see <link linkend=""
+                >https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team</link>. If
+            successful, the changes would usually be merged to the <filename>master</filename>
+            branch. If not successful, someone would respond to the changes on the mailing list
+            explaining that there was a failure in testing. The choice of quick or full would depend
+            on the type of changes and the speed with which the result was required.</para>
+        <para>The Autobuilder does build the <filename>master</filename> branch once daily for
+            several reasons, in particular, to ensure the current <filename>master</filename> branch
+            does build, but also to keep <filename>yocto-testresults</filename> (<link linkend=""
+                >http://git.yoctoproject.org/cgit.cgi/yocto-testresults/</link>), buildhistory
+                (<link linkend="">http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/</link>),
+            and our sstate up to date. On the weekend, there is a master-next build instead to
+            ensure the test results are updated for the less frequently run targets.</para>
+        <para>Performance builds (buildperf-* targets in the console) are triggered separately every
+            six hours and automatically push their results to the buildstats repository at: <link
+                linkend="">http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/</link>. </para>
+        <para>The 'quick' targets have been selected to be the ones which catch the most failures or
+            give the most valuable data. We run 'fast' ptests in this case for example but not the
+            ones which take a long time. The quick target doesn't include *-lsb builds for all
+            architectures, some world builds and doesn't trigger performance tests or ltp testing.
+            The full build includes all these things and is slower but more comprehensive.</para>
+    </section>
+    <section id='test-yocto-project-autobuilder-overview'>
+        <title>Release Builds</title>
+        <para>The project typically has two major releases a year with a six month cadence in April
+            and October. Between these there would be a number of milestone releases (usually four)
+            with the final one being stablization only along with point releases of our stable
+            branches.</para>
+        <para>The build and release process for these project releases is similar to that in <link
+                linkend="test-daily-devel">Day to Day Development</link>, in that the a-full target
+            of the Autobuilder is used but in addition the form is configured to generate and
+            publish artefacts and the milestone number, version, release candidate number and other
+            information is entered. The box to "generate an email to QA"is also checked.</para>
+        <para>When the build completes, an email is sent out using the send-qa-email script in the
+                <filename>yocto-autobuilder-helper</filename> repository to the list of people
+            configured for that release. Release builds are placed into a directory in <link
+                linkend="">https://autobuilder.yocto.io/pub/releases</link> on the Autobuilder which
+            is included in the email. The process from here is more manual and control is
+            effectively passed to release engineering. The next steps include:<itemizedlist>
+                <listitem>
+                    <para>QA teams respond to the email saying which tests they plan to run and when
+                        the results will be available.</para>
+                </listitem>
+                <listitem>
+                    <para>QA teams run their tests and share their results in the yocto-
+                        testresults-contrib repository, along with a summary of their findings.
+                    </para>
+                </listitem>
+                <listitem>
+                    <para>Release engineering prepare the release as per their process. </para>
+                </listitem>
+                <listitem>
+                    <para>Test results from the QA teams are included into the release in separate
+                        directories and also uploaded to the yocto-testresults repository alongside
+                        the other test results for the given revision.</para>
+                </listitem>
+                <listitem>
+                    <para>The QA report in the final release is regenerated using resulttool to
+                        include the new test results and the test summaries from the teams (as
+                        headers to the generated report).</para>
+                </listitem>
+                <listitem>
+                    <para>The release is checked against the release checklist and release readiness
+                        criteria.</para>
+                </listitem>
+                <listitem>
+                    <para>A final decision on whether to release is made by the YP TSC who have
+                        final oversight on release readiness.</para>
+                </listitem>
+            </itemizedlist></para>
+    </section>
+</chapter>
+<!--
+vim: expandtab tw=80 ts=4
+-->

diff --git a/documentation/test-manual/test-manual-test-process.xml b/documentation/test-manual/test-manual-test-process.xml new file mode 100644 index 0000000000..0b5036cd2c --- /dev/null +++ b/documentation/test-manual/test-manual-test-process.xml
@@ -0,0 +1,109 @@
	1	<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
	2	"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
	3	[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >
	4
	5	<chapter id='test-manual-test-process'>
	6
	7	<title>Project Testing and Release Process</title>
	8	<section id='test-daily-devel'>
	9	<title>Day to Day Development</title>
	10
	11	<para>This section details how the project tests changes, through automation on the
	12	Autobuilder or with the assistance of QA teams, through to making releases.</para>
	13
	14	<para>The project aims to test changes against our test matrix before those changes are
	15	merged into the master branch. As such, changes are queued up in batches either in the
	16	<filename>master-next</filename> branch in the main trees, or in user trees such as
	17	<filename>ross/mut</filename> in <filename>poky-contrib</filename> (Ross Burton
	18	helps review and test patches and this is his testing tree).</para>
	19	<para>We have two broad categories of test builds, including "full" and "quick". On the
	20	Autobuilder, these can be seen as "a-quick" and "a-full", simply for ease of sorting in
	21	the UI. Use our Autobuilder console view to see where me manage most test-related items,
	22	available at: <link linkend=""
	23	>https://autobuilder.yoctoproject.org/typhoon/#/console</link>.</para>
	24	<para>Builds are triggered manually when the test branches are ready. The builds are
	25	monitored by the SWAT team. For additional information, see <link linkend=""
	26	>https://wiki.yoctoproject.org/wiki/Yocto_Build_Failure_Swat_Team</link>. If
	27	successful, the changes would usually be merged to the <filename>master</filename>
	28	branch. If not successful, someone would respond to the changes on the mailing list
	29	explaining that there was a failure in testing. The choice of quick or full would depend
	30	on the type of changes and the speed with which the result was required.</para>
	31	<para>The Autobuilder does build the <filename>master</filename> branch once daily for
	32	several reasons, in particular, to ensure the current <filename>master</filename> branch
	33	does build, but also to keep <filename>yocto-testresults</filename> (<link linkend=""
	34	>http://git.yoctoproject.org/cgit.cgi/yocto-testresults/</link>), buildhistory
	35	(<link linkend="">http://git.yoctoproject.org/cgit.cgi/poky-buildhistory/</link>),
	36	and our sstate up to date. On the weekend, there is a master-next build instead to
	37	ensure the test results are updated for the less frequently run targets.</para>
	38	<para>Performance builds (buildperf-* targets in the console) are triggered separately every
	39	six hours and automatically push their results to the buildstats repository at: <link
	40	linkend="">http://git.yoctoproject.org/cgit.cgi/yocto-buildstats/</link>. </para>
	41	<para>The 'quick' targets have been selected to be the ones which catch the most failures or
	42	give the most valuable data. We run 'fast' ptests in this case for example but not the
	43	ones which take a long time. The quick target doesn't include *-lsb builds for all
	44	architectures, some world builds and doesn't trigger performance tests or ltp testing.
	45	The full build includes all these things and is slower but more comprehensive.</para>
	46	</section>
	47
	48	<section id='test-yocto-project-autobuilder-overview'>
	49	<title>Release Builds</title>
	50
	51	<para>The project typically has two major releases a year with a six month cadence in April
	52	and October. Between these there would be a number of milestone releases (usually four)
	53	with the final one being stablization only along with point releases of our stable
	54	branches.</para>
	55	<para>The build and release process for these project releases is similar to that in <link
	56	linkend="test-daily-devel">Day to Day Development</link>, in that the a-full target
	57	of the Autobuilder is used but in addition the form is configured to generate and
	58	publish artefacts and the milestone number, version, release candidate number and other
	59	information is entered. The box to "generate an email to QA"is also checked.</para>
	60	<para>When the build completes, an email is sent out using the send-qa-email script in the
	61	<filename>yocto-autobuilder-helper</filename> repository to the list of people
	62	configured for that release. Release builds are placed into a directory in <link
	63	linkend="">https://autobuilder.yocto.io/pub/releases</link> on the Autobuilder which
	64	is included in the email. The process from here is more manual and control is
	65	effectively passed to release engineering. The next steps include:<itemizedlist>
	66	<listitem>
	67	<para>QA teams respond to the email saying which tests they plan to run and when
	68	the results will be available.</para>
	69	</listitem>
	70	<listitem>
	71	<para>QA teams run their tests and share their results in the yocto-
	72	testresults-contrib repository, along with a summary of their findings.
	73	</para>
	74	</listitem>
	75	<listitem>
	76	<para>Release engineering prepare the release as per their process. </para>
	77	</listitem>
	78	<listitem>
	79	<para>Test results from the QA teams are included into the release in separate
	80	directories and also uploaded to the yocto-testresults repository alongside
	81	the other test results for the given revision.</para>
	82	</listitem>
	83	<listitem>
	84	<para>The QA report in the final release is regenerated using resulttool to
	85	include the new test results and the test summaries from the teams (as
	86	headers to the generated report).</para>
	87	</listitem>
	88	<listitem>
	89	<para>The release is checked against the release checklist and release readiness
	90	criteria.</para>
	91	</listitem>
	92	<listitem>
	93	<para>A final decision on whether to release is made by the YP TSC who have
	94	final oversight on release readiness.</para>
	95	</listitem>
	96	</itemizedlist></para>
	97	</section>
	98
	99
	100
	101
	102
	103
	104
	105
	106	</chapter>
	107	<!--
	108	vim: expandtab tw=80 ts=4
	109	-->