1 files changed, 131 insertions, 0 deletions
diff --git a/documentation/ref-manual/ref-release-process.xml b/documentation/ref-manual/ref-release-process.xml
index 1b337f8bd7..fe3ba0933e 100644
--- a/documentation/ref-manual/ref-release-process.xml
+++ b/documentation/ref-manual/ref-release-process.xml
@@ -117,6 +117,137 @@
    </para>
 </section>
+<section id='testing-and-quality-assurance'>
+    <title>Testing and Quality Assurance</title>
+    <para>
+        Part of the Yocto Project development and release process is quality
+        assurance through the execution of test strategies.
+        Test strategies provide the Yocto Project team a way to ensure a
+        release is validated.
+        Additionally, because the test strategies are visible to you as a
+        developer, you can validate your projects.
+        This section overviews the available test infrastructure used in the
+        Yocto Project.
+        For information on how to run available tests on your projects, see the
+        "<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
+        section in the Yocto Project Development Manual.
+    </para>
+    <para>
+        The QA/testing infrastructure is woven into the project to the point
+        where core developers take some of it for granted.
+        The infrastructure consists of the following pieces:
+        <itemizedlist>
+            <listitem><para>
+                <filename>bitbake-selftest</filename>:
+                A standalone command that runs unit tests on key pieces of
+                BitBake and its fetchers.
+                </para></listitem>
+            <listitem><para>
+                <link linkend='ref-classes-sanity'><filename>sanity.bbclass</filename></link>:
+                This automatically included class checks the build environment
+                for missing tools (e.g. <filename>gcc</filename>) or common
+                misconfigurations such as
+                <link linkend='var-MACHINE'><filename>MACHINE</filename></link>
+                set incorrectly.
+                </para></listitem>
+            <listitem><para>
+                <link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>:
+                This class checks the generated output from builds for sanity.
+                For example, if building for an ARM target, did the build
+                produce ARM binaries.
+                If, for example, the build produced PPC binaries then there
+                is a problem.
+                </para></listitem>
+            <listitem><para>
+                <link linkend='ref-classes-testimage*'><filename>testimage.bbclass</filename></link>:
+                This class performs runtime testing of images after they are
+                built.
+                The tests are usually used with
+                <ulink url='&YOCTO_DOCS_DEV_URL;#dev-manual-qemu'>QEMU</ulink>
+                to boot the images and check the combined runtime result
+                boot operation and functions.
+                However, the test can also use the IP address of a machine to
+                test.
+                </para></listitem>
+            <listitem><para>
+                <ulink url='&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest'><filename>ptest</filename></ulink>:
+                Runs tests against packages produced during the build for a
+                given piece of software.
+                The test allows the packages to be be run within a target
+                image.
+                </para></listitem>
+            <listitem><para>
+                <filename>oe-selftest</filename>:
+                Tests combination BitBake invocations.
+                These tests operate outside the OpenEmbedded build system
+                itself.
+                The <filename>oe-selftest</filename> can run all tests by
+                default or can run selected tests or test suites.
+                <note>
+                    Running <filename>oe-selftest</filename> requires
+                    host packages beyond the "Essential" grouping.
+                    See the
+                    "<link linkend='required-packages-for-the-host-development-system'>Required Packages for the Host Development System</link>"
+                    section for more information.
+                </note>
+                </para></listitem>
+        </itemizedlist>
+    </para>
+    <para>
+        Originally, much of this testing was done manually.
+        However, significant effort has been made to automate the tests so
+        that more people can use them and the Yocto Project development team
+        can run them faster and more efficiently.
+    </para>
+    <para>
+        The Yocto Project's main Autobuilder
+        (<filename>autobuilder.yoctoproject.org</filename>) publicly tests
+        each Yocto Project release's code in the OE-Core, Poky, and BitBake
+        repositories.
+        The testing occurs for both the current state of the
+        "master" branch and also for submitted patches.
+        Testing for submitted patches usually occurs in the
+        "ross/mut" branch in the <filename>poky-contrib</filename> repository
+        (i.e. the master-under-test branch) or in the "master-next" branch
+        in the <filename>poky</filename> repository.
+        <note>
+            You can find all these branches in the Yocto Project
+            <ulink url='&YOCTO_DOCS_DEV_URL;#source-repositories'>Source Repositories</ulink>.
+        </note>
+        Testing within these public branches ensures in a publicly visible way
+        that all of the main supposed architectures and recipes in OE-Core
+        successfully build and behave properly.
+    </para>
+    <para>
+        Various features such as <filename>multilib</filename>, sub
+        architectures (e.g. <filename>x32</filename>,
+        <filename>poky-tiny</filename>, <filename>musl</filename>,
+        <filename>no-x11</filename> and and so forth),
+        <filename>bitbake-selftest</filename>, and
+        <filename>oe-selftest</filename> are tested as part of
+        the QA process of a release.
+        Complete testing and validation for a release takes the Autobuilder
+        workers several hours.
+        <note>
+            The Autobuilder workers are non-homogeneous, which means regular
+            testing across a variety of Linux distributions occurs.
+            The Autobuilder is limited to only testing QEMU-based setups and
+            not real hardware.
+        </note>
+    </para>
+    <para>
+        Finally, in addition to the Autobuilder's tests, the Yocto Project
+        QA team also performs testing on a variety of platforms, which includes
+        actual hardware, to ensure expected results.
+    </para>
+</section>
 </chapter>
 <!--
 vim: expandtab tw=80 ts=4

diff --git a/documentation/ref-manual/ref-release-process.xml b/documentation/ref-manual/ref-release-process.xml index 1b337f8bd7..fe3ba0933e 100644 --- a/documentation/ref-manual/ref-release-process.xml +++ b/documentation/ref-manual/ref-release-process.xml
@@ -117,6 +117,137 @@
117	</para>	117	</para>
118	</section>	118	</section>
119		119
		120	<section id='testing-and-quality-assurance'>
		121	<title>Testing and Quality Assurance</title>
		122
		123	<para>
		124	Part of the Yocto Project development and release process is quality
		125	assurance through the execution of test strategies.
		126	Test strategies provide the Yocto Project team a way to ensure a
		127	release is validated.
		128	Additionally, because the test strategies are visible to you as a
		129	developer, you can validate your projects.
		130	This section overviews the available test infrastructure used in the
		131	Yocto Project.
		132	For information on how to run available tests on your projects, see the
		133	"<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
		134	section in the Yocto Project Development Manual.
		135	</para>
		136
		137	<para>
		138	The QA/testing infrastructure is woven into the project to the point
		139	where core developers take some of it for granted.
		140	The infrastructure consists of the following pieces:
		141	<itemizedlist>
		142	<listitem><para>
		143	<filename>bitbake-selftest</filename>:
		144	A standalone command that runs unit tests on key pieces of
		145	BitBake and its fetchers.
		146	</para></listitem>
		147	<listitem><para>
		148	<link linkend='ref-classes-sanity'><filename>sanity.bbclass</filename></link>:
		149	This automatically included class checks the build environment
		150	for missing tools (e.g. <filename>gcc</filename>) or common
		151	misconfigurations such as
		152	<link linkend='var-MACHINE'><filename>MACHINE</filename></link>
		153	set incorrectly.
		154	</para></listitem>
		155	<listitem><para>
		156	<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>:
		157	This class checks the generated output from builds for sanity.
		158	For example, if building for an ARM target, did the build
		159	produce ARM binaries.
		160	If, for example, the build produced PPC binaries then there
		161	is a problem.
		162	</para></listitem>
		163	<listitem><para>
		164	<link linkend='ref-classes-testimage*'><filename>testimage.bbclass</filename></link>:
		165	This class performs runtime testing of images after they are
		166	built.
		167	The tests are usually used with
		168	<ulink url='&YOCTO_DOCS_DEV_URL;#dev-manual-qemu'>QEMU</ulink>
		169	to boot the images and check the combined runtime result
		170	boot operation and functions.
		171	However, the test can also use the IP address of a machine to
		172	test.
		173	</para></listitem>
		174	<listitem><para>
		175	<ulink url='&YOCTO_DOCS_DEV_URL;#testing-packages-with-ptest'><filename>ptest</filename></ulink>:
		176	Runs tests against packages produced during the build for a
		177	given piece of software.
		178	The test allows the packages to be be run within a target
		179	image.
		180	</para></listitem>
		181	<listitem><para>
		182	<filename>oe-selftest</filename>:
		183	Tests combination BitBake invocations.
		184	These tests operate outside the OpenEmbedded build system
		185	itself.
		186	The <filename>oe-selftest</filename> can run all tests by
		187	default or can run selected tests or test suites.
		188	<note>
		189	Running <filename>oe-selftest</filename> requires
		190	host packages beyond the "Essential" grouping.
		191	See the
		192	"<link linkend='required-packages-for-the-host-development-system'>Required Packages for the Host Development System</link>"
		193	section for more information.
		194	</note>
		195	</para></listitem>
		196	</itemizedlist>
		197	</para>
		198
		199	<para>
		200	Originally, much of this testing was done manually.
		201	However, significant effort has been made to automate the tests so
		202	that more people can use them and the Yocto Project development team
		203	can run them faster and more efficiently.
		204	</para>
		205
		206	<para>
		207	The Yocto Project's main Autobuilder
		208	(<filename>autobuilder.yoctoproject.org</filename>) publicly tests
		209	each Yocto Project release's code in the OE-Core, Poky, and BitBake
		210	repositories.
		211	The testing occurs for both the current state of the
		212	"master" branch and also for submitted patches.
		213	Testing for submitted patches usually occurs in the
		214	"ross/mut" branch in the <filename>poky-contrib</filename> repository
		215	(i.e. the master-under-test branch) or in the "master-next" branch
		216	in the <filename>poky</filename> repository.
		217	<note>
		218	You can find all these branches in the Yocto Project
		219	<ulink url='&YOCTO_DOCS_DEV_URL;#source-repositories'>Source Repositories</ulink>.
		220	</note>
		221	Testing within these public branches ensures in a publicly visible way
		222	that all of the main supposed architectures and recipes in OE-Core
		223	successfully build and behave properly.
		224	</para>
		225
		226	<para>
		227	Various features such as <filename>multilib</filename>, sub
		228	architectures (e.g. <filename>x32</filename>,
		229	<filename>poky-tiny</filename>, <filename>musl</filename>,
		230	<filename>no-x11</filename> and and so forth),
		231	<filename>bitbake-selftest</filename>, and
		232	<filename>oe-selftest</filename> are tested as part of
		233	the QA process of a release.
		234	Complete testing and validation for a release takes the Autobuilder
		235	workers several hours.
		236	<note>
		237	The Autobuilder workers are non-homogeneous, which means regular
		238	testing across a variety of Linux distributions occurs.
		239	The Autobuilder is limited to only testing QEMU-based setups and
		240	not real hardware.
		241	</note>
		242	</para>
		243
		244	<para>
		245	Finally, in addition to the Autobuilder's tests, the Yocto Project
		246	QA team also performs testing on a variety of platforms, which includes
		247	actual hardware, to ensure expected results.
		248	</para>
		249	</section>
		250
120	</chapter>	251	</chapter>
121	<!--	252	<!--
122	vim: expandtab tw=80 ts=4	253	vim: expandtab tw=80 ts=4