ref-manual, dev-manual: Edits to running tests on hardware

Fixes [YOCTO #5554]

Applied edits to the section in the dev-manual on running tests
on hardware.

Edited the TEST_TARGET and TEST_IMAGE variables.

(From yocto-docs rev: a26ba11c739aabab4009a944d9b622e2814c7fca)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

2 files changed, 69 insertions, 38 deletions

diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml
index 63aba00abb..fa6e9cd9b8 100644
--- a/documentation/dev-manual/dev-manual-common-tasks.xml
+++ b/documentation/dev-manual/dev-manual-common-tasks.xml
@@ -5660,7 +5660,7 @@
             </para>
 
             <section id='qemu-image-enabling-tests'>
-                <title>QEMU</title>
+                <title>Enabling Runtime Tests on QEMU</title>
 
                 <para>
                     In order to run tests, you need to do the following:
@@ -5750,7 +5750,43 @@
             </section>
 
             <section id='hardware-image-enabling-tests'>
-                <title>Hardware</title>
+                <title>Enabling Runtime Tests on Hardware</title>
+
+                <para>
+                    The OpenEmbedded build system can run tests on real
+                    hardware, and for certain devices it can also deploy
+                    the image to be tested onto the device beforehand.
+                </para>
+
+                <para>
+                    For automated deployment, a "master image" is installed
+                    onto the hardware once as part of setup.
+                    Then, each time tests are to be run, the following
+                    occurs:
+                    <orderedlist>
+                        <listitem><para>The master image is booted into and
+                            used to write the image to be tested to
+                            a second partition.
+                            </para></listitem>
+                        <listitem><para>The device is then rebooted using an
+                            external script that you need to provide.
+                            </para></listitem>
+                        <listitem><para>The device boots into the image to be
+                            tested.
+                            </para></listitem>
+                    </orderedlist>
+                </para>
+
+                <para>
+                    When running tests (independent of whether the image
+                    has been deployed automatically or not), the device is
+                    expected to be connected to a network on a
+                    pre-determined IP address.
+                    You can either use static IP addresses written into
+                    the image, or set the image to use DHCP and have your
+                    DHCP server on the test network assign a known IP address
+                    based on the MAC address of the device.
+                </para>
 
                 <para>
                     In order to run tests on hardware, you need to set
@@ -5762,13 +5798,17 @@
                     <itemizedlist>
                         <listitem><para><emphasis>"SimpleRemoteTarget":</emphasis>
                             Choose "SimpleRemoteTarget" if you are going to
-                            run tests on a target system that is already up and
-                            running either on the network or as an image in
-                            QEMU.
+                            run tests on a target system that is already
+                            running the image to be tested and is available
+                            on the network.
+                            You can use "SimpleRemoteTarget" in conjunction
+                            with either real hardware or an image running
+                            within a separately started QEMU or any
+                            other virtual machine manager.
                             </para></listitem>
                         <listitem><para><emphasis>"GummiBootTarget":</emphasis>
                             Choose "GummiBootTarget" if your hardware is
-                            a running EFI machine with
+                            an EFI-based machine with
                             <filename>gummiboot</filename> as bootloader and
                             <filename>core-image-testmaster</filename>
                             (or something similar) is installed.
@@ -5781,8 +5821,8 @@
 
                 <para>
                     If you set <filename>TEST_TARGET</filename> to
-                    "GummiBootTarget", you also need to
-                    setup your master image by doing the following:
+                    "GummiBootTarget", you also need to perform a one-time
+                    setup of your master image by doing the following:
                     <orderedlist>
                         <listitem><para><emphasis>Set <filename>EFI_PROVIDER</filename>:</emphasis>
                             Be sure that <filename>EFI_PROVIDER</filename>
@@ -5797,6 +5837,8 @@
                             </para></listitem>
                         <listitem><para><emphasis>Install image:</emphasis>
                             Install the image on the target system.
+                            &lt;Need some clarification here from Stefan,
+                            partition layout, etc.&gt;
                             </para></listitem>
                     </orderedlist>
                 </para>
@@ -5853,14 +5895,17 @@
                             <note>
                                 You need to customize
                                 <filename>TEST_POWERCONTROL_CMD</filename>
-                                and <filename>_ARGS</filename> for your own
-                                setup.
+                                and
+                                <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename>
+                                for your own setup.
                                 The one requirement is that it accepts
                                 "on", "off", and "cycle" as the last argument.
                             </note>
                             </para></listitem>
                         <listitem><para>
-                            When no command is defined, it uses classic reboot.
+                            When no command is defined, it connects to the
+                            device over SSH and uses the classic reboot command
+                            to reboot the device.
                             Classic reboot is fine as long as the machine
                             actually reboots (i.e. the SSH test has not
                             failed).
@@ -5890,18 +5935,6 @@
                         <literallayout class='monospaced'>
      TEST_IMAGE = "1"
                         </literallayout>
-                        Next, also in the <filename>local.conf</filename>, set the
-                        <filename>TEST_TARGET</filename> variable to
-                        "simpleremote" if you want to run tests on real hardware or
-                        set it to "qemu" if you want to run tests using QEMU.
-                        file:
-                        <literallayout class='monospaced'>
-     TEST_TARGET = "simpleremote"
-                        </literallayout>
-                        or
-                        <literallayout class='monospaced'>
-     TEST_TARGET = "qemu"
-                        </literallayout>
                         Next, build your image.
                         If the image successfully builds, the tests will be
                         run:
@@ -6013,7 +6046,6 @@
                 Exporting tests is required if you want to be able to hand
                 the test execution off to a scheduler.
                 You can only export tests that are defined in
-                Tests exported are the ones defined in
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>.
             </para>
 
diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml
index 164ea1c817..fbf0a43dee 100644
--- a/documentation/ref-manual/ref-variables.xml
+++ b/documentation/ref-manual/ref-variables.xml
@@ -7620,10 +7620,9 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                 <para>
                     Automatically runs the series of automated tests for
                     images when an image is successfully built.
-                    <note>
-                        Currently, there is only support for running these tests
-                        under QEMU.
-                    </note>
+                </para>
+
+                <para>
                     These tests are written in Python making use of the
                     <filename>unittest</filename> module, and the majority of
                     them run commands on the target system over
@@ -7722,24 +7721,21 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                     You can provide the following arguments with
                     <filename>TEST_TARGET</filename>:
                     <itemizedlist>
-                        <listitem><para><emphasis>"qemu":</emphasis>
+                        <listitem><para><emphasis>"qemu" and "QemuTarget":</emphasis>
                             Boots a QEMU image and runs the tests.
                             See the
-                            "<ulink url='&YOCTO_DOCS_DEV_URL;#qemu-image-enabling-tests'>QEMU</ulink>"
+                            "<ulink url='&YOCTO_DOCS_DEV_URL;#qemu-image-enabling-tests'>Enabling Runtime Tests on QEMU</ulink>"
                             section in the Yocto Project Development Manual for
                             more information.
                             </para></listitem>
-                        <listitem><para><emphasis>"QemuTarget":</emphasis>
-                            Identical to "qemu".
-                            </para></listitem>
-                        <listitem><para><emphasis>"simpleremote":</emphasis>
+                        <listitem><para><emphasis>"simpleremote" and "SimpleRemoteTarget":</emphasis>
                             Runs the tests on target hardware that is already
                             up and running.
                             The hardware can be on the network or it can be
                             a device running an image on QEMU.
                             You must also set
                             <link linkend='var-TEST_TARGET_IP'><filename>TEST_TARGET_IP</filename></link>
-                            when you use "simpleremote".
+                            when you use "simpleremote" or "SimpleRemoteTarget".
                             <note>
                                 This argument is defined in
                                 <filename>meta/lib/oeqa/targetcontrol.py</filename>.
@@ -7747,9 +7743,6 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                                 reasons.
                             </note>
                             </para></listitem>
-                        <listitem><para><emphasis>"SimpleRemoteTarget":</emphasis>
-                            Identical to "simpleremote".
-                            </para></listitem>
                         <listitem><para><emphasis>"GummibootTarget":</emphasis>
                             Automatically deploys and runs tests on an
                             EFI-enabled machine that has a master image
@@ -7761,6 +7754,12 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
                             </para></listitem>
                     </itemizedlist>
                 </para>
+
+                <para>
+                    For information on running tests on hardware, see the
+                    "<ulink url='&YOCTO_DOCS_DEV_URL;#hardware-image-enabling-tests'>Enabling Runtime Tests on Hardware</ulink>"
+                    section in the Yocto Project Development Manual.
+                </para>
             </glossdef>
         </glossentry>