testexport: support for executing tests over serial

Uses TEST_SERIALCONTROL_CMD to open a serial connection to the target
and execute commands. This is a drop in replacement for the ssh target,
fully supporting the same API. Supported with testexport.

To use, set the following in local.conf:
- TEST_TARGET to "serial"
- TEST_SERIALCONTROL_CMD to a shell command or script which connects to
  the serial console of the target and forwards that connection to
  standard input/output.
- TEST_SERIALCONTROL_EXTRA_ARGS (optional) any parameters that must be
  passed to the serial control command.
- TEST_SERIALCONTROL_PS1 (optional) A regex string representing an empty
  prompt on the target terminal. Example: "root@target:.*# ". This is
  used to find an empty shell after each command is run. This field is
  optional and will default to "root@{MACHINE}:.*# " if no other value is
  given.
- TEST_SERIALCONTROL_CONNECT_TIMEOUT (optional) Specifies the timeout in
  seconds for the initial connection to the target. Defaults to 10 if no
  other value is given.

The serial target does have some additional limitations over the ssh
target.
1. Only supports one "run" command at a time. If two threads attempt to
   call "run", one will block until it finishes. This is a limitation of
   the serial link, since two connections cannot be opened at once.
2. For file transfer, the target needs a shell and the base32 program.
   The file transfer implementation was chosen to be as generic as
   possible, so it could support as many targets as possible.
3. Transferring files is significantly slower. On a 115200 baud serial
   connection, the fastest observed speed was 30kbps. This is due to
   overhead in the implementation due to decisions documented in #2
   above.

(From OE-Core rev: d817b27d73d29ba2beffa2e0a4e31a14dbe0f1bf)

Signed-off-by: Andrew Oppelt <andrew.j.oppelt@boeing.com>
Signed-off-by: Matthew Weber <matthew.l.weber3@boeing.com>
Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com>

--

Tested with core-image-sato on real hardware. TEST_SERIALCONTROL_CMD
was set to a bash script which connected with telnet to the target.

Additionally tested with QEMU by setting TEST_SERIALCONTROL_CMD to
"ssh -o StrictHostKeyChecking=no root@192.168.7.2". This imitates
a serial connection to the QEMU instance.

Steps:
1) Set the following in local.conf:
  - IMAGE_CLASSES += "testexport"
  - TEST_TARGET = "serial"
  - TEST_SERIALCONTROL_CMD="ssh -o StrictHostKeyChecking=no root@192.168.7.2"
2) Build an image
  - bitbake core-image-sato
3) Run the test export
  - bitbake -c testexport core-image-sato
4) Run the image in qemu
  - runqemu nographic core-image-sato
5) Navigate to the test export directory
6) Run the exported tests with target-type set to serial
 - ./oe-test runtime --test-data-file ./data/testdata.json --packages-manifest ./data/manifest --debug --target-type serial

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 2 insertions, 0 deletions

diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index e912e91265..3f130120c0 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -429,7 +429,9 @@ TEST_SUITES[doc] = "An ordered list of tests (modules) to run against an image w
 TEST_POWERCONTROL_CMD[doc] = "For automated hardware testing, specifies the command to use to control the power of the target machine under test"
 TEST_POWERCONTROL_EXTRA_ARGS[doc] = "For automated hardware testing, specifies additional arguments to pass through to the command specified in TEST_POWERCONTROL_CMD"
 TEST_SERIALCONTROL_CMD[doc] = "For automated hardware testing, specifies the command to use to connect to the serial console of the target machine under test"
+TEST_SERIALCONTROL_CONNECT_TIMEOUT[doc] = "For automated hardware testing, specifies the timeout in seconds for the initial connection to the target. Defaults to '10'."
 TEST_SERIALCONTROL_EXTRA_ARGS[doc] = "For automated hardware testing, specifies additional arguments to pass through to the command specified in TEST_SERIALCONTROL_CMD"
+TEST_SERIALCONTROL_PS1[doc] = "For automated hardware testing, specifies a regex string representing an empty prompt on the target terminal. Example: 'root@target:.*#'. Defaults to 'root@${MACHINE}:.*#'."
 TEST_TARGET[doc] = "For automated runtime testing, specifies the method of deploying the image and running tests on the target machine"
 THISDIR[doc] = "The directory in which the file BitBake is currently parsing is located."
 TIME[doc] = "The time the build was started using HMS format."