bitbake: toaster: tts: execute tests in numeric order

As the tests are verifying different type of the functionality,
it is usually the case that a failing early test will completely
make the subsequent tests failing, e.g. if the system cannot
start due to a bug, there is little point in testing other
functions.

In order to prevent uneeded test runs, and to generate repeatable
test patterns, the test cases have now a numeric order in the
class name (e.g. Test01XXX). The tests are executed in this order,
and the first test failing will stop the test run.

(Bitbake rev: 639c46a08e524902018e28367fcb4e26362cd3e3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 3 insertions, 3 deletions

diff --git a/bitbake/lib/toaster/contrib/tts/tests.py b/bitbake/lib/toaster/contrib/tts/tests.py
index 9acef46a38..3a4eed1e9f 100644
--- a/bitbake/lib/toaster/contrib/tts/tests.py
+++ b/bitbake/lib/toaster/contrib/tts/tests.py
@@ -30,7 +30,7 @@ import config
 import pexpect
 import sys, os, signal, time
 
-class TestPyCompilable(unittest.TestCase):
+class Test00PyCompilable(unittest.TestCase):
     ''' Verifies that all Python files are syntactically correct '''
     def test_compile_file(self):
         try:
@@ -38,7 +38,7 @@ class TestPyCompilable(unittest.TestCase):
         except ShellCmdException as exc:
             self.fail("Error compiling python files: %s" % (exc))
 
-class TestPySystemStart(unittest.TestCase):
+class Test01PySystemStart(unittest.TestCase):
     ''' Attempts to start Toaster, verify that it is succesfull, and stop it '''
     def setUp(self):
         run_shell_cmd("bash -c 'rm -f build/*log'")
@@ -55,7 +55,7 @@ class TestPySystemStart(unittest.TestCase):
         except ShellCmdException as exc:
             self.fail("Failed starting managed mode: %s" % (exc))
 
-class TestHTML5Compliance(unittest.TestCase):
+class Test02HTML5Compliance(unittest.TestCase):
     def setUp(self):
         self.origdir = os.getcwd()
         self.crtdir = os.path.dirname(config.TESTDIR)