summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime/context.py
Commit message (Collapse)AuthorAgeFilesLines
* testexport: support for executing tests over serialAndrew Oppelt2024-09-011-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Uses TEST_SERIALCONTROL_CMD to open a serial connection to the target and execute commands. This is a drop in replacement for the ssh target, fully supporting the same API. Supported with testexport. To use, set the following in local.conf: - TEST_TARGET to "serial" - TEST_SERIALCONTROL_CMD to a shell command or script which connects to the serial console of the target and forwards that connection to standard input/output. - TEST_SERIALCONTROL_EXTRA_ARGS (optional) any parameters that must be passed to the serial control command. - TEST_SERIALCONTROL_PS1 (optional) A regex string representing an empty prompt on the target terminal. Example: "root@target:.*# ". This is used to find an empty shell after each command is run. This field is optional and will default to "root@{MACHINE}:.*# " if no other value is given. - TEST_SERIALCONTROL_CONNECT_TIMEOUT (optional) Specifies the timeout in seconds for the initial connection to the target. Defaults to 10 if no other value is given. The serial target does have some additional limitations over the ssh target. 1. Only supports one "run" command at a time. If two threads attempt to call "run", one will block until it finishes. This is a limitation of the serial link, since two connections cannot be opened at once. 2. For file transfer, the target needs a shell and the base32 program. The file transfer implementation was chosen to be as generic as possible, so it could support as many targets as possible. 3. Transferring files is significantly slower. On a 115200 baud serial connection, the fastest observed speed was 30kbps. This is due to overhead in the implementation due to decisions documented in #2 above. (From OE-Core rev: d817b27d73d29ba2beffa2e0a4e31a14dbe0f1bf) Signed-off-by: Andrew Oppelt <andrew.j.oppelt@boeing.com> Signed-off-by: Matthew Weber <matthew.l.weber3@boeing.com> Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com> -- Tested with core-image-sato on real hardware. TEST_SERIALCONTROL_CMD was set to a bash script which connected with telnet to the target. Additionally tested with QEMU by setting TEST_SERIALCONTROL_CMD to "ssh -o StrictHostKeyChecking=no root@192.168.7.2". This imitates a serial connection to the QEMU instance. Steps: 1) Set the following in local.conf: - IMAGE_CLASSES += "testexport" - TEST_TARGET = "serial" - TEST_SERIALCONTROL_CMD="ssh -o StrictHostKeyChecking=no root@192.168.7.2" 2) Build an image - bitbake core-image-sato 3) Run the test export - bitbake -c testexport core-image-sato 4) Run the image in qemu - runqemu nographic core-image-sato 5) Navigate to the test export directory 6) Run the exported tests with target-type set to serial - ./oe-test runtime --test-data-file ./data/testdata.json --packages-manifest ./data/manifest --debug --target-type serial Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage/oeqa: Drop testimage_dump_host functionalityThomas Roos2023-06-291-10/+1
| | | | | | | | | | | | | | | | | The intent behind these functions was to dump the system state when issues occured but it has never really worked as we'd planned. Regular monitoring as the build runs has largely replaced this as that allows a trend to be seen rather than a spot value which was never really useful. The code is bitrotting and not functioning correctly so drop it. [YOCTO #13872] RP: Reword commit message (From OE-Core rev: dea37ba49a236029da73d5cfbfc069bffc38b508) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa context.py: fix --target-ip comment to include ssh port numberMikko Rapeli2023-02-081-2/+2
| | | | | | | | | | Providing ssh port number is supported too with "--target-ip 192.168.0.10:22". (From OE-Core rev: 637919b9df0abc06da5b2f9b389cf25376bd6b7c) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context: remove duplicate sys.path entries when looking for modulesRoss Burton2022-03-261-1/+5
| | | | | | | | | | | | | | | sys.path can contain duplicate entries for each layer, which means that the search in add_controller_list() will find the same name twice and abort. As duplicate directories should be harmless, remove any duplicates before iterating through the entries. (From OE-Core rev: e478381ac1cccc5f882198fd11c8757db7e3741a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: search sys.path explicitly for modulesRoss Burton2021-10-231-8/+7
| | | | | | | | | | | | | | | | | | | The controller module loading code needs to be told what directories to search for modules via the target_modules_path keyword argument, which is set to BBPATH. However, as the actual module loading is done via importlib this relies on the paths being on sys.path, which it is as base.bbclass puts each layer's lib/ in sys.path. Simplify the code by removing this indirection, and simply search sys.path directly. (From OE-Core rev: 570a19581f582f77e04d6892adb647cd649a6943) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: load modules using importlibRoss Burton2021-10-231-9/+5
| | | | | | | | | | | Instead of using __import__() which is low-level and discouraged, use importlib. (From OE-Core rev: 9f501d22eab5dbd565f3f5783f4f484a6d1f70a2) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: default runtime json results for testexportStefan Kral2020-03-291-0/+1
| | | | | | | | | | | | | Set the json result output dir in the oeqe runtime context to create testresults.json file by default for exported runtime test runs. Use current datetime for the json result property name (not DATETIME from build) to allow multiple result entries. (From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f) Signed-off-by: Stefan Kral <sk@typedivision.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: fix typoStefan Kral2020-03-121-2/+2
| | | | | | | (From OE-Core rev: bf959e6fd175d0841a1c042bf925c54200bea3d4) Signed-off-by: Stefan Kral <sk@typedivision.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: support listening port in TEST_SERVER_IPAndré Draszik2019-12-161-0/+6
| | | | | | | | | | | | | | | | | Similar to the existing possibility to specify a port in TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a port. The intention is for this port to be passed into e.g. the http server from the apt / dnf / opkg tests, or any other (custom) tests that might need the target to connect to a service spawned by bitbake / oeqa, where bitbake is actually running inside a docker container. (From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: ignore more files when loading controllersAndré Draszik2019-10-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When loading controllers as (external) modules, the code currently tries to load all files ending with .py. This is a problem when during development using an editor that creates a lock-file in the same directory as the .py file, as the lock file is typically called '.#xxxx.py'. Python will try to load the lock file and fail miserably with an exception: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_testimage(d) 0003: File: 'poky/meta/classes/testimage.bbclass', lineno: 114, function: do_testimage 0110: netstat -an 0111:} 0112: 0113:python do_testimage() { *** 0114: testimage_main(d) 0115:} 0116: 0117:addtask testimage 0118:do_testimage[nostamp] = "1" File: 'poky/meta/classes/testimage.bbclass', lineno: 294, function: testimage_main 0290: 0291: # the robot dance 0292: target = OERuntimeTestContextExecutor.getTarget( 0293: d.getVar("TEST_TARGET"), logger, d.getVar("TEST_TARGET_IP"), *** 0294: d.getVar("TEST_SERVER_IP"), **target_kwargs) 0295: 0296: # test context 0297: tc = OERuntimeTestContext(td, logger, target, host_dumper, 0298: image_packages, extract_dir) File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 116, function: getTarget 0112: # XXX: Don't base your targets on this code it will be refactored 0113: # in the near future. 0114: # Custom target module loading 0115: target_modules_path = kwargs.get('target_modules_path', '') *** 0116: controller = OERuntimeTestContextExecutor.getControllerModule(target_type, target_modules_path) 0117: target = controller(logger, target_ip, server_ip, **kwargs) 0118: 0119: return target 0120: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 128, function: getControllerModule 0124: # ImportError raised if a provided module can not be imported. 0125: @staticmethod 0126: def getControllerModule(target, target_modules_path): 0127: controllerslist = OERuntimeTestContextExecutor._getControllerModulenames(target_modules_path) *** 0128: controller = OERuntimeTestContextExecutor._loadControllerFromName(target, controllerslist) 0129: return controller 0130: 0131: # Return a list of all python modules in lib/oeqa/controllers for each 0132: # layer in bbpath File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 163, function: _loadControllerFromName 0159: # Raise ImportError if a provided module can not be imported 0160: @staticmethod 0161: def _loadControllerFromName(target, modulenames): 0162: for name in modulenames: *** 0163: obj = OERuntimeTestContextExecutor._loadControllerFromModule(target, name) 0164: if obj: 0165: return obj 0166: raise AttributeError("Unable to load {0} from available modules: {1}".format(target, str(modulenames))) 0167: File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 173, function: _loadControllerFromModule 0169: @staticmethod 0170: def _loadControllerFromModule(target, modulename): 0171: obj = None 0172: # import module, allowing it to raise import exception *** 0173: module = __import__(modulename, globals(), locals(), [target]) 0174: # look for target class in the module, catching any exceptions as it 0175: # is valid that a module may not have the target class. 0176: try: 0177: obj = getattr(module, target) Exception: ImportError: No module named 'oeqa.controllers.' Simply ignore those when collecting the list of files to try to load. (From OE-Core rev: 682f223cf2e2dabe8cf60634b6779bb2d5e359bd) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-091-1/+4
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: make it work for multiple usersRobert Yang2019-01-111-4/+1
| | | | | | | | | | | | | | | | | | | | | There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. (From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/qemu & runtime: qemu do not need ip input from externalYeoh Ee Peng2018-11-231-1/+1
| | | | | | | | | | | Qemu do not use the ip input from external. It will retrieve ip from QemuRunner instance and assign ip value. (From OE-Core rev: 14d99dc6c39c963ba3e0d9a30274846bd5369210) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* context.py: Do not mask exceptions needlessly.Paulo Neves2018-08-081-10/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | There were a lot of assumptions in the controller import code of context.py which were not true anymore. These assumptions reflected themselves by catching exceptions and re-raising them with unhelpful error messages in other parts of the code. This commit does not fix the classes controller classes that became broken after the refactor but at least it allows for the exceptions to be thrown where the imports fail, thus actually showing what exactly went wrong with the import. An example of such an improvement is that before if the controller class failed during it's init contructor the controller would just be skipped and the task would just complain it could not find the controller. Now for example, if there is a NamerError due to a variable not being declared, the user will get that report. (From OE-Core rev: 719b0e88b875b067551067874778061f99b8c4b4) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: Add support to specify port in target_ipAníbal Limón2017-06-061-0/+6
| | | | | | | (From OE-Core rev: 7fe4c074611eb21abdf811d8f216d4df7ab6a3ea) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: Fix use of getTarget() with testexportMariano Lopez2017-02-231-1/+10
| | | | | | | | | | | | | | | | The idea on getTarget is to use kwargs to send custom variables to different targets, instead of this, a new variable was added (just used for custom targets) and this broke testexport. So in order to fix it, just add the custom variable to kwargs. This fixes the use of getTarget() in testexport class that was introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf. (From OE-Core rev: cf138029a1f18f991fc7a28c81d85884942e9d56) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage: Implement custom target loadingJuro Bystricky2017-02-151-3/+71
| | | | | | | | | | | | | | This patch implements custom target loading for testimage, currently missing due to major changes to the test framework. Custom targets can be defined in various meta-layers, so we need an extra path information in order to find them. Any other additional info is retrieved as usual via the variables TEST_TARGET and TEST_SUITES (From OE-Core rev: 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: Add defaults for runtime contextMariano Lopez2017-01-231-9/+13
| | | | | | | | | | | | | This adds default values to OERuntimeTestContextExecutor class in order to make easier the execution of exported test that were generated with testexport class. [YOCTO #10686] (From OE-Core rev: c78aeaac3b75610bada62b138c9670815a07ee80) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* testimage.bbclass: Add package install featureMariano Lopez2017-01-231-1/+11
| | | | | | | | | | | | This allows to use the package install feature with the new OEQA framework. [YOCTO #10234] (From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: Add logger to getTargetMariano Lopez2017-01-231-3/+3
| | | | | | | | | | | | | Current targets (ssh and qemu) require a logger in their constructors, so in order to get a new target we need to provide the logger. [YOCTO #10686] (From OE-Core rev: 26bae3c255bc1e1cc8d81db0cffc809de0182a43) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context.py: Prepare for qemuAníbal Limón2017-01-231-5/+13
| | | | | | | | | | This commit will prepare context to use qemu and not just simpleremote. (From OE-Core rev: 7485a08c967916fb6edff4cc573d9314ec577031) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor classAníbal Limón2017-01-231-1/+17
| | | | | | | | | | | | To avoid getVar calls inside a utils module, also moves get_host_dumper import inside testexport isn't needed. [YOCTO #10231] (From OE-Core rev: f8beaf94e943a8b20d146be47a756af312ef107c) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context: Move helper functions for process args to executorAníbal Limón2017-01-231-20/+36
| | | | | | | (From OE-Core rev: 07ee2fa0008a50da87fd840aea5e8bb6051c68fa) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/context: Add runtime option group and options for target type ↵Aníbal Limón2017-01-231-2/+20
| | | | | | | | | and server ip. (From OE-Core rev: d3cc6761a13fd1da673829797b45323aed7997d0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/files: Move runtime files from old directoryMariano Lopez2017-01-231-0/+2
| | | | | | | | | | | | | As part of the refactor we require to move the files used in runtime testing to the new directory. This also adds the path to the runtime test context. [YOCTO #10234] (From OE-Core rev: f80c21b46f69c42ff0853bf8abd01cf0c082c346) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: Add case, context and loader classes for runtime testingMariano Lopez2017-01-231-0/+69
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader to be used for runtime testing. As expected there are some changes in runtime context: - Adds the target to be used for runtime testing, the default is a SSH connection to the device under test running a OE image. - Runtime context requires image manifest because several tests are skipped if a package is missing or installed. - Several tests require the output of the ps command and it changes its output and arguments if busybox o procps is installed, so the case must use the correct ps command. [YOCTO #10234] (From OE-Core rev: f995f178de79d6d11422cd879d06371811f50651) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-06-15 11:29:12 +0000