diff options
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.xml')
-rw-r--r-- | documentation/dev-manual/dev-manual-common-tasks.xml | 414 |
1 files changed, 414 insertions, 0 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index 7f59c39f2d..2dbc3d14b8 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml | |||
@@ -4252,6 +4252,420 @@ | |||
4252 | </section> | 4252 | </section> |
4253 | </section> | 4253 | </section> |
4254 | 4254 | ||
4255 | <section id="running-and-writing-tests-for-a-qemu-image"> | ||
4256 | <title>Running and Writing Tests for a QEMU Image</title> | ||
4257 | |||
4258 | <para> | ||
4259 | The OpenEmbedded build system makes available a series of automated | ||
4260 | tests for QEMU images. | ||
4261 | These tests are commands that run on the target system over | ||
4262 | <filename>ssh</filename> and are written in Python, thus making | ||
4263 | use of the <filename>unittest</filename> module. | ||
4264 | </para> | ||
4265 | |||
4266 | <para> | ||
4267 | This section describes how you set up the environment to use these | ||
4268 | tests, run available tests, and write and add your own tests. | ||
4269 | You can the same information in a different form and with example | ||
4270 | log files of test runs on the | ||
4271 | <ulink url='https://wiki.yoctoproject.org/wiki/Image_tests'>Image Tests</ulink> | ||
4272 | Wiki page. | ||
4273 | </para> | ||
4274 | |||
4275 | <section id="qemu-image-enabling-tests"> | ||
4276 | <title>Enabling Tests</title> | ||
4277 | |||
4278 | <para> | ||
4279 | In order to run tests, you need to do the following: | ||
4280 | <itemizedlist> | ||
4281 | <listitem><para><emphasis>Ensure you run the test | ||
4282 | automatically with no interaction and under | ||
4283 | <filename>sudo</filename>:</emphasis> | ||
4284 | To accomplish this, you must do one of the | ||
4285 | following: | ||
4286 | <itemizedlist> | ||
4287 | <listitem><para>Add | ||
4288 | <filename>NOPASSWD</filename> for your user | ||
4289 | in <filename>/etc/sudoers</filename> either for | ||
4290 | ALL commands or just for | ||
4291 | <filename>runqemu-ifup</filename>. | ||
4292 | You must provide the full path as that can | ||
4293 | change if you are using multiple clones of the | ||
4294 | source repository. | ||
4295 | <note> | ||
4296 | On some distributions, you also need to | ||
4297 | comment out "Defaults requiretty" in | ||
4298 | <filename>/etc/sudoers</filename>. | ||
4299 | </note></para></listitem> | ||
4300 | <listitem><para>Manually configure a tap interface | ||
4301 | for your system.</para></listitem> | ||
4302 | <listitem><para>Run as root the script in | ||
4303 | <filename>scripts/runqemu-gen-tapdev</filename>, | ||
4304 | which should generate a list of tap devices. | ||
4305 | List generation is usually done in environments | ||
4306 | similar to AutoBuilder.</para></listitem> | ||
4307 | </itemizedlist></para></listitem> | ||
4308 | <listitem><para><emphasis>Set up the | ||
4309 | <filename>DISPLAY</filename> variable:</emphasis> | ||
4310 | You need to set this variable so that you have an X | ||
4311 | server available (e.g. start | ||
4312 | <filename>vncserver</filename> for a headless machine). | ||
4313 | </para></listitem> | ||
4314 | <listitem><para><emphasis>Be sure your host's firewall | ||
4315 | accepts incoming connections from | ||
4316 | 192.168.7.0/24:</emphasis> | ||
4317 | Some of the tests (in particular smart tests) start a | ||
4318 | HTTP server on a random high number port, which is | ||
4319 | used to serve files to the target. | ||
4320 | The smart module serves | ||
4321 | <filename>${DEPLOY_DIR}/rpm</filename> so it can run | ||
4322 | smart channel commands. That means your host's firewall | ||
4323 | must accept incoming connections from 192.168.7.0/24, | ||
4324 | which is the default class used for | ||
4325 | <filename>tap0</filename> devices by | ||
4326 | <filename>runqemu</filename>.</para></listitem> | ||
4327 | <listitem><para><emphasis>Have your QEMU image built:</emphasis> | ||
4328 | The QEMU image on which you want to run tests needs to | ||
4329 | be built. | ||
4330 | For example, the following command would build the | ||
4331 | <filename>core-image-sato</filename> image when | ||
4332 | <filename>MACHINE</filename> is set to "qemu". | ||
4333 | <literallayout class='monospaced'> | ||
4334 | bitbake core-image-sato | ||
4335 | </literallayout></para></listitem> | ||
4336 | <listitem><para><emphasis>Globally inherit | ||
4337 | <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink></emphasis> | ||
4338 | Edit your <filename>local.conf</filename> file as | ||
4339 | follows: | ||
4340 | <literallayout class='monospaced'> | ||
4341 | INHERIT += "testimage" | ||
4342 | </literallayout></para></listitem> | ||
4343 | </itemizedlist> | ||
4344 | </para> | ||
4345 | |||
4346 | <note> | ||
4347 | If you built your QEMU image using <filename>rm_work</filename>, | ||
4348 | most of the tests will fail with errors because they rely on | ||
4349 | <filename>${WORKDIR}/installed_pkgs.txt</filename>. | ||
4350 | </note> | ||
4351 | </section> | ||
4352 | |||
4353 | <section id="qemu-image-running-tests"> | ||
4354 | <title>Running Tests</title> | ||
4355 | |||
4356 | <para> | ||
4357 | After setting up to run the tests, use BitBake to start | ||
4358 | the standard suite of tests: | ||
4359 | <literallayout class='monospaced'> | ||
4360 | bitbake core-image-sato -c testimage | ||
4361 | </literallayout> | ||
4362 | </para> | ||
4363 | |||
4364 | <para> | ||
4365 | Once you start the tests, the following happens: | ||
4366 | <itemizedlist> | ||
4367 | <listitem><para>A copy of the root filesystem is written | ||
4368 | to <filename>${WORKDIR}/testimage</filename>. | ||
4369 | </para></listitem> | ||
4370 | <listitem><para>The image is booted under QEMU using the | ||
4371 | standard <filename>runqemu</filename> script. | ||
4372 | </para></listitem> | ||
4373 | <listitem><para>A default timeout of 500 seconds occurs | ||
4374 | to allow for the boot process to reach the login prompt. | ||
4375 | You can change the timeout period by setting | ||
4376 | <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink> | ||
4377 | in the <filename>local.conf</filename> file. | ||
4378 | </para></listitem> | ||
4379 | <listitem><para>Once the boot process is reached and the | ||
4380 | login prompt appears, the tests run. | ||
4381 | The full boot log is written to | ||
4382 | <filename>${WORKDIR}/testimage/qemu_boot_log</filename>. | ||
4383 | </para></listitem> | ||
4384 | <listitem><para>Each test module loads in the order found | ||
4385 | in <filename>TEST_SUITES</filename>. | ||
4386 | You can find the full output of the commands run over | ||
4387 | <filename>ssh</filename> in | ||
4388 | <filename>${WORKDIR}/testimgage/ssh_target_log</filename>. | ||
4389 | </para></listitem> | ||
4390 | <listitem><para>If no failures occur, the task running the | ||
4391 | tests ends successfully. | ||
4392 | You can find the output from the | ||
4393 | <filename>unittest</filename> in the task log at | ||
4394 | <filename>${WORKDIR}/temp/log.do_testimage</filename>. | ||
4395 | </para></listitem> | ||
4396 | </itemizedlist> | ||
4397 | </para> | ||
4398 | |||
4399 | <para> | ||
4400 | All test files reside in | ||
4401 | <filename>meta/lib/oeqa/runtime</filename> in the | ||
4402 | <link linkend='source-directory'>Source Directory</link>. | ||
4403 | Tests (also referred to as modules) have a one-to-one | ||
4404 | relationship with the filenames in <filename>runtime</filename>. | ||
4405 | Modules can have multiple classes and test methods and are | ||
4406 | usually grouped together by the area tested (e.g tests for | ||
4407 | <filename>systemd</filename> reside in | ||
4408 | <filename>meta/lib/oeqa/runtime/systemd.py</filename>). | ||
4409 | </para> | ||
4410 | |||
4411 | <para> | ||
4412 | You can add tests to any layer provided you place them in the | ||
4413 | proper area and you extend | ||
4414 | <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> | ||
4415 | in the <filename>local.conf</filename> file. | ||
4416 | Be sure that tests reside in | ||
4417 | <filename><layer>/lib/oeqa/runtime</filename>. | ||
4418 | <note> | ||
4419 | Be sure that module names do not collide with module names | ||
4420 | provided by the Yocto Project. | ||
4421 | </note> | ||
4422 | </para> | ||
4423 | |||
4424 | <para> | ||
4425 | You can change the set of tests run by appending or overriding | ||
4426 | <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink> | ||
4427 | variable in <filename>local.conf</filename>. | ||
4428 | Each name in <filename>TEST_SUITES</filename> represents a | ||
4429 | required test for the image. | ||
4430 | Module skipping is not allowed even if a test is not suitable | ||
4431 | for an image (e.g. running the <filename>rpm</filename> tests on | ||
4432 | an image without <filename>rpm</filename>). | ||
4433 | Appending "auto" to <filename>TEST_SUITES</filename> causes the | ||
4434 | build system to try to run all tests that are suitable for the | ||
4435 | image (i.e. the build system decides whether to run each test). | ||
4436 | </para> | ||
4437 | |||
4438 | <para> | ||
4439 | The order you list tests in <filename>TEST_SUITES</filename> | ||
4440 | is important. | ||
4441 | The order influences test dependencies. | ||
4442 | Consequently, tests that depend on other tests should be added | ||
4443 | after the test on which they depend. | ||
4444 | For example, since <filename>ssh</filename> depends on the | ||
4445 | <filename>ping</filename> test, <filename>ssh</filename> | ||
4446 | needs to come after <filename>ping</filename> in the list. | ||
4447 | The test class provides no re-ordering or dependency handling. | ||
4448 | <note> | ||
4449 | Each module can have multiple classes with multiple test | ||
4450 | methods. | ||
4451 | And, Python <filename>unittest</filename> rules apply. | ||
4452 | </note> | ||
4453 | </para> | ||
4454 | |||
4455 | <para> | ||
4456 | The following list summarizes how to run the tests: | ||
4457 | <itemizedlist> | ||
4458 | <listitem><para>Run the default set of tests simply by | ||
4459 | running the following: | ||
4460 | <literallayout class='monospaced'> | ||
4461 | bitbake <qemu_image> -c testimage | ||
4462 | </literallayout></para></listitem> | ||
4463 | <listitem><para>The default tests for the image are defined | ||
4464 | as: | ||
4465 | <literallayout class='monospaced'> | ||
4466 | DEFAULT_TEST_SUITES_pn-<qemu_image> = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg" | ||
4467 | </literallayout></para></listitem> | ||
4468 | <listitem><para>Add your own test to the list of the | ||
4469 | by using the following: | ||
4470 | <literallayout class='monospaced'> | ||
4471 | TEST_SUITES_append = " mytest" | ||
4472 | </literallayout></para></listitem> | ||
4473 | <listitem><para>Run a specific list of tests as follows: | ||
4474 | <literallayout class='monospaced'> | ||
4475 | TEST_SUITES = "test1 test2 test3" | ||
4476 | </literallayout> | ||
4477 | Remember, order is important. | ||
4478 | Be sure to place a test that is dependent on another test | ||
4479 | later in the order.</para></listitem> | ||
4480 | </itemizedlist> | ||
4481 | </para> | ||
4482 | </section> | ||
4483 | |||
4484 | <section id="qemu-image-writing-new-tests"> | ||
4485 | <title>Writing New Tests</title> | ||
4486 | |||
4487 | <para> | ||
4488 | As mentioned previously, all new test files need to be in the | ||
4489 | proper place for the build system to find them. | ||
4490 | New tests can go in the | ||
4491 | <link linkend='source-directory'>Source Directory</link> at | ||
4492 | <filename>meta/lib/oeqa/runtime</filename>. | ||
4493 | Alternatively, a layer can add its own tests in | ||
4494 | <filename><layer>/lib/oeqa/runtime</filename> as long | ||
4495 | as you extend <filename>BBPATH</filename>. | ||
4496 | Just remember that filenames need to map directory to test | ||
4497 | (module) names and you not use module names that collide with | ||
4498 | existing core tests. | ||
4499 | </para> | ||
4500 | |||
4501 | <para> | ||
4502 | To create a new test, start by copying an existing module | ||
4503 | (e.g. <filename>syslog.py</filename> or | ||
4504 | <filename>gcc.py</filename> are good ones to use). | ||
4505 | Test modules can use code from | ||
4506 | <filename>meta/lib/oeqa/utils</filename>, which are helper | ||
4507 | classes. | ||
4508 | </para> | ||
4509 | |||
4510 | <note> | ||
4511 | Structure shell commands such that you rely on them and they | ||
4512 | return a single code for success. | ||
4513 | Be aware that sometimes you will need to parse the output. | ||
4514 | See the <filename>df.py</filename> and | ||
4515 | <filename>date.py</filename> modules for examples. | ||
4516 | </note> | ||
4517 | |||
4518 | <para> | ||
4519 | You will notice that all test classes inherit | ||
4520 | <filename>oeRuntimeTest</filename>, which is found in | ||
4521 | <filename>meta/lib/oetest.py</filename>. | ||
4522 | This base class offers some helper attributes, which are | ||
4523 | described in the following sections: | ||
4524 | </para> | ||
4525 | |||
4526 | <section id='qemu-image-writing-tests-class-methods'> | ||
4527 | <title>Class Methods</title> | ||
4528 | |||
4529 | <para> | ||
4530 | Class methods are as follows: | ||
4531 | <itemizedlist> | ||
4532 | <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis> | ||
4533 | Returns "True" if <filename>pkg</filename> is in the | ||
4534 | installed package list of the image, which is based | ||
4535 | on | ||
4536 | <filename>${WORKDIR}/installed_pkgs.txt</filename> | ||
4537 | that is generated during the | ||
4538 | <filename>do.rootfs</filename> task. | ||
4539 | </para></listitem> | ||
4540 | <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis> | ||
4541 | Returns "True" if the feature is in | ||
4542 | <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> | ||
4543 | or | ||
4544 | <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>. | ||
4545 | </para></listitem> | ||
4546 | <listitem><para><emphasis><filename>restartTarget(params)</filename>:</emphasis> | ||
4547 | Restarts the QEMU image optionally passing | ||
4548 | <filename>params</filename> to the | ||
4549 | <filename>runqemu</filename> script's | ||
4550 | <filename>qemuparams</filename> list (e.g "-m 1024" for | ||
4551 | more memory).</para></listitem> | ||
4552 | </itemizedlist> | ||
4553 | </para> | ||
4554 | </section> | ||
4555 | |||
4556 | <section id='qemu-image-writing-tests-class-attributes'> | ||
4557 | <title>Class Attributes</title> | ||
4558 | |||
4559 | <para> | ||
4560 | Class attributes are as follows: | ||
4561 | <itemizedlist> | ||
4562 | <listitem><para><emphasis><filename>pscmd</filename>:</emphasis> | ||
4563 | Equals "ps -ef" if <filename>procps</filename> is | ||
4564 | installed in the image. | ||
4565 | Otherwise, <filename>pscmd</filename> equals | ||
4566 | "ps" (busybox). | ||
4567 | </para></listitem> | ||
4568 | <listitem><para><emphasis><filename>tc</filename>:</emphasis> | ||
4569 | The called text context, which gives access to the | ||
4570 | following attributes: | ||
4571 | <itemizedlist> | ||
4572 | <listitem><para><emphasis><filename>d</filename>:</emphasis> | ||
4573 | The BitBake data store, which allows you to | ||
4574 | use stuff such as | ||
4575 | <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>. | ||
4576 | </para></listitem> | ||
4577 | <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis> | ||
4578 | Used internally. | ||
4579 | The tests do not need these. | ||
4580 | </para></listitem> | ||
4581 | <listitem><para><emphasis><filename>filesdir</filename>:</emphasis> | ||
4582 | The absolute path to | ||
4583 | <filename>meta/lib/oeqa/runtime/files</filename>, | ||
4584 | which contains helper files for tests meant | ||
4585 | for copying on the target such as small | ||
4586 | files written in C for compilation. | ||
4587 | </para></listitem> | ||
4588 | <listitem><para><emphasis><filename>qemu</filename>:</emphasis> | ||
4589 | Provides access to the | ||
4590 | <filename>QemuRunner</filename> object, | ||
4591 | which is the class that boots the image. | ||
4592 | The <filename>qemu</filename> attribute | ||
4593 | provides the following useful attributes: | ||
4594 | <itemizedlist> | ||
4595 | <listitem><para><emphasis><filename>ip</filename>:</emphasis> | ||
4596 | The machine's IP address. | ||
4597 | </para></listitem> | ||
4598 | <listitem><para><emphasis><filename>host_ip</filename>:</emphasis> | ||
4599 | The host IP address, which is only | ||
4600 | used by smart tests. | ||
4601 | </para></listitem> | ||
4602 | </itemizedlist></para></listitem> | ||
4603 | <listitem><para><emphasis><filename>target</filename>:</emphasis> | ||
4604 | The <filename>SSHControl</filename> object, | ||
4605 | which is used for running the following | ||
4606 | commands on the image: | ||
4607 | <itemizedlist> | ||
4608 | <listitem><para><emphasis><filename>host</filename>:</emphasis> | ||
4609 | Used internally. | ||
4610 | The tests do not use this command. | ||
4611 | </para></listitem> | ||
4612 | <listitem><para><emphasis><filename>timeout</filename>:</emphasis> | ||
4613 | A global timeout for commands run on | ||
4614 | the target for the instance of a | ||
4615 | test. | ||
4616 | The default is 300 seconds. | ||
4617 | </para></listitem> | ||
4618 | <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis> | ||
4619 | The single, most used method. | ||
4620 | This command is a wrapper for: | ||
4621 | <filename>ssh root@host "cmd"</filename>. | ||
4622 | The command returns a tuple: | ||
4623 | (status, output), which are what | ||
4624 | their names imply - the return code | ||
4625 | of 'cmd' and whatever output | ||
4626 | it produces. | ||
4627 | The optional timeout argument | ||
4628 | represents the number of seconds the | ||
4629 | test should wait for 'cmd' to | ||
4630 | return. | ||
4631 | If the argument is "None", the | ||
4632 | test uses the default instance's | ||
4633 | timeout period, which is 300 | ||
4634 | seconds. | ||
4635 | If the argument is "0", the test | ||
4636 | runs until the command returns. | ||
4637 | </para></listitem> | ||
4638 | <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis> | ||
4639 | <filename>scp localpath root@ip:remotepath</filename>. | ||
4640 | </para></listitem> | ||
4641 | <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis> | ||
4642 | <filename>scp root@host:remotepath localpath</filename>. | ||
4643 | </para></listitem> | ||
4644 | </itemizedlist></para></listitem> | ||
4645 | </itemizedlist></para></listitem> | ||
4646 | </itemizedlist> | ||
4647 | </para> | ||
4648 | </section> | ||
4649 | |||
4650 | <section id='qemu-image-writing-tests-instance-attributes'> | ||
4651 | <title>Instance Attributes</title> | ||
4652 | |||
4653 | <para> | ||
4654 | A single instance attribute exists, which is | ||
4655 | <filename>target</filename>. | ||
4656 | The <filename>target</filename> instance attribute is | ||
4657 | identical to the class attribute of the same name, which | ||
4658 | is described in the previous section. | ||
4659 | This attribute exists as both an instance and class | ||
4660 | attribute so tests can use | ||
4661 | <filename>self.target.run(cmd)</filename> in instance | ||
4662 | methods instead of | ||
4663 | <filename>oeRuntimeTest.tc.target.run(cmd)</filename>. | ||
4664 | </para> | ||
4665 | </section> | ||
4666 | </section> | ||
4667 | </section> | ||
4668 | |||
4255 | <section id="platdev-gdb-remotedebug"> | 4669 | <section id="platdev-gdb-remotedebug"> |
4256 | <title>Debugging With the GNU Project Debugger (GDB) Remotely</title> | 4670 | <title>Debugging With the GNU Project Debugger (GDB) Remotely</title> |
4257 | 4671 | ||