diff options
Diffstat (limited to 'bitbake/doc')
-rw-r--r-- | bitbake/doc/user-manual/user-manual-metadata.xml | 208 |
1 files changed, 8 insertions, 200 deletions
diff --git a/bitbake/doc/user-manual/user-manual-metadata.xml b/bitbake/doc/user-manual/user-manual-metadata.xml index 8d37f6bee1..cabf25fb6f 100644 --- a/bitbake/doc/user-manual/user-manual-metadata.xml +++ b/bitbake/doc/user-manual/user-manual-metadata.xml | |||
@@ -2,40 +2,15 @@ | |||
2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> | 2 | "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> |
3 | 3 | ||
4 | <chapter id="user-manual-metadata"> | 4 | <chapter id="user-manual-metadata"> |
5 | <title>Metadata</title> | 5 | <title>Syntax and Operators</title> |
6 | 6 | ||
7 | <section> | 7 | <para> |
8 | <title>Overview</title> | 8 | Bitbake files have their own syntax. |
9 | 9 | The syntax has similarities to several | |
10 | <para> | 10 | other languages but also has some unique features. |
11 | The BitBake task executor together with various types of configuration | 11 | This section describes the available syntax and operators |
12 | files form the OpenEmbedded Core. | 12 | as well as provides examples. |
13 | This section provides an overview of the task executor and the | 13 | </para> |
14 | configuration files by describing their use and interaction. | ||
15 | </para> | ||
16 | |||
17 | <para> | ||
18 | BitBake handles the parsing and execution of the data files. | ||
19 | The data itself is of various types: | ||
20 | <itemizedlist> | ||
21 | <listitem><para><emphasis>Recipes:</emphasis> | ||
22 | Details about particular pieces of software. | ||
23 | </para></listitem> | ||
24 | <listitem><para><emphasis>Class Data:</emphasis> | ||
25 | An abstraction of common build information | ||
26 | (e.g. how to build a Linux kernel). | ||
27 | </para></listitem> | ||
28 | <listitem><para><emphasis>Configuration Data:</emphasis> | ||
29 | Machine-specific settings, policy decisions, | ||
30 | and so forth. | ||
31 | Configuration data acts as the glue to bind everything | ||
32 | together.</para></listitem> | ||
33 | </itemizedlist> | ||
34 | The remainder of this chapter provides examples of BitBake metadata. | ||
35 | Any syntax not supported in any of the previously listed areas | ||
36 | is documented as such. | ||
37 | </para> | ||
38 | </section> | ||
39 | 14 | ||
40 | <section id='basic-syntax'> | 15 | <section id='basic-syntax'> |
41 | <title>Basic Syntax</title> | 16 | <title>Basic Syntax</title> |
@@ -1453,172 +1428,5 @@ | |||
1453 | </para></listitem> | 1428 | </para></listitem> |
1454 | </itemizedlist> | 1429 | </itemizedlist> |
1455 | </para> | 1430 | </para> |
1456 | |||
1457 | <section id='setscene'> | ||
1458 | <title>Setscene</title> | ||
1459 | |||
1460 | <para> | ||
1461 | This section needs to get the concept of the setscene across. | ||
1462 | The reader needs to know what it is and what it is used for during | ||
1463 | the build process. | ||
1464 | </para> | ||
1465 | </section> | ||
1466 | |||
1467 | <section id='checksums'> | ||
1468 | <title>Checksums (Signatures)</title> | ||
1469 | |||
1470 | <para> | ||
1471 | A checksum is a unique signature of a task's inputs. | ||
1472 | The setscene code uses a checksum to determine if a task needs | ||
1473 | to be run. | ||
1474 | Because it is a change in a task's inputs that triggers running | ||
1475 | the task, the process needs to detect all the inputs to a given task. | ||
1476 | For shell tasks, this turns out to be fairly easy because | ||
1477 | BitBake generates a "run" shell script for each task and | ||
1478 | it is possible to create a checksum that gives you a good idea of when | ||
1479 | the task's data changes. | ||
1480 | </para> | ||
1481 | |||
1482 | <para> | ||
1483 | To complicate the problem, some things should not be included in | ||
1484 | the checksum. | ||
1485 | First, there is the actual specific build path of a given task - | ||
1486 | the working directory. | ||
1487 | It does not matter if the working directory changes because it should not | ||
1488 | affect the output for target packages. | ||
1489 | The simplistic approach for excluding the working directory is to set | ||
1490 | it to some fixed value and create the checksum for the "run" script. | ||
1491 | </para> | ||
1492 | |||
1493 | <para> | ||
1494 | Another problem results from the "run" scripts containing functions that | ||
1495 | might or might not get called. | ||
1496 | The incremental build solution contains code that figures out dependencies | ||
1497 | between shell functions. | ||
1498 | This code is used to prune the "run" scripts down to the minimum set, | ||
1499 | thereby alleviating this problem and making the "run" scripts much more | ||
1500 | readable as a bonus. | ||
1501 | </para> | ||
1502 | |||
1503 | <para> | ||
1504 | So far we have solutions for shell scripts. | ||
1505 | What about Python tasks? | ||
1506 | The same approach applies even though these tasks are more difficult. | ||
1507 | The process needs to figure out what variables a Python function accesses | ||
1508 | and what functions it calls. | ||
1509 | Again, the incremental build solution contains code that first figures out | ||
1510 | the variable and function dependencies, and then creates a checksum for the data | ||
1511 | used as the input to the task. | ||
1512 | </para> | ||
1513 | |||
1514 | <para> | ||
1515 | Like the working directory case, situations exist where dependencies | ||
1516 | should be ignored. | ||
1517 | For these cases, you can instruct the build process to ignore a dependency | ||
1518 | by using a line like the following: | ||
1519 | <literallayout class='monospaced'> | ||
1520 | PACKAGE_ARCHS[vardepsexclude] = "MACHINE" | ||
1521 | </literallayout> | ||
1522 | This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not | ||
1523 | depend on the value of <filename>MACHINE</filename>, even if it does reference it. | ||
1524 | </para> | ||
1525 | |||
1526 | <para> | ||
1527 | Equally, there are cases where we need to add dependencies BitBake | ||
1528 | is not able to find. | ||
1529 | You can accomplish this by using a line like the following: | ||
1530 | <literallayout class='monospaced'> | ||
1531 | PACKAGE_ARCHS[vardeps] = "MACHINE" | ||
1532 | </literallayout> | ||
1533 | This example explicitly adds the <filename>MACHINE</filename> variable as a | ||
1534 | dependency for <filename>PACKAGE_ARCHS</filename>. | ||
1535 | </para> | ||
1536 | |||
1537 | <para> | ||
1538 | Consider a case with in-line Python, for example, where BitBake is not | ||
1539 | able to figure out dependencies. | ||
1540 | When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake | ||
1541 | produces output when it discovers something for which it cannot figure out | ||
1542 | dependencies. | ||
1543 | </para> | ||
1544 | |||
1545 | <para> | ||
1546 | Thus far, this section has limited discussion to the direct inputs into a task. | ||
1547 | Information based on direct inputs is referred to as the "basehash" in the | ||
1548 | code. | ||
1549 | However, there is still the question of a task's indirect inputs - the | ||
1550 | things that were already built and present in the build directory. | ||
1551 | The checksum (or signature) for a particular task needs to add the hashes | ||
1552 | of all the tasks on which the particular task depends. | ||
1553 | Choosing which dependencies to add is a policy decision. | ||
1554 | However, the effect is to generate a master checksum that combines the basehash | ||
1555 | and the hashes of the task's dependencies. | ||
1556 | </para> | ||
1557 | |||
1558 | <para> | ||
1559 | At the code level, there are a variety of ways both the basehash and the | ||
1560 | dependent task hashes can be influenced. | ||
1561 | Within the BitBake configuration file, we can give BitBake some extra information | ||
1562 | to help it construct the basehash. | ||
1563 | The following statement effectively results in a list of global variable | ||
1564 | dependency excludes - variables never included in any checksum. | ||
1565 | This example uses variables from OpenEmbedded to help illustrate | ||
1566 | the concept: | ||
1567 | <literallayout class='monospaced'> | ||
1568 | BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \ | ||
1569 | SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \ | ||
1570 | USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \ | ||
1571 | PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \ | ||
1572 | CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX" | ||
1573 | </literallayout> | ||
1574 | The previous example excludes the work directory, which is part of | ||
1575 | <filename>TMPDIR</filename>. | ||
1576 | </para> | ||
1577 | |||
1578 | <para> | ||
1579 | The rules for deciding which hashes of dependent tasks to include through | ||
1580 | dependency chains are more complex and are generally accomplished with a | ||
1581 | Python function. | ||
1582 | The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples | ||
1583 | of this and also illustrates how you can insert your own policy into the system | ||
1584 | if so desired. | ||
1585 | This file defines the two basic signature generators OpenEmbedded Core | ||
1586 | uses: "OEBasic" and "OEBasicHash". | ||
1587 | By default, there is a dummy "noop" signature handler enabled in BitBake. | ||
1588 | This means that behavior is unchanged from previous versions. | ||
1589 | <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default | ||
1590 | through this setting in the <filename>bitbake.conf</filename> file: | ||
1591 | <literallayout class='monospaced'> | ||
1592 | BB_SIGNATURE_HANDLER ?= "OEBasicHash" | ||
1593 | </literallayout> | ||
1594 | The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the | ||
1595 | "OEBasic" version but adds the task hash to the stamp files. | ||
1596 | This results in any metadata change that changes the task hash, automatically | ||
1597 | causing the task to be run again. | ||
1598 | This removes the need to bump | ||
1599 | <link linkend='var-PR'><filename>PR</filename></link> | ||
1600 | values, and changes to metadata automatically ripple across the build. | ||
1601 | </para> | ||
1602 | |||
1603 | <para> | ||
1604 | It is also worth noting that the end result of these signature generators is to | ||
1605 | make some dependency and hash information available to the build. | ||
1606 | This information includes: | ||
1607 | <itemizedlist> | ||
1608 | <listitem><para><filename>BB_BASEHASH_task-<taskname></filename>: | ||
1609 | The base hashes for each task in the recipe. | ||
1610 | </para></listitem> | ||
1611 | <listitem><para><filename>BB_BASEHASH_<filename:taskname></filename>: | ||
1612 | The base hashes for each dependent task. | ||
1613 | </para></listitem> | ||
1614 | <listitem><para><filename>BBHASHDEPS_<filename:taskname></filename>: | ||
1615 | The task dependencies for each task. | ||
1616 | </para></listitem> | ||
1617 | <listitem><para><filename>BB_TASKHASH</filename>: | ||
1618 | The hash of the currently running task. | ||
1619 | </para></listitem> | ||
1620 | </itemizedlist> | ||
1621 | </para> | ||
1622 | </section> | ||
1623 | </section> | 1431 | </section> |
1624 | </chapter> | 1432 | </chapter> |