summaryrefslogtreecommitdiffstats
path: root/bitbake/doc
diff options
context:
space:
mode:
Diffstat (limited to 'bitbake/doc')
-rw-r--r--bitbake/doc/user-manual/user-manual-metadata.xml208
1 files changed, 8 insertions, 200 deletions
diff --git a/bitbake/doc/user-manual/user-manual-metadata.xml b/bitbake/doc/user-manual/user-manual-metadata.xml
index 8d37f6bee1..cabf25fb6f 100644
--- a/bitbake/doc/user-manual/user-manual-metadata.xml
+++ b/bitbake/doc/user-manual/user-manual-metadata.xml
@@ -2,40 +2,15 @@
2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> 2"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
3 3
4<chapter id="user-manual-metadata"> 4<chapter id="user-manual-metadata">
5 <title>Metadata</title> 5 <title>Syntax and Operators</title>
6 6
7 <section> 7 <para>
8 <title>Overview</title> 8 Bitbake files have their own syntax.
9 9 The syntax has similarities to several
10 <para> 10 other languages but also has some unique features.
11 The BitBake task executor together with various types of configuration 11 This section describes the available syntax and operators
12 files form the OpenEmbedded Core. 12 as well as provides examples.
13 This section provides an overview of the task executor and the 13 </para>
14 configuration files by describing their use and interaction.
15 </para>
16
17 <para>
18 BitBake handles the parsing and execution of the data files.
19 The data itself is of various types:
20 <itemizedlist>
21 <listitem><para><emphasis>Recipes:</emphasis>
22 Details about particular pieces of software.
23 </para></listitem>
24 <listitem><para><emphasis>Class Data:</emphasis>
25 An abstraction of common build information
26 (e.g. how to build a Linux kernel).
27 </para></listitem>
28 <listitem><para><emphasis>Configuration Data:</emphasis>
29 Machine-specific settings, policy decisions,
30 and so forth.
31 Configuration data acts as the glue to bind everything
32 together.</para></listitem>
33 </itemizedlist>
34 The remainder of this chapter provides examples of BitBake metadata.
35 Any syntax not supported in any of the previously listed areas
36 is documented as such.
37 </para>
38 </section>
39 14
40 <section id='basic-syntax'> 15 <section id='basic-syntax'>
41 <title>Basic Syntax</title> 16 <title>Basic Syntax</title>
@@ -1453,172 +1428,5 @@
1453 </para></listitem> 1428 </para></listitem>
1454 </itemizedlist> 1429 </itemizedlist>
1455 </para> 1430 </para>
1456
1457 <section id='setscene'>
1458 <title>Setscene</title>
1459
1460 <para>
1461 This section needs to get the concept of the setscene across.
1462 The reader needs to know what it is and what it is used for during
1463 the build process.
1464 </para>
1465 </section>
1466
1467 <section id='checksums'>
1468 <title>Checksums (Signatures)</title>
1469
1470 <para>
1471 A checksum is a unique signature of a task's inputs.
1472 The setscene code uses a checksum to determine if a task needs
1473 to be run.
1474 Because it is a change in a task's inputs that triggers running
1475 the task, the process needs to detect all the inputs to a given task.
1476 For shell tasks, this turns out to be fairly easy because
1477 BitBake generates a "run" shell script for each task and
1478 it is possible to create a checksum that gives you a good idea of when
1479 the task's data changes.
1480 </para>
1481
1482 <para>
1483 To complicate the problem, some things should not be included in
1484 the checksum.
1485 First, there is the actual specific build path of a given task -
1486 the working directory.
1487 It does not matter if the working directory changes because it should not
1488 affect the output for target packages.
1489 The simplistic approach for excluding the working directory is to set
1490 it to some fixed value and create the checksum for the "run" script.
1491 </para>
1492
1493 <para>
1494 Another problem results from the "run" scripts containing functions that
1495 might or might not get called.
1496 The incremental build solution contains code that figures out dependencies
1497 between shell functions.
1498 This code is used to prune the "run" scripts down to the minimum set,
1499 thereby alleviating this problem and making the "run" scripts much more
1500 readable as a bonus.
1501 </para>
1502
1503 <para>
1504 So far we have solutions for shell scripts.
1505 What about Python tasks?
1506 The same approach applies even though these tasks are more difficult.
1507 The process needs to figure out what variables a Python function accesses
1508 and what functions it calls.
1509 Again, the incremental build solution contains code that first figures out
1510 the variable and function dependencies, and then creates a checksum for the data
1511 used as the input to the task.
1512 </para>
1513
1514 <para>
1515 Like the working directory case, situations exist where dependencies
1516 should be ignored.
1517 For these cases, you can instruct the build process to ignore a dependency
1518 by using a line like the following:
1519 <literallayout class='monospaced'>
1520 PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
1521 </literallayout>
1522 This example ensures that the <filename>PACKAGE_ARCHS</filename> variable does not
1523 depend on the value of <filename>MACHINE</filename>, even if it does reference it.
1524 </para>
1525
1526 <para>
1527 Equally, there are cases where we need to add dependencies BitBake
1528 is not able to find.
1529 You can accomplish this by using a line like the following:
1530 <literallayout class='monospaced'>
1531 PACKAGE_ARCHS[vardeps] = "MACHINE"
1532 </literallayout>
1533 This example explicitly adds the <filename>MACHINE</filename> variable as a
1534 dependency for <filename>PACKAGE_ARCHS</filename>.
1535 </para>
1536
1537 <para>
1538 Consider a case with in-line Python, for example, where BitBake is not
1539 able to figure out dependencies.
1540 When running in debug mode (i.e. using <filename>-DDD</filename>), BitBake
1541 produces output when it discovers something for which it cannot figure out
1542 dependencies.
1543 </para>
1544
1545 <para>
1546 Thus far, this section has limited discussion to the direct inputs into a task.
1547 Information based on direct inputs is referred to as the "basehash" in the
1548 code.
1549 However, there is still the question of a task's indirect inputs - the
1550 things that were already built and present in the build directory.
1551 The checksum (or signature) for a particular task needs to add the hashes
1552 of all the tasks on which the particular task depends.
1553 Choosing which dependencies to add is a policy decision.
1554 However, the effect is to generate a master checksum that combines the basehash
1555 and the hashes of the task's dependencies.
1556 </para>
1557
1558 <para>
1559 At the code level, there are a variety of ways both the basehash and the
1560 dependent task hashes can be influenced.
1561 Within the BitBake configuration file, we can give BitBake some extra information
1562 to help it construct the basehash.
1563 The following statement effectively results in a list of global variable
1564 dependency excludes - variables never included in any checksum.
1565 This example uses variables from OpenEmbedded to help illustrate
1566 the concept:
1567 <literallayout class='monospaced'>
1568 BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH DL_DIR \
1569 SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
1570 USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
1571 PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
1572 CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_DISABLE LICENSE_PATH SDKPKGSUFFIX"
1573 </literallayout>
1574 The previous example excludes the work directory, which is part of
1575 <filename>TMPDIR</filename>.
1576 </para>
1577
1578 <para>
1579 The rules for deciding which hashes of dependent tasks to include through
1580 dependency chains are more complex and are generally accomplished with a
1581 Python function.
1582 The code in <filename>meta/lib/oe/sstatesig.py</filename> shows two examples
1583 of this and also illustrates how you can insert your own policy into the system
1584 if so desired.
1585 This file defines the two basic signature generators OpenEmbedded Core
1586 uses: "OEBasic" and "OEBasicHash".
1587 By default, there is a dummy "noop" signature handler enabled in BitBake.
1588 This means that behavior is unchanged from previous versions.
1589 <filename>OE-Core</filename> uses the "OEBasicHash" signature handler by default
1590 through this setting in the <filename>bitbake.conf</filename> file:
1591 <literallayout class='monospaced'>
1592 BB_SIGNATURE_HANDLER ?= "OEBasicHash"
1593 </literallayout>
1594 The "OEBasicHash" <filename>BB_SIGNATURE_HANDLER</filename> is the same as the
1595 "OEBasic" version but adds the task hash to the stamp files.
1596 This results in any metadata change that changes the task hash, automatically
1597 causing the task to be run again.
1598 This removes the need to bump
1599 <link linkend='var-PR'><filename>PR</filename></link>
1600 values, and changes to metadata automatically ripple across the build.
1601 </para>
1602
1603 <para>
1604 It is also worth noting that the end result of these signature generators is to
1605 make some dependency and hash information available to the build.
1606 This information includes:
1607 <itemizedlist>
1608 <listitem><para><filename>BB_BASEHASH_task-&lt;taskname&gt;</filename>:
1609 The base hashes for each task in the recipe.
1610 </para></listitem>
1611 <listitem><para><filename>BB_BASEHASH_&lt;filename:taskname&gt;</filename>:
1612 The base hashes for each dependent task.
1613 </para></listitem>
1614 <listitem><para><filename>BBHASHDEPS_&lt;filename:taskname&gt;</filename>:
1615 The task dependencies for each task.
1616 </para></listitem>
1617 <listitem><para><filename>BB_TASKHASH</filename>:
1618 The hash of the currently running task.
1619 </para></listitem>
1620 </itemizedlist>
1621 </para>
1622 </section>
1623 </section> 1431 </section>
1624</chapter> 1432</chapter>