path: root/documentation/ref-manual/usingpoky.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

<chapter id='usingpoky'>
<title>Using the Yocto Project</title>

    <para>
        This chapter describes common usage for the Yocto Project.
        The information is introductory in nature as other manuals in the Yocto Project
        documentation set provide more details on how to use the Yocto Project.
    </para>

<section id='ref-quick-emulator-qemu'>
    <title>Quick EMUlator (QEMU)</title>

    <para>
        The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
        Open Source project as part of the Yocto Project development "tool
        set".
    </para>

    <para>
        Within the context of the Yocto Project, QEMU is an
        emulator and virtualization machine that allows you to run a complete
        image you have built using the Yocto Project as just another task
        on your build system.
        QEMU is useful for running and testing images and applications on
        supported Yocto Project architectures without having actual hardware.
        Among other things, the Yocto Project uses QEMU to run automated
        Quality Assurance (QA) tests on final images shipped with each
        release.
        <note>
            This implementation is not the same as QEMU in general.
        </note>
        This section provides a brief reference for the Yocto Project
        implementation of QEMU.
    </para>

    <para>
        For official information and documentation on QEMU in general, see the
        following references:
        <itemizedlist>
            <listitem><para>
                <emphasis><ulink url='http://wiki.qemu.org/Main_Page'>QEMU Website</ulink>:</emphasis>
                The official website for the QEMU Open Source project.
                </para></listitem>
            <listitem><para>
                <emphasis><ulink url='http://wiki.qemu.org/Manual'>Documentation</ulink>:</emphasis>
                The QEMU user manual.
                </para></listitem>
        </itemizedlist>
    </para>

    <para>
        For information on how to use the Yocto Project implementation of
        QEMU, see the
        "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-manual-qemu'>Using the Quick EMUlator (QEMU)</ulink>"
        chapter in the Yocto Project Development Tasks Manual.
    </para>

    <section id='qemu-availability'>
        <title>QEMU Availability</title>

        <para>
            QEMU is made available with the Yocto Project a number of ways.
            One method is to install a Software Development Kit (SDK).
            For more information on how to make sure you have
            QEMU available, see
            "<ulink url='&YOCTO_DOCS_SDK_URL;#the-qemu-emulator'>The QEMU Emulator</ulink>"
            section in the Yocto Project Application Development and the
            Extensible Software Development Kit (eSDK) manual.
        </para>
    </section>

    <section id='qemu-performance'>
        <title>QEMU Performance</title>

        <para>
            Using QEMU to emulate your hardware can result in speed issues
            depending on the target and host architecture mix.
            For example, using the <filename>qemux86</filename> image in the
            emulator on an Intel-based 32-bit (x86) host machine is fast
            because the target and host architectures match.
            On the other hand, using the <filename>qemuarm</filename> image
            on the same Intel-based host can be slower.
            But, you still achieve faithful emulation of ARM-specific issues.
        </para>

        <para>
            To speed things up, the QEMU images support using
            <filename>distcc</filename> to call a cross-compiler outside the
            emulated system.
            If you used <filename>runqemu</filename> to start QEMU, and the
            <filename>distccd</filename> application is present on the host
            system, any BitBake cross-compiling toolchain available from the
            build system is automatically used from within QEMU simply by
            calling <filename>distcc</filename>.
            You can accomplish this by defining the cross-compiler variable
            (e.g. <filename>export CC="distcc"</filename>).
            Alternatively, if you are using a suitable SDK image or the
            appropriate stand-alone toolchain is present, the toolchain is
            also automatically used.
        </para>

        <note>
            Several mechanisms exist that let you connect to the system
            running on the QEMU emulator:
            <itemizedlist>
                <listitem><para>
                    QEMU provides a framebuffer interface that makes standard
                    consoles available.
                    </para></listitem>
                <listitem><para>
                    Generally, headless embedded devices have a serial port.
                    If so, you can configure the operating system of the
                    running image to use that port to run a console.
                    The connection uses standard IP networking.
                    </para></listitem>
                <listitem><para>
                    SSH servers exist in some QEMU images.
                    The <filename>core-image-sato</filename> QEMU image has a
                    Dropbear secure shell (SSH) server that runs with the root
                    password disabled.
                    The <filename>core-image-full-cmdline</filename> and
                    <filename>core-image-lsb</filename> QEMU images
                    have OpenSSH instead of Dropbear.
                    Including these SSH servers allow you to use standard
                    <filename>ssh</filename> and <filename>scp</filename>
                    commands.
                    The <filename>core-image-minimal</filename> QEMU image,
                    however, contains no SSH server.
                    </para></listitem>
                <listitem><para>
                    You can use a provided, user-space NFS server to boot
                    the QEMU session using a local copy of the root
                    filesystem on the host.
                    In order to make this connection, you must extract a
                    root filesystem tarball by using the
                    <filename>runqemu-extract-sdk</filename> command.
                    After running the command, you must then point the
                    <filename>runqemu</filename>
                    script to the extracted directory instead of a root
                    filesystem image file.
                    See the
                    "<ulink url='&YOCTO_DOCS_DEV_URL;#qemu-running-under-a-network-file-system-nfs-server'>Running Under a Network File System (NFS) Server</ulink>"
                    section in the Yocto Project Development Tasks Manual for
                    more information.
                    </para></listitem>
            </itemizedlist>
        </note>
    </section>

    <section id='qemu-command-line-syntax'>
        <title>QEMU Command-Line Syntax</title>

        <para>
            The basic <filename>runqemu</filename> command syntax is as
            follows:
            <literallayout class='monospaced'>
     $ runqemu [<replaceable>option</replaceable> ]  [...]
            </literallayout>
            Based on what you provide on the command line,
            <filename>runqemu</filename> does a good job of figuring out what
            you are trying to do.
            For example, by default, QEMU looks for the most recently built
            image according to the timestamp when it needs to look for an
            image.
            Minimally, through the use of options, you must provide either
            a machine name, a virtual machine image
            (<filename>*wic.vmdk</filename>), or a kernel image
            (<filename>*.bin</filename>).
        </para>

        <para>
            Following is the command-line help output for the
            <filename>runqemu</filename> command:
            <literallayout class='monospaced'>
     $ runqemu --help

     Usage: you can run this script with any valid combination
     of the following environment variables (in any order):
       KERNEL - the kernel image file to use
       ROOTFS - the rootfs image file or nfsroot directory to use
       MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
       Simplified QEMU command-line options can be passed with:
         nographic - disable video console
         serial - enable a serial console on /dev/ttyS0
         slirp - enable user networking, no root privileges is required
         kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
         kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
         publicvnc - enable a VNC server open to all hosts
         audio - enable audio
         [*/]ovmf* - OVMF firmware file or base name for booting with UEFI
       tcpserial=&lt;port&gt; - specify tcp serial port number
       biosdir=&lt;dir&gt; - specify custom bios dir
       biosfilename=&lt;filename&gt; - specify bios filename
       qemuparams=&lt;xyz&gt; - specify custom parameters to QEMU
       bootparams=&lt;xyz&gt; - specify custom kernel parameters during boot
       help, -h, --help: print this text

     Examples:
       runqemu
       runqemu qemuarm
       runqemu tmp/deploy/images/qemuarm
       runqemu tmp/deploy/images/qemux86/&lt;qemuboot.conf&gt;
       runqemu qemux86-64 core-image-sato ext4
       runqemu qemux86-64 wic-image-minimal wic
       runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
       runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
       runqemu qemux86 qemuparams="-m 256"
       runqemu qemux86 bootparams="psplash=false"
       runqemu path/to/&lt;image&gt;-&lt;machine&gt;.wic
       runqemu path/to/&lt;image&gt;-&lt;machine&gt;.wic.vmdk
            </literallayout>
        </para>
    </section>

    <section id='runqemu-command-line-options'>
        <title><filename>runqemu</filename> Command-Line Options</title>

        <para>
            Following is a description of <filename>runqemu</filename>
            options you can provide on the command line:
            <note><title>Tip</title>
                If you do provide some "illegal" option combination or perhaps
                you do not provide enough in the way of options,
                <filename>runqemu</filename> provides appropriate error
                messaging to help you correct the problem.
            </note>
            <itemizedlist>
                <listitem><para>
                    <replaceable>QEMUARCH</replaceable>:
                    The QEMU machine architecture, which must be "qemuarm",
                    "qemuarm64", "qemumips", "qemumips64", "qemuppc",
                    "qemux86", or "qemux86-64".
                    </para></listitem>
                <listitem><para>
                    <filename><replaceable>VM</replaceable></filename>:
                    The virtual machine image, which must be a
                    <filename>.wic.vmdk</filename> file.
                    Use this option when you want to boot a
                    <filename>.wic.vmdk</filename> image.
                    The image filename you provide must contain one of the
                    following strings: "qemux86-64", "qemux86", "qemuarm",
                    "qemumips64", "qemumips", "qemuppc", or "qemush4".
                    </para></listitem>
                <listitem><para>
                    <replaceable>ROOTFS</replaceable>:
                    A root filesystem that has one of the following
                    filetype extensions: "ext2", "ext3", "ext4", "jffs2",
                    "nfs", or "btrfs".
                    If the filename you provide for this option uses “nfs”, it
                    must provide an explicit root filesystem path.
                    </para></listitem>
                <listitem><para>
                    <replaceable>KERNEL</replaceable>:
                    A kernel image, which is a <filename>.bin</filename> file.
                    When you provide a <filename>.bin</filename> file,
                    <filename>runqemu</filename> detects it and assumes the
                    file is a kernel image.
                    </para></listitem>
                <listitem><para>
                    <replaceable>MACHINE</replaceable>:
                    The architecture of the QEMU machine, which must be one
                    of the following: "qemux86", "qemux86-64", "qemuarm",
                    "qemuarm64", "qemumips", “qemumips64", or "qemuppc".
                    The <replaceable>MACHINE</replaceable> and
                    <replaceable>QEMUARCH</replaceable> options are basically
                    identical.
                    If you do not provide a <replaceable>MACHINE</replaceable>
                    option, <filename>runqemu</filename> tries to determine
                    it based on other options.
                    </para></listitem>
                <listitem><para>
                    <filename>ramfs</filename>:
                    Indicates you are booting an initial RAM disk (initramfs)
                    image, which means the <filename>FSTYPE</filename> is
                    <filename>cpio.gz</filename>.
                    </para></listitem>
                <listitem><para>
                    <filename>iso</filename>:
                    Indicates you are booting an ISO image, which means the
                    <filename>FSTYPE</filename> is
                    <filename>.iso</filename>.
                    </para></listitem>
                <listitem><para>
                    <filename>nographic</filename>:
                    Disables the video console, which sets the console to
                    "ttys0".
                    </para></listitem>
                <listitem><para>
                    <filename>serial</filename>:
                    Enables a serial console on
                    <filename>/dev/ttyS0</filename>.
                    </para></listitem>
                <listitem><para>
                    <filename>biosdir</filename>:
                    Establishes a custom directory for BIOS, VGA BIOS and
                    keymaps.
                    </para></listitem>
                <listitem><para>
                    <filename>biosfilename</filename>:
                    Establishes a custom BIOS name.
                    </para></listitem>
                <listitem><para>
                    <filename>qemuparams=\"<replaceable>xyz</replaceable>\"</filename>:
                    Specifies custom QEMU parameters.
                    Use this option to pass options other than the simple
                    "kvm" and "serial" options.
                    </para></listitem>
                <listitem><para><filename>bootparams=\"<replaceable>xyz</replaceable>\"</filename>:
                    Specifies custom boot parameters for the kernel.
                    </para></listitem>
                <listitem><para>
                    <filename>audio</filename>:
                    Enables audio in QEMU.
                    The <replaceable>MACHINE</replaceable> option must be
                    either "qemux86" or "qemux86-64" in order for audio to be
                    enabled.
                    Additionally, the <filename>snd_intel8x0</filename>
                    or <filename>snd_ens1370</filename> driver must be
                    installed in linux guest.
                    </para></listitem>
                <listitem><para>
                    <filename>slirp</filename>:
                    Enables "slirp" networking, which is a different way
                    of networking that does not need root access
                    but also is not as easy to use or comprehensive
                    as the default.
                    </para></listitem>
                <listitem><para id='kvm-cond'>
                    <filename>kvm</filename>:
                    Enables KVM when running "qemux86" or "qemux86-64"
                    QEMU architectures.
                    For KVM to work, all the following conditions must be met:
                    <itemizedlist>
                        <listitem><para>
                            Your <replaceable>MACHINE</replaceable> must be either
qemux86" or "qemux86-64".
                            </para></listitem>
                        <listitem><para>
                            Your build host has to have the KVM modules
                            installed, which are
                            <filename>/dev/kvm</filename>.
                            </para></listitem>
                        <listitem><para>
                            The  build host <filename>/dev/kvm</filename>
                            directory has to be both writable and readable.
                            </para></listitem>
                    </itemizedlist>
                    </para></listitem>
                <listitem><para>
                    <filename>kvm-vhost</filename>:
                    Enables KVM with VHOST support when running "qemux86"
                    or "qemux86-64" QEMU architectures.
                    For KVM with VHOST to work, the following conditions must
                    be met:
                    <itemizedlist>
                        <listitem><para>
                            <link linkend='kvm-cond'>kvm</link> option
                            conditions must be met.
                            </para></listitem>
                        <listitem><para>
                            Your build host has to have virtio net device, which
                            are <filename>/dev/vhost-net</filename>.
                            </para></listitem>
                        <listitem><para>
                            The build host <filename>/dev/vhost-net</filename>
                            directory has to be either readable or writable
                            and “slirp-enabled”.
                            </para></listitem>
                    </itemizedlist>
                    </para></listitem>
                <listitem><para>
                    <filename>publicvnc</filename>:
                    Enables a VNC server open to all hosts.
                    </para></listitem>
            </itemizedlist>
        </para>
    </section>
</section>

<section id='maintaining-build-output-quality'>
    <title>Maintaining Build Output Quality</title>

    <para>
        Many factors can influence the quality of a build.
        For example, if you upgrade a recipe to use a new version of an upstream software
        package or you experiment with some new configuration options, subtle changes
        can occur that you might not detect until later.
        Consider the case where your recipe is using a newer version of an upstream package.
        In this case, a new version of a piece of software might introduce an optional
        dependency on another library, which is auto-detected.
        If that library has already been built when the software is building,
        the software will link to the built library and that library will be pulled
        into your image along with the new software even if you did not want the
        library.
    </para>

    <para>
        The
        <link linkend='ref-classes-buildhistory'><filename>buildhistory</filename></link>
        class exists to help you maintain
        the quality of your build output.
        You can use the class to highlight unexpected and possibly unwanted
        changes in the build output.
        When you enable build history, it records information about the contents of
        each package and image and then commits that information to a local Git
        repository where you can examine the information.
    </para>

    <para>
        The remainder of this section describes the following:
        <itemizedlist>
           <listitem><para>How you can enable and disable
               build history</para></listitem>
           <listitem><para>How to understand what the build history contains
               </para></listitem>
           <listitem><para>How to limit the information used for build history
               </para></listitem>
           <listitem><para>How to examine the build history from both a
               command-line and web interface</para></listitem>
       </itemizedlist>
    </para>

    <section id='enabling-and-disabling-build-history'>
        <title>Enabling and Disabling Build History</title>

        <para>
            Build history is disabled by default.
            To enable it, add the following <filename>INHERIT</filename>
            statement and set the
            <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></link>
            variable to "1" at the end of your
            <filename>conf/local.conf</filename> file found in the
            <link linkend='build-directory'>Build Directory</link>:
            <literallayout class='monospaced'>
     INHERIT += "buildhistory"
     BUILDHISTORY_COMMIT = "1"
            </literallayout>
            Enabling build history as previously described causes the
            OpenEmbedded build system to collect build output information and
            commit it as a single commit to a local
            <ulink url='&YOCTO_DOCS_OVERVIEW_URL;#git'>Git</ulink> repository.
            <note>
                Enabling build history increases your build times slightly,
                particularly for images, and increases the amount of disk
                space used during the build.
            </note>
        </para>

        <para>
            You can disable build history by removing the previous statements
            from your <filename>conf/local.conf</filename> file.
        </para>
    </section>

    <section id='understanding-what-the-build-history-contains'>
        <title>Understanding What the Build History Contains</title>

        <para>
            Build history information is kept in
            <filename>${</filename><link linkend='var-TOPDIR'><filename>TOPDIR</filename></link><filename>}/buildhistory</filename>
            in the Build Directory as defined by the
            <link linkend='var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></link>
            variable.
            The following is an example abbreviated listing:
            <imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" />
        </para>

        <para>
            At the top level, there is a <filename>metadata-revs</filename> file
            that lists the revisions of the repositories for the layers enabled
            when the build was produced.
            The rest of the data splits into separate
            <filename>packages</filename>, <filename>images</filename> and
            <filename>sdk</filename> directories, the contents of which are
            described below.
        </para>

        <section id='build-history-package-information'>
            <title>Build History Package Information</title>

            <para>
                The history for each package contains a text file that has
                name-value pairs with information about the package.
                For example, <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename>
                contains the following:
                <literallayout class='monospaced'>
     PV = 1.22.1
     PR = r32
     RPROVIDES =
     RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
     RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
     PKGSIZE = 540168
     FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
        /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
        /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
        /usr/share/pixmaps /usr/share/applications /usr/share/idl \
        /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
     FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
        /etc/busybox.links.nosuid /etc/busybox.links.suid
                </literallayout>
                Most of these name-value pairs correspond to variables used
                to produce the package.
                The exceptions are <filename>FILELIST</filename>, which is the
                actual list of files in the package, and
                <filename>PKGSIZE</filename>, which is the total size of files
                in the package in bytes.
            </para>

            <para>
                There is also a file corresponding to the recipe from which the
                package came (e.g.
                <filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>):
                <literallayout class='monospaced'>
     PV = 1.22.1
     PR = r32
     DEPENDS = initscripts kern-tools-native update-rc.d-native \
        virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
        virtual/libc virtual/update-alternatives
     PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
        busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
        busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
                </literallayout>
            </para>

            <para>
                Finally, for those recipes fetched from a version control
                system (e.g., Git), a file exists that lists source revisions
                that are specified in the recipe and lists the actual revisions
                used during the build.
                Listed and actual revisions might differ when
                <link linkend='var-SRCREV'><filename>SRCREV</filename></link>
                is set to
                <filename>${<link linkend='var-AUTOREV'>AUTOREV</link>}</filename>.
                Here is an example assuming
                <filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>):
                <literallayout class='monospaced'>
     # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
     SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
     # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
     SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
                </literallayout>
                You can use the <filename>buildhistory-collect-srcrevs</filename>
                command with the <filename>-a</filename> option to
                collect the stored <filename>SRCREV</filename> values
                from build history and report them in a format suitable for
                use in global configuration (e.g.,
                <filename>local.conf</filename> or a distro include file) to
                override floating <filename>AUTOREV</filename> values to a
                fixed set of revisions.
                Here is some example output from this command:
                <literallayout class='monospaced'>
     $ buildhistory-collect-srcrevs -a
     # i586-poky-linux
     SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072"
     SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072"
     SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a"
     SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
     # x86_64-linux
     SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa"
     SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf"
     SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11"
     SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072"
     SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3"
     SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca"
     SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a"
     SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff"
     SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4"
     # qemux86-poky-linux
     SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
     SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f"
     # all-poky-linux
     SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11"
                </literallayout>
                <note>
                    Here are some notes on using the
                    <filename>buildhistory-collect-srcrevs</filename> command:
                    <itemizedlist>
                        <listitem><para>By default, only values where the
                            <filename>SRCREV</filename> was
                            not hardcoded (usually when <filename>AUTOREV</filename>
                            was used) are reported.
                            Use the <filename>-a</filename> option to see all
                            <filename>SRCREV</filename> values.
                            </para></listitem>
                        <listitem><para>The output statements might not have any effect
                            if overrides are applied elsewhere in the build system
                            configuration.
                            Use the <filename>-f</filename> option to add the
                            <filename>forcevariable</filename> override to each output line
                            if you need to work around this restriction.
                            </para></listitem>
                        <listitem><para>The script does apply special handling when
                            building for multiple machines.
                            However, the script does place a
                            comment before each set of values that specifies
                            which triplet to which they belong as shown above
                            (e.g., <filename>i586-poky-linux</filename>).
                            </para></listitem>
                    </itemizedlist>
                </note>
            </para>
        </section>

        <section id='build-history-image-information'>
            <title>Build History Image Information</title>

            <para>
                The files produced for each image are as follows:
                <itemizedlist>
                    <listitem><para><filename>image-files:</filename>
                        A directory containing selected files from the root
                        filesystem.
                        The files are defined by
                        <link linkend='var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></link>.
                        </para></listitem>
                    <listitem><para><filename>build-id.txt:</filename>
                        Human-readable information about the build configuration
                        and metadata source revisions.
                        This file contains the full build header as printed
                        by BitBake.</para></listitem>
                    <listitem><para><filename>*.dot:</filename>
                        Dependency graphs for the image that are
                        compatible with <filename>graphviz</filename>.
                        </para></listitem>
                    <listitem><para><filename>files-in-image.txt:</filename>
 	                    A list of files in the image with permissions,
                        owner, group, size, and symlink information.
                        </para></listitem>
                    <listitem><para><filename>image-info.txt:</filename>
                        A text file containing name-value pairs with information
                        about the image.
                        See the following listing example for more information.
                        </para></listitem>
                    <listitem><para><filename>installed-package-names.txt:</filename>
                        A list of installed packages by name only.</para></listitem>
                    <listitem><para><filename>installed-package-sizes.txt:</filename>
                        A list of installed packages ordered by size.
                        </para></listitem>
                    <listitem><para><filename>installed-packages.txt:</filename>
                        A list of installed packages with full package
                        filenames.</para></listitem>
                </itemizedlist>
                <note>
                    Installed package information is able to be gathered and
                    produced even if package management is disabled for the final
                    image.
                </note>
            </para>

            <para>
                Here is an example of <filename>image-info.txt</filename>:
                <literallayout class='monospaced'>
     DISTRO = poky
     DISTRO_VERSION = 1.7
     USER_CLASSES = buildstats image-mklibs image-prelink
     IMAGE_CLASSES = image_types
     IMAGE_FEATURES = debug-tweaks
     IMAGE_LINGUAS =
     IMAGE_INSTALL = packagegroup-core-boot run-postinsts
     BAD_RECOMMENDATIONS =
     NO_RECOMMENDATIONS =
     PACKAGE_EXCLUDE =
     ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \
        write_image_manifest ; buildhistory_list_installed_image ; \
        buildhistory_get_image_installed ; ssh_allow_empty_password;  \
        postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ;
     IMAGE_POSTPROCESS_COMMAND =   buildhistory_get_imageinfo ;
     IMAGESIZE = 6900
                </literallayout>
                Other than <filename>IMAGESIZE</filename>, which is the
                total size of the files in the image in Kbytes, the
                name-value pairs are variables that may have influenced the
                content of the image.
                This information is often useful when you are trying to determine
                why a change in the package or file listings has occurred.
            </para>
        </section>

        <section id='using-build-history-to-gather-image-information-only'>
            <title>Using Build History to Gather Image Information Only</title>

            <para>
                As you can see, build history produces image information,
                including dependency graphs, so you can see why something
                was pulled into the image.
                If you are just interested in this information and not
                interested in collecting specific package or SDK information,
                you can enable writing only image information without
                any history by adding the following to your
                <filename>conf/local.conf</filename> file found in the
                <link linkend='build-directory'>Build Directory</link>:
                <literallayout class='monospaced'>
     INHERIT += "buildhistory"
     BUILDHISTORY_COMMIT = "0"
     BUILDHISTORY_FEATURES = "image"
                </literallayout>
                Here, you set the
                <link linkend='var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></link>
                variable to use the image feature only.
            </para>
        </section>

        <section id='build-history-sdk-information'>
            <title>Build History SDK Information</title>

            <para>
                Build history collects similar information on the contents
                of SDKs
                (e.g. <filename>bitbake -c populate_sdk imagename</filename>)
                as compared to information it collects for images.
                Furthermore, this information differs depending on whether an
                extensible or standard SDK is being produced.
            </para>

            <para>
                The following list shows the files produced for SDKs:
                <itemizedlist>
                    <listitem><para><filename>files-in-sdk.txt:</filename>
                        A list of files in the SDK with permissions,
                        owner, group, size, and symlink information.
                        This list includes both the host and target parts
                        of the SDK.
                        </para></listitem>
                    <listitem><para><filename>sdk-info.txt:</filename>
                        A text file containing name-value pairs with information
                        about the SDK.
                        See the following listing example for more information.
                        </para></listitem>
                    <listitem><para><filename>sstate-task-sizes.txt:</filename>
                        A text file containing name-value pairs with information
                        about task group sizes
                        (e.g. <filename>do_populate_sysroot</filename> tasks
                        have a total size).
                        The <filename>sstate-task-sizes.txt</filename> file
                        exists only when an extensible SDK is created.
                        </para></listitem>
                    <listitem><para><filename>sstate-package-sizes.txt:</filename>
                        A text file containing name-value pairs with information
                        for the shared-state packages and sizes in the SDK.
                        The <filename>sstate-package-sizes.txt</filename> file
                        exists only when an extensible SDK is created.
                        </para></listitem>
                    <listitem><para><filename>sdk-files:</filename>
                        A folder that contains copies of the files mentioned in
                        <filename>BUILDHISTORY_SDK_FILES</filename> if the
                        files are present in the output.
                        Additionally, the default value of
                        <filename>BUILDHISTORY_SDK_FILES</filename> is specific
                        to the extensible SDK although you can set it
                        differently if you would like to pull in specific files
                        from the standard SDK.</para>
                        <para>The default files are
                        <filename>conf/local.conf</filename>,
                        <filename>conf/bblayers.conf</filename>,
                        <filename>conf/auto.conf</filename>,
                        <filename>conf/locked-sigs.inc</filename>, and
                        <filename>conf/devtool.conf</filename>.
                        Thus, for an extensible SDK, these files get copied
                        into the <filename>sdk-files</filename> directory.
                        </para></listitem>
                    <listitem><para>The following information appears under
                        each of the <filename>host</filename>
                        and <filename>target</filename> directories
                        for the portions of the SDK that run on the host and
                        on the target, respectively:
                        <note>
                            The following files for the most part are empty
                            when producing an extensible SDK because this
                            type of SDK is not constructed from packages as is
                            the standard SDK.
                        </note>
                        <itemizedlist>
                            <listitem><para><filename>depends.dot:</filename>
                                Dependency graph for the SDK that is
                                compatible with <filename>graphviz</filename>.
                                </para></listitem>
                            <listitem><para><filename>installed-package-names.txt:</filename>
                                A list of installed packages by name only.
                                </para></listitem>
                            <listitem><para><filename>installed-package-sizes.txt:</filename>
                                A list of installed packages ordered by size.
                                </para></listitem>
                            <listitem><para><filename>installed-packages.txt:</filename>
                                A list of installed packages with full package
                                filenames.</para></listitem>
                            </itemizedlist>
                        </para></listitem>
                </itemizedlist>
            </para>

            <para>
                Here is an example of <filename>sdk-info.txt</filename>:
                <literallayout class='monospaced'>
     DISTRO = poky
     DISTRO_VERSION = 1.3+snapshot-20130327
     SDK_NAME = poky-glibc-i686-arm
     SDK_VERSION = 1.3+snapshot
     SDKMACHINE =
     SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
     BAD_RECOMMENDATIONS =
     SDKSIZE = 352712
                </literallayout>
                Other than <filename>SDKSIZE</filename>, which is the
                total size of the files in the SDK in Kbytes, the
                name-value pairs are variables that might have influenced the
                content of the SDK.
                This information is often useful when you are trying to
                determine why a change in the package or file listings
                has occurred.
            </para>
        </section>

        <section id='examining-build-history-information'>
            <title>Examining Build History Information</title>

            <para>
                You can examine build history output from the command line or
                from a web interface.
            </para>

            <para>
                To see any changes that have occurred (assuming you have
                <link linkend='var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT = "1"</filename></link>),
                you can simply
                use any Git command that allows you to view the history of
                a repository.
                Here is one method:
                <literallayout class='monospaced'>
      $ git log -p
                </literallayout>
                You need to realize, however, that this method does show
                changes that are not significant (e.g. a package's size
                changing by a few bytes).
            </para>

            <para>
                A command-line tool called <filename>buildhistory-diff</filename>
                does exist, though, that queries the Git repository and prints just
                the differences that might be significant in human-readable form.
                Here is an example:
                <literallayout class='monospaced'>
     $ ~/poky/poky/scripts/buildhistory-diff . HEAD^
     Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
        /etc/anotherpkg.conf was added
        /sbin/anotherpkg was added
        * (installed-package-names.txt):
        *   anotherpkg was added
     Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
        anotherpkg was added
     packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
        * PR changed from "r0" to "r1"
        * PV changed from "0.1.10" to "0.1.12"
     packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
        * PR changed from "r0" to "r1"
        * PV changed from "0.1.10" to "0.1.12"
                </literallayout>
                <note>
                    The <filename>buildhistory-diff</filename> tool requires
                    the <filename>GitPython</filename> package.
                    Be sure to install it using Pip3 as follows:
                    <literallayout class='monospaced'>
   $ pip3 install GitPython --user
                    </literallayout>
                    Alternatively, you can install
                    <filename>python3-git</filename> using the appropriate
                    distribution package manager (e.g.
                    <filename>apt-get</filename>, <filename>dnf</filename>, or
                    <filename>zipper</filename>).
                </note>
            </para>

            <para>
                To see changes to the build history using a web interface, follow
                the instruction in the <filename>README</filename> file here.
                <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>.
            </para>

            <para>
                Here is a sample screenshot of the interface:
                <imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" />
            </para>
        </section>
    </section>
</section>

<section id='speeding-up-the-build'>
    <title>Speeding Up the Build</title>

    <para>
        Build time can be an issue.
        By default, the build system uses simple controls to try and maximize
        build efficiency.
        In general, the default settings for all the following variables
        result in the most efficient build times when dealing with single
        socket systems (i.e. a single CPU).
        If you have multiple CPUs, you might try increasing the default
        values to gain more speed.
        See the descriptions in the glossary for each variable for more
        information:
        <itemizedlist>
            <listitem><para>
                <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</link>
                The maximum number of threads BitBake simultaneously executes.
                </para></listitem>
            <listitem><para>
                <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink>
                The number of threads BitBake uses during parsing.
                </para></listitem>
            <listitem><para>
                <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</link>
                Extra options passed to the <filename>make</filename> command
                during the
                <link linkend='ref-tasks-compile'><filename>do_compile</filename></link>
                task in order to specify parallel compilation on the
                local build host.
                </para></listitem>
            <listitem><para>
                <link linkend='var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</link>
                Extra options passed to the <filename>make</filename> command
                during the
                <link linkend='ref-tasks-install'><filename>do_install</filename></link>
                task in order to specify parallel installation on the
                local build host.
                </para></listitem>
        </itemizedlist>
        As mentioned, these variables all scale to the number of processor
        cores available on the build system.
        For single socket systems, this auto-scaling ensures that the build
        system fundamentally takes advantage of potential parallel operations
        during the build based on the build machine's capabilities.
    </para>

    <para>
        Following are additional factors that can affect build speed:
        <itemizedlist>
            <listitem><para>
                File system type:
                The file system type that the build is being performed on can
                also influence performance.
                Using <filename>ext4</filename> is recommended as compared
                to <filename>ext2</filename> and <filename>ext3</filename>
                due to <filename>ext4</filename> improved features
                such as extents.
                </para></listitem>
            <listitem><para>
                Disabling the updating of access time using
                <filename>noatime</filename>:
                The <filename>noatime</filename> mount option prevents the
                build system from updating file and directory access times.
                </para></listitem>
            <listitem><para>
                Setting a longer commit:
                Using the "commit=" mount option increases the interval
                in seconds between disk cache writes.
                Changing this interval from the five second default to
                something longer increases the risk of data loss but decreases
                the need to write to the disk, thus increasing the build
                performance.
                </para></listitem>
            <listitem><para>
                Choosing the packaging backend:
                Of the available packaging backends, IPK is the fastest.
                Additionally, selecting a singular packaging backend also
                helps.
                </para></listitem>
            <listitem><para>
                Using <filename>tmpfs</filename> for
                <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
                as a temporary file system:
                While this can help speed up the build, the benefits are
                limited due to the compiler using
                <filename>-pipe</filename>.
                The build system goes to some lengths to avoid
                <filename>sync()</filename> calls into the
                file system on the principle that if there was a significant
                failure, the
                <link linkend='build-directory'>Build Directory</link>
                contents could easily be rebuilt.
                </para></listitem>
            <listitem><para>
                Inheriting the
                <link linkend='ref-classes-rm-work'><filename>rm_work</filename></link>
                class:
                Inheriting this class has shown to speed up builds due to
                significantly lower amounts of data stored in the data
                cache as well as on disk.
                Inheriting this class also makes cleanup of
                <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
                faster, at the expense of being easily able to dive into the
                source code.
                File system maintainers have recommended that the fastest way
                to clean up large numbers of files is to reformat partitions
                rather than delete files due to the linear nature of partitions.
                This, of course, assumes you structure the disk partitions and
                file systems in a way that this is practical.
                </para></listitem>
        </itemizedlist>
        Aside from the previous list, you should keep some trade offs in
        mind that can help you speed up the build:
        <itemizedlist>
            <listitem><para>
                Remove items from
                <link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link>
                that you might not need.
                </para></listitem>
            <listitem><para>
                Exclude debug symbols and other debug information:
                If you do not need these symbols and other debug information,
                disabling the <filename>*-dbg</filename> package generation
                can speed up the build.
                You can disable this generation by setting the
                <link linkend='var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></link>
                variable to "1".
                </para></listitem>
            <listitem><para>
                Disable static library generation for recipes derived from
                <filename>autoconf</filename> or <filename>libtool</filename>:
                Following is an example showing how to disable static
                libraries and still provide an override to handle exceptions:
                <literallayout class='monospaced'>
     STATICLIBCONF = "--disable-static"
     STATICLIBCONF_sqlite3-native = ""
     EXTRA_OECONF += "${STATICLIBCONF}"
                </literallayout>
                <note><title>Notes</title>
                    <itemizedlist>
                        <listitem><para>
                            Some recipes need static libraries in order to work
                            correctly (e.g. <filename>pseudo-native</filename>
                            needs <filename>sqlite3-native</filename>).
                            Overrides, as in the previous example, account for
                            these kinds of exceptions.
                            </para></listitem>
                        <listitem><para>
                            Some packages have packaging code that assumes the
                            presence of the static libraries.
                            If so, you might need to exclude them as well.
                            </para></listitem>
                    </itemizedlist>
                </note>
            </para></listitem>
        </itemizedlist>
    </para>
</section>
</chapter>
<!--
vim: expandtab tw=80 ts=4
-->