path: root/documentation/poky-ref-manual/ref-structure.xml

<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] >

<chapter id='ref-structure'>

<title>Source Directory Structure</title>

<para>
    The <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink> consists of several components.
    Understanding them and knowing where they are located is key to using the Yocto Project well.
    This chapter describes the Source Directory and gives information about the various 
    files and directories.
</para>

<para>
    For information on how to establish a local Source Directory on your development system, see the
    "<ulink url='&YOCTO_DOCS_DEV_URL;#getting-setup'>Getting Set Up</ulink>"
    section in the Yocto Project Development Manual.
</para>

<note>
    The OpenEmbedded build system does not support file or directory names that
    contain spaces.
    Be sure that the Source Directory you use does not contain these types
    of names.
</note>

<section id='structure-core'>
    <title>Top level core components</title>

    <section id='structure-core-bitbake'>
        <title><filename>bitbake/</filename></title>

        <para>
            The <ulink url='source-directory'>Source Directory</ulink>
            includes a copy of BitBake for ease of use.
            The copy usually matches the current stable BitBake release from the BitBake project. 
            BitBake, a metadata interpreter, reads the Yocto Project metadata and runs the tasks 
            defined by that data. 
            Failures are usually from the metadata and not from BitBake itself.
            Consequently, most users do not need to worry about BitBake.
        </para>

        <para>
            When you run the <filename>bitbake</filename> command, the wrapper script in 
            <filename>scripts/</filename> is executed to run the main BitBake executable, 
            which resides in the <filename>bitbake/bin/</filename> directory.
            Sourcing the <link linkend="structure-core-script">&OE_INIT_FILE;</link> 
            script places the <filename>scripts</filename> and <filename>bitbake/bin</filename>
            directories (in that order) into the shell's <filename>PATH</filename> environment 
            variable.
        </para>

        <para>
            For more information on BitBake, see the BitBake documentation 
            inculded in the <filename>bitbake/doc/manual</filename> directory of the 
            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
        </para>
    </section>

    <section id='structure-core-build'>
        <title><filename>build/</filename></title>

        <para>
            This directory contains user configuration files and the output 
            generated by the OpenEmbedded build system in its standard configuration where 
            the source tree is combined with the output.
            The <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
            is created initially when you <filename>source</filename>
            the OpenEmbedded build environment setup script <filename>&OE_INIT_FILE;</filename>.
        </para>

        <para> 
            It is also possible to place output and configuration 
            files in a directory separate from the 
            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>
            by providing a directory name when you <filename>source</filename>
            the setup script.
            For information on separating output from your local Source Directory files, see <link 
            linkend='structure-core-script'>&OE_INIT_FILE;</link>.
        </para>
    </section>

    <section id='handbook'>
        <title><filename>documentation</filename></title>

        <para>
            This directory holds the source for the Yocto Project documentation
            as well as templates and tools that allow you to generate PDF and HTML
            versions of the manuals.  
            Each manual is contained in a sub-folder.  
            For example, the files for this manual reside in 
            <filename>poky-ref-manual</filename>.
        </para>
    </section>

    <section id='structure-core-meta'>
        <title><filename>meta/</filename></title>

        <para>
            This directory contains the OpenEmbedded Core metadata. 
            The directory holds recipes, common classes, and machine
            configuration for emulated targets (qemux86, qemuarm,
            and so on.)
        </para>
    </section>

    <section id='structure-core-meta-yocto'>
        <title><filename>meta-yocto/</filename></title>

        <para>
            This directory contains the configuration for the Poky
            reference distribution.
        </para>
    </section>

    <section id='structure-core-meta-yocto-bsp'>
        <title><filename>meta-yocto-bsp/</filename></title>

        <para>
            This directory contains the Yocto Project reference
            hardware BSPs.
        </para>
    </section>

    <section id='structure-meta-hob'>
        <title><filename>meta-hob/</filename></title>

        <para>
            This directory contains template recipes used by the
            <ulink url='&YOCTO_HOME_URL;/projects/hob'>Hob</ulink>
            build UI.
        </para>
    </section>

    <section id='structure-meta-skeleton'>
        <title><filename>meta-skeleton/</filename></title>

        <para>
            This directory contains template recipes for BSP and kernel development. 
        </para>
    </section>

    <section id='structure-core-scripts'>
        <title><filename>scripts/</filename></title>

        <para>
            This directory contains various integration scripts that implement 
            extra functionality in the Yocto Project environment (e.g. QEMU scripts).
            The <link linkend="structure-core-script">&OE_INIT_FILE;</link> script appends this
            directory to the shell's <filename>PATH</filename> environment variable.
        </para>

        <para>
            The <filename>scripts</filename> directory has useful scripts that assist contributing
            back to the Yocto Project, such as <filename>create_pull_request</filename> and 
            <filename>send_pull_request</filename>.
        </para>
    </section>

    <section id='structure-core-script'>
        <title><filename>&OE_INIT_FILE;</filename></title>

        <para>
            This script sets up the OpenEmbedded build environment. 
            Running this script with the <filename>source</filename> command in
            a shell makes changes to <filename>PATH</filename> and sets other core BitBake variables based on the
            current working directory. 
            You need to run this script before running BitBake commands.
            The script uses other scripts within the <filename>scripts</filename> directory to do 
            the bulk of the work.
        </para>

        <para>
            By default, running this script without a Build Directory argument creates the 
            <filename>build</filename> directory. 
            If you provide a Build Directory argument when you <filename>source</filename>
            the script, you direct OpenEmbedded build system to create a 
            <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink> of your choice.
            For example, the following command creates a Build Directory named 
            <filename>mybuilds</filename> that is outside of the 
            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>:
            <literallayout class='monospaced'>
     $ source &OE_INIT_FILE; ~/mybuilds
            </literallayout>
            <note>
                The OpenEmbedded build system does not support file or directory names that
                contain spaces.
                If you attempt to run the <filename>&OE_INIT_FILE;</filename> script
                from a Source Directory that contains spaces in either the filenames
                or directory names, the script returns an error indicating no such 
                file or directory.
                Be sure to use a Source Directory free of names containing spaces.
            </note>
        </para>
    </section>

    <section id='structure-basic-top-level'>
        <title><filename>LICENSE, README, and README.hardware</filename></title>

        <para>
            These files are standard top-level files. 
        </para>
    </section>
</section>

<section id='structure-build'>
    <title>The Build Directory - <filename>build/</filename></title>

    <section id='structure-build-pseudodone'>
        <title><filename>build/pseudodone</filename></title>

        <para>
            This tag file indicates that the initial pseudo binary was created. 
            The file is built the first time BitBake is invoked. 
        </para>
    </section>

    <section id='structure-build-conf-local.conf'>
        <title><filename>build/conf/local.conf</filename></title>

        <para>
            This file contains all the local user configuration for your build environment. 
            If there is no <filename>local.conf</filename> present, it is created from 
            <filename>local.conf.sample</filename>. 
            The <filename>local.conf</filename> file contains documentation on the various configuration options.  
            Any variable set here overrides any variable set elsewhere within the environment unless 
            that variable is hard-coded within a file (e.g. by using '=' instead of '?='). 
            Some variables are hard-coded for various reasons but these variables are 
            relatively rare.
        </para>

        <para>
            Edit this file to set the <filename><link linkend='var-MACHINE'>MACHINE</link></filename> 
            for which you want to build, which package types you wish to use 
            (<link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>), 
            where you want to downloaded files
            (<filename><link linkend='var-DL_DIR'>DL_DIR</link></filename>), 
            and how you want your host machine to use resources 
            (<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link> and 
            <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>).
        </para>
    </section>

    <section id='structure-build-conf-bblayers.conf'>
        <title><filename>build/conf/bblayers.conf</filename></title>

        <para>
            This file defines layers, which are directory trees, traversed (or walked) by BitBake.
            If <filename>bblayers.conf</filename> 
            is not present, it is created from <filename>bblayers.conf.sample</filename> when 
            you <filename>source</filename> the environment setup script.
        </para>

        <para>
            The <filename>bblayers.conf</filename> file uses the
            <link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link> variable to
            list the layers BitBake tries to find.
            The file uses the
            <link linkend='var-BBLAYERS_NON_REMOVABLE'><filename>BBLAYERS_NON_REMOVABLE</filename></link>
            variable to list layers that must not be removed.
        </para>
    </section>

    <section id='structure-build-conf-sanity_info'>
        <title><filename>build/conf/sanity_info</filename></title>

        <para>
            This file is created during the build to indicate the state of the sanity checks.
        </para>
    </section>

    <section id='structure-build-downloads'>
        <title><filename>build/downloads/</filename></title>

        <para>
            This directory is used for the upstream source tarballs.
            The directory can be reused by multiple builds or moved to another location. 
            You can control the location of this directory through the
            <filename><link linkend='var-DL_DIR'>DL_DIR</link></filename> variable.
        </para>
    </section>

    <section id='structure-build-sstate-cache'>
        <title><filename>build/sstate-cache/</filename></title>

        <para>
            This directory is used for the shared state cache.
            The directory can be reused by multiple builds or moved to another location. 
            You can control the location of this directory through the
            <filename><link linkend='var-SSTATE_DIR'>SSTATE_DIR</link></filename> variable.
        </para>
    </section>

    <section id='structure-build-tmp'>
        <title><filename>build/tmp/</filename></title>

        <para>
            This directory receives all the OpenEmbedded build system's output.
            BitBake creates this directory if it does not exist. 
            As a last resort, to clean up a build and start it from scratch (other than the downloads), 
            you can remove everything in the <filename>tmp</filename> directory or get rid of the 
            directory completely.
            If you do, you should also completely remove the <filename>build/sstate-cache</filename>
            directory as well.
        </para>
    </section>

    <section id='structure-build-tmp-buildstats'>
        <title><filename>build/tmp/buildstats/</filename></title>

        <para>
            This directory stores the build statistics.
        </para>
    </section>

    <section id='structure-build-tmp-cache'>
        <title><filename>build/tmp/cache/</filename></title>

        <para>
            When BitBake parses the metadata, it creates a cache file of the result that can
            be used when subsequently running commands. 
            These results are stored here on a per-machine basis.
        </para>
    </section>

    <section id='structure-build-tmp-deploy'>
        <title><filename>build/tmp/deploy/</filename></title>

        <para>
            This directory contains any 'end result' output from the OpenEmbedded build process.
        </para>
    </section>

    <section id='structure-build-tmp-deploy-deb'>
        <title><filename>build/tmp/deploy/deb/</filename></title>

        <para>
            This directory receives any <filename>.deb</filename> packages produced by 
            the build process.
            The packages are sorted into feeds for different architecture types.
        </para>
    </section>

    <section id='structure-build-tmp-deploy-rpm'>
        <title><filename>build/tmp/deploy/rpm/</filename></title>

        <para>
            This directory receives any <filename>.rpm</filename> packages produced by 
            the build process.  
            The packages are sorted into feeds for different architecture types.
        </para>
    </section>

    <section id='structure-build-tmp-deploy-licenses'>
        <title><filename>build/tmp/deploy/licenses/</filename></title>

        <para>
            This directory receives package licensing information.
            For example, the directory contains sub-directories for <filename>bash</filename>,
            <filename>busybox</filename>, and <filename>eglibc</filename> (among others) that in turn
            contain appropriate <filename>COPYING</filename> license files with other licensing information.
        </para>
    </section>

    <section id='structure-build-tmp-deploy-images'>
        <title><filename>build/tmp/deploy/images/</filename></title>

        <para>
            This directory receives complete filesystem images. 
            If you want to flash the resulting image from a build onto a device, look here for the image.
        </para>

        <para>
            Be careful when deleting files in this directory. 
            You can safely delete old images from this directory (e.g. 
            <filename>core-image-*</filename>, <filename>hob-image-*</filename>,
            etc.). 
            However, the kernel (<filename>*zImage*</filename>, <filename>*uImage*</filename>, etc.), 
            bootloader and other supplementary files might be deployed here prior to building an
            image.
            Because these files, however, are not directly produced from the image, if you
            delete them they will not be automatically re-created when you build the image again.
        </para>

        <para>
            If you do accidentally delete files here, you will need to force them to be 
            re-created. 
            In order to do that, you will need to know the target that produced them.
            For example, these commands rebuild and re-create the kernel files:
            <literallayout class='monospaced'>
     $ bitbake -c clean virtual/kernel
     $ bitbake virtual/kernel
            </literallayout>
        </para>
    </section>

    <section id='structure-build-tmp-deploy-ipk'>
        <title><filename>build/tmp/deploy/ipk/</filename></title>

        <para>
            This directory receives <filename>.ipk</filename> packages produced by 
            the build process.</para>
    </section>

    <section id='structure-build-tmp-sysroots'>
        <title><filename>build/tmp/sysroots/</filename></title>

        <para>
            This directory contains shared header files and libraries as well as other shared 
            data.  
            Packages that need to share output with other packages do so within this directory. 
            The directory is subdivided by architecture so multiple builds can run within
            the one Build Directory.
        </para>
    </section>

    <section id='structure-build-tmp-stamps'>
        <title><filename>build/tmp/stamps/</filename></title>

        <para>
            This directory holds information that that BitBake uses for accounting purposes 
            to track what tasks have run and when they have run.
            The directory is sub-divided by architecture. 
            The files in the directory are empty of data.
            However, BitBake uses the filenames and timestamps for tracking purposes.
        </para>
    </section>

    <section id='structure-build-tmp-log'>
        <title><filename>build/tmp/log/</filename></title>

        <para>
            This directory contains general logs that are not otherwise placed using the 
            package's <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>.
            Examples of logs are the output from the <filename>check_pkg</filename> or 
            <filename>distro_check</filename> tasks.
            Running a build does not necessarily mean this directory is created.
        </para>
    </section>

    <section id='structure-build-tmp-pkgdata'>
        <title><filename>build/tmp/pkgdata/</filename></title>

        <para>
            This directory contains intermediate packaging data that is used later in the packaging process. 
            For more information, see the "<link linkend='ref-classes-package'>Packaging - package*.bbclass</link>" section.
        </para>
    </section>

    <section id='structure-build-tmp-work'>
        <title><filename>build/tmp/work/</filename></title>

        <para>
            This directory contains architecture-specific work sub-directories for packages built by BitBake. 
            All tasks execute from a work directory.
            For example, the source for a particular package is unpacked, patched, configured and compiled all
            within its own work directory.
            Within the work directory, organization is based on the package group for which the source
            is being compiled.
        </para>

        <para>
            It is worth considering the structure of a typical work directory. 
            As an example, consider the <filename>linux-yocto-kernel-3.0</filename>
            on the machine <filename>qemux86</filename> 
            built within the Yocto Project.  
            For this package, a work directory of 
            <filename>tmp/work/qemux86-poky-linux/linux-yocto-3.0+git1+&lt;.....&gt;</filename>, 
            referred to as <filename><link linkend='var-WORKDIR'>WORKDIR</link></filename>, is created.  
            Within this directory, the source is unpacked to 
            <filename>linux-qemux86-standard-build</filename> and then patched by Quilt 
            (see the 
            "<ulink url='&YOCTO_DOCS_DEV_URL;#using-a-quilt-workflow'>Modifying Package 
            Source Code with Quilt</ulink>" section in the Yocto Project Development Manual.   
            Within the <filename>linux-qemux86-standard-build</filename> directory, 
            standard Quilt directories <filename>linux-3.0/patches</filename>
            and <filename>linux-3.0/.pc</filename> are created,
            and standard Quilt commands can be used.
        </para>

        <para>
            There are other directories generated within WORKDIR. 
            The most important directory is WORKDIR<filename>/temp/</filename>, which has log files for each 
            task (<filename>log.do_*.pid</filename>) and contains the scripts BitBake runs for 
            each task (<filename>run.do_*.pid</filename>). 
            The WORKDIR<filename>/image/</filename> directory is where "make 
            install" places its output that is then split into sub-packages 
            within WORKDIR<filename>/packages-split/</filename>.
        </para>
    </section>
</section>

<section id='structure-meta'>
    <title>The Metadata - <filename>meta/</filename></title>

    <para>
        As mentioned previously, metadata is the core of the Yocto Project. 
        Metadata has several important subdivisions:
    </para>

    <section id='structure-meta-classes'>
        <title><filename>meta/classes/</filename></title>

        <para>
            This directory contains the <filename>*.bbclass</filename> files. 
            Class files are used to abstract common code so it can be reused by multiple 
            packages. 
            Every package inherits the <filename>base.bbclass</filename> file.
            Examples of other important classes are <filename>autotools.bbclass</filename>, which 
            in theory allows any Autotool-enabled package to work with the Yocto Project with minimal effort.
            Another example is <filename>kernel.bbclass</filename> that contains common code and functions 
            for working with the Linux kernel. 
            Functions like image generation or packaging also have their specific class files 
            such as <filename>image.bbclass</filename>, <filename>rootfs_*.bbclass</filename> and 
            <filename>package*.bbclass</filename>.
        </para>
    </section>

    <section id='structure-meta-conf'>
        <title><filename>meta/conf/</filename></title>

        <para>
            This directory contains the core set of configuration files that start from 
            <filename>bitbake.conf</filename> and from which all other configuration 
            files are included.
            See the include statements at the end of the file and you will note that even 
            <filename>local.conf</filename> is loaded from there. 
            While <filename>bitbake.conf</filename> sets up the defaults, you can often override 
            these by using the (<filename>local.conf</filename>) file, machine file or 
            the distribution configuration file.
        </para>
    </section>

    <section id='structure-meta-conf-machine'>
        <title><filename>meta/conf/machine/</filename></title>

        <para>
            This directory contains all the machine configuration files. 
            If you set <filename>MACHINE="qemux86"</filename>, 
            the OpenEmbedded build system looks for a <filename>qemux86.conf</filename> file in this 
            directory. 
            The <filename>include</filename> directory contains various data common to multiple machines. 
            If you want to add support for a new machine to the Yocto Project, look in this directory.
        </para>
    </section>

    <section id='structure-meta-conf-distro'>
        <title><filename>meta/conf/distro/</filename></title>

        <para>
            Any distribution-specific configuration is controlled from this directory. 
            For the Yocto Project, the <filename>defaultsetup.conf</filename> is the main file here.  
            This directory includes the versions and the 
            <filename>SRCDATE</filename> definitions for applications that are configured here. 
            An example of an alternative configuration might be <filename>poky-bleeding.conf</filename>.
            Although this file mainly inherits its configuration from Poky.
        </para>
    </section>

    <section id='structure-meta-recipes-bsp'>
        <title><filename>meta/recipes-bsp/</filename></title>

        <para>
            This directory contains anything linking to specific hardware or hardware 
            configuration information such as "u-boot" and "grub".
        </para>
    </section>

    <section id='structure-meta-recipes-connectivity'>
        <title><filename>meta/recipes-connectivity/</filename></title>

        <para>
            This directory contains libraries and applications related to communication with other devices.
        </para>
    </section>

    <section id='structure-meta-recipes-core'>
        <title><filename>meta/recipes-core/</filename></title>

        <para>
            This directory contains what is needed to build a basic working Linux image 
            including commonly used dependencies.
        </para>
    </section>

    <section id='structure-meta-recipes-devtools'>
        <title><filename>meta/recipes-devtools/</filename></title>

        <para>
            This directory contains tools that are primarily used by the build system.
            The tools, however, can also be used on targets.
        </para>
    </section>

    <section id='structure-meta-recipes-extended'>
        <title><filename>meta/recipes-extended/</filename></title>

        <para>
            This directory contains non-essential applications that add features compared to the 
            alternatives in core. 
            You might need this directory for full tool functionality or for Linux Standard Base (LSB)
            compliance.
        </para>
    </section>

    <section id='structure-meta-recipes-gnome'>
        <title><filename>meta/recipes-gnome/</filename></title>

        <para>
            This directory contains all things related to the GTK+ application framework.
        </para>
    </section>

    <section id='structure-meta-recipes-graphics'>
        <title><filename>meta/recipes-graphics/</filename></title>

        <para>
            This directory contains X and other graphically related system libraries
        </para>
    </section>

    <section id='structure-meta-recipes-kernel'>
        <title><filename>meta/recipes-kernel/</filename></title>

        <para>
            This directory contains the kernel and generic applications and libraries that 
            have strong kernel dependencies.
        </para>
    </section>

    <section id='structure-meta-recipes-multimedia'>
        <title><filename>meta/recipes-multimedia/</filename></title>

        <para>
            This directory contains codecs and support utilities for audio, images and video.
        </para>
    </section>

    <section id='structure-meta-recipes-qt'>
        <title><filename>meta/recipes-qt/</filename></title>

        <para>
            This directory contains all things related to the Qt application framework.
        </para>
    </section>

    <section id='structure-meta-recipes-rt'>
        <title><filename>meta/recipes-rt/</filename></title>

        <para>
            This directory contains package and image recipes for using and testing
            the <filename>PREEMPT_RT</filename> kernel. 
        </para>
    </section>

    <section id='structure-meta-recipes-sato'>
        <title><filename>meta/recipes-sato/</filename></title>

        <para>
            This directory contains the Sato demo/reference UI/UX and its associated applications
            and configuration data.
        </para>
    </section>

    <section id='structure-meta-recipes-support'>
        <title><filename>meta/recipes-support/</filename></title>

        <para>
            This directory contains recipes that used by other recipes, but that are not directly 
            included in images (i.e. dependencies of other recipes).
        </para>
    </section>

    <section id='structure-meta-site'>
        <title><filename>meta/site/</filename></title>

        <para>
            This directory contains a list of cached results for various architectures.
            Because certain "autoconf" test results cannot be determined when cross-compiling due to 
            the tests not able to run on a live system, the information in this directory is 
            passed to "autoconf" for the various architectures. 
        </para>
    </section>

    <section id='structure-meta-recipes-txt'>
        <title><filename>meta/recipes.txt</filename></title>

        <para>
            This file is a description of the contents of <filename>recipes-*</filename>.
        </para>
    </section>
</section>

</chapter>
<!-- 
vim: expandtab tw=80 ts=4 
-->