%poky; ] > Working with Advanced Metadata
Overview In addition to supporting configuration fragments and patches, the Yocto Project kernel tools also support rich Metadata that you can use to define complex policies and Board Support Package (BSP) support. The purpose of the Metadata and the tools that manage it, known as the kern-tools (kern-tools-native_git.bb), is to help you manage the complexity of the configuration and sources used to support multiple BSPs and Linux kernel types. In particular, the kernel tools allow you to specify only what you must, and nothing more. Where a complete Linux kernel .config includes all the automatically selected CONFIG options, the configuration fragments only need to contain the highest level visible CONFIG options as presented by the Linux kernel menuconfig system. This reduces your maintenance effort and allows you to further separate your configuration in ways that make sense for your project. A common split is policy and hardware. For example, all your kernels might support the proc and sys filesystems, but only specific boards will require sound, USB, or specific drivers. Specifying these individually allows you to aggregate them together as needed, but maintain them in only one place. Similar logic applies to source changes. Original Text: In addition to configuration fragments and patches, the Yocto Project kernel tools support rich metadata which you can use to define complex policies and BSP support. The purpose of the metadata and the tools to manage it, known as the kern-tools (kern-tools-native_git.bb), is to assist in managing the complexity of the configuration and sources in support of multiple BSPs and Linux kernel types. In particular, the kernel tools allow you to specify only what you must, and nothing more. Where a complete Linux kernel .config includes all the automatically selected CONFIG options, the configuration fragments only need to contain the highest level visible CONFIG options as presented by the Linux kernel menuconfig system. This reduces your maintenance effort and allows you to further separate your configuration in ways that make sense for your project. A common split is policy and hardware. For example, all your kernels may support the proc and sys filesystems, but only specific boards will require sound, usb, or specific drivers. Specifying these individually allows you to aggregate them together as needed, but maintain them in only one place. Similar logic applies to source changes.
Using Kernel Metadata in a Recipe The kernel sources in the Yocto Project contain kernel Metadata, which is located in the meta branches of the kernel source Git repositories. This Metadata defines Board Support Packages (BSPs) that correspond to definitions in linux-yocto recipes for the same BSPs. A BSP consists of an aggregation of kernel policy and hardware-specific feature enablement. The BSP can be influenced from within the linux-yocto recipe. Linux kernel source that contains kernel Metadata is said to be "linux-yocto style" kernel source. A Linux kernel recipe that inherits from the linux-yocto.inc include file is said to be a "linux-yocto style" recipe. Every linux-yocto style recipe must define the KMACHINE variable. This variable is typically set to the same value as the MACHINE variable, which is used by BitBake (e.g. "routerstationpro" or "fri2"). Multiple BSPs can reuse the same KMACHINE name if they are built using the same BSP description. The "fri2" and "fri2-noemgd" BSP combination in the meta-intel layer is a good example of two BSPs using the same KMACHINE value (i.e. "fri2"). See the BSP Descriptions section for more information. The linux-yocto style recipes can optionally define the following variables: KBRANCH KERNEL_FEATURES KBRANCH_DEFAULT LINUX_KERNEL_TYPE KBRANCH_DEFAULT defines the Linux kernel source repository's default branch to use to build the Linux kernel. The value is used as the default for KBRANCH, which can define an alternate branch typically with a machine override as follows: KBRANCH_fri2 = "standard/fri2" Unless you specify otherwise, KBRANCH_DEFAULT initializes to "master". LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the configuration. If you do not specify a LINUX_KERNEL_TYPE, it defaults to "standard". Together with KMACHINE, LINUX_KERNEL_TYPE defines the search arguments used by the kernel tools to find the appropriate description within the kernel Metadata with which to build out the sources and configuration. The linux-yocto recipes define "standard", "tiny", and "preempt-rt" kernel types. See the Kernel Types section for more inforation on kernel types. During the build, the kern-tools search for the BSP description file that most closely matches the KMACHINE and LINUX_KERNEL_TYPE variables passed in from the recipe. The tools use the first BSP description it finds that match both variables. If the tools cannot find a match, they issue a warning such as the following: WARNING: Can't find any BSP hardware or required configuration fragments. WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and meta/cfg/broken/fri2-broken/required_frags.txt in directory: meta/cfg/broken/fri2-broken In this example, KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE was set to "broken". The tools first search for the KMACHINE and then for the LINUX_KERNEL_TYPE. If the tools cannot find a partial match, they will use the sources from the KBRANCH and any configuration specified in the SRC_URI. You can use the KERNEL_FEATURES variable to include features (configuration fragments, patches, or both) that are not already included by the KMACHINE and LINUX_KERNEL_TYPE variable combination. For example, to include a feature specified as "features/netfilter.scc", specify: KERNEL_FEATURES += "features/netfilter.scc" To include a feature called "cfg/sound.scc" just for the qemux86 machine, specify: KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc" The value of the entries in KERNEL_FEATURES are dependent on their location within the kernel Metadata itself. The examples here are taken from the linux-yocto-3.4 repository where "features" and "cfg" are subdirectories within the meta/cfg/kernel-cache directory. For more information, see the "Kernel Metadata Syntax" section. The processing of the these variables has evolved some between the 0.9 and 1.3 releases of the Yocto Project and associated kern-tools sources. The descriptions in this section are accurate for 1.3 and later releases of the Yocto Project. Original Text. The metadata provided with any linux-yocto style Linux kernel sources must define a BSP that corresponds to the definition laid out in the recipe. A BSP consists of an aggregation of kernel policy and hardware specific feature enablement. This can be influenced from within the recipe. Every linux-yocto style recipe must define the following variables: KMACHINE KMACHINE is typically set to the same value as used within the recipe-space BSP definition, such as "routerstationpro" or "fri2". However, multiple BSPs can reuse the same KMACHINE name if they are built using the same BSP description (see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples of such a situation where each specifies KMACHINE as "fri2". They may optionally define the following variables: KBRANCH KERNEL_FEATURES KBRANCH_DEFAULT LINUX_KERNEL_TYPE KBRANCH_DEFAULT defines the default source branch within the Linux kernel source repository to be used to build the Linux kernel. It is used as the default value for KBRANCH which may define an alternate branch, typically with a machine override, such as: KBRANCH_fri2 = "standard/fri2" Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master". LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the configuration and defaults to "standard" if you do not specify otherwise. Together with KMACHINE, this defines the search arguments used by the Yocto Project Linux kernel tools to find the appropriate description within the metadata with which to build out the sources and configuration. The linux-yocto recipes define "standard", "tiny", and "preempt-rt" kernel types. See 3.3.4 for more inforation on kernel types. During the build, the kern-tools will search for the BSP description file that most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the recipe. It will use the first BSP description it finds matching both variables. Failing that it will issue a warning such as the following: WARNING: Can't find any BSP hardware or required configuration fragments. WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and meta/cfg/broken/fri2-broken/required_frags.txt in directory: meta/cfg/broken/fri2-broken In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE was set to "broken". It will then search first for the KMACHINE and then for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will use the sources from the KBRANCH and any configuration specified in the SRC_URI. KERNEL_FEATURES can be used to include features (configuration fragments, patches, or both) that are not already included by the KMACHINE and LINUX_KERNEL_TYPE combination. To include a feature specified as "features/netfilter.scc" for example, specify: KERNEL_FEATURES += "features/netfilter.scc" To include a feature called "cfg/sound.scc" just for the qemux86 machine, specify: KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc" The value of the entries in KERNEL_FEATURES are dependent on their location within the metadata itself. The examples here are taken from the linux-yocto-3.4 repository where "features" and "cfg" are subdirectories of the metadata directory. For details, see 3.3. Note: The processing of the these variables has evolved some between the 0.9 and 1.3 releases of the Yocto Project and associated kern-tools sources. The above is accurate for 1.3 and later releases of the Yocto Project.
Kernel Metadata Location Kernel Metadata can be defined in either the kernel recipe (recipe-space) or in the kernel tree (in-tree). Where you choose to define the Metadata depends on what you want to do and how you intend to work. Regardless of where you define the kernel Metadata, the syntax used applies equally. If you are unfamiliar with the Linux kernel and only wish to apply a configuration and possibly a couple of patches provided to you by others, the recipe-space method is recommended. This method is also a good approach if you are working with Linux kernel sources you do not control or if you just do not want to maintain a Linux kernel Git repository on your own. For partial information on how you can define kernel Metadata in the recipe-space, see the "Modifying an Existing Recipe" section. Conversely, if you are actively developing a kernel and are already maintaining a Linux kernel Git repository of your own, you might find it more convenient to work with the kernel Metadata in the same repository as the Linux kernel sources. This method can make iterative development of the Linux kernel more efficient outside of the BitBake environment. Original Text: This meta-data can be defined along with the Linux kernel recipe (recipe-space) as partially described in section 2.2 as well as within the Linux kernel sources themselves (in-tree). Where you choose to store the meta-data depends on what you want to do and how you intend to work. If you are unfamiliar with the Linux kernel and only wish to apply a config and possibly a couple of patches provided to you by others, you may find the recipe-space mechanism to be easier to work with. This is also a good approach if you are working with Linux kernel sources you do not control or if you just don't want to maintain a Linux kernel git repository on your own. If you are doing active kernel development and are already maintaining a Linux kernel git repository of your own, you may find it more convenient to work with the meta-data in the same repository as the Linux kernel sources. This can make iterative development of the Linux kernel more efficient outside of the bitbake environment. Regardless of where the meta-data is stored, the syntax as described in the following sections applies equally.
Recipe-Space Metadata When stored in recipe-space, the kernel Metadata files reside in a directory hierarchy below FILESEXTRAPATHS. For a linux-yocto recipe or for a Linux kernel recipe derived by copying and modifying oe-core/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb to a recipe in your layer, FILESEXTRAPATHS is typically set to ${THISDIR}/${PN}. See the "Modifying an Existing Recipe" section for more information. Here is an example that shows a trivial tree of kernel Metadata stored in recipe-space within a BSP layer: meta-my_bsp_layer/ `-- recipes-kernel `-- linux `-- linux-yocto |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg When the Metadata is stored in recipe-space, you must take steps to ensure BitBake has the necessary information to decide what files to fetch and when they need to be fetched again. It is only necessary to specify the .scc files on the SRC_URI. BitBake parses them and fetches any files referenced in the .scc files by the include, patch, or kconf commands. Because of this, it is necessary to bump the recipe PR value when changing the content of files not explicitly listed in the SRC_URI. Original text: When stored in recipe-space, the meta-data files reside in a directory hierarchy below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a linux-yocto or linux-yocto-custom derived Linux kernel recipe. See 2.2. By way of example, a trivial tree of meta-data stored in recipe-space within a BSP layer might look like the following: meta/ `-- recipes-kernel `-- linux `-- linux-yocto |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg When the meta-data is stored in recipe-space, you must take steps to ensure bitbake has the necessary information to decide which files to fetch and when they need to be fetched again. It is only necessary to specify the .scc files on the SRC_URI; bitbake will parse them and fetch any files referenced in the .scc files by the include, patch, or kconf commands. Because of this, it is necessary to bump the recipe PR value when changing the content of files not explicitly listed in the SRC_URI.
In-Tree Metadata When stored in-tree, the kernel Metadata files reside in the meta directory of the Linux kernel sources. The meta directory can be present in the same repository branch as the sources, such as "master", or meta can be its own orphan branch. An orphan branch in Git is a branch with unique history and content to the other branches in the repository. Orphan branches are useful to track Metadata changes independently from the sources of the Linux kernel, while still keeping them together in the same repository. For the purposes of this document, we will discuss all in-tree Metadata as residing below the meta/cfg/kernel-cache directory. Following is an example that shows how a trivial tree of Metadata is stored in a custom Linux kernel Git repository: meta/ `-- cfg `-- kernel-cache |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg To use a branch different from where the sources reside, specify the branch in the KMETA variable in your Linux kernel recipe. Here is an example: KMETA = "meta" To use the same branch as the sources, set KMETA to an empty string: KMETA = "" If you are working with your own sources and want to create an orphan meta branch, use these commands from within your Linux kernel Git repository: $ git checkout --orphan meta $ git rm -rf . $ git commit --allow-empty -m "Create orphan meta branch" Original text: When stored in-tree, the meta-data files reside in the "meta" directory of the Linux kernel sources. They may be present in the same branch as the sources, such as "master", or in their own orphan branch, typically named "meta". An orphan branch in git is a branch with unique history and content to the other branches in the repository. This is useful to track meta-data changes independently from the sources of the Linux kernel, while still keeping them together in the same repository. For the purposes of this document, we will discuss all in-tree meta-data as residing below the "meta/cfg/kernel-cache" directory. By way of example, a trivial tree of meta-data stored in a custom Linux kernel git repository might look like the following: meta/ `-- cfg `-- kernel-cache |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg To use a specific branch for the meta-data, specify the branch in the KMETA variable in your Linux kernel recipe, for example: KMETA = "meta" To use the same branch as the sources, set KMETA to the empty string: KMETA = "" If you are working with your own sources and want to create an orphan meta branch, you can do so using the following commands from within your Linux kernel git repository: $ git checkout --orphan meta $ git rm -rf . $ git commit --allow-empty -m "Create orphan meta branch"
Kernel Metadata Syntax The kernel Metadata consists of three primary types of files: scc scc stands for Series Configuration Control, but the naming has less significance in the current implementation of the tooling than it had in the past. Consider scc files to be description files. description files, configuration fragments, and patches. The scc files define variables and include or otherwise reference any of the three file types. The description files are used to aggregate all types of kernel Metadata into what ultimately describes the sources and the configuration required to build a Linux kernel tailored to a specific machine. The scc description files are used to define two fundamental types of kernel Metadata: Features Board Support Packages (BSPs) Features aggregate sources in the form of patches and configuration fragments into a modular reusable unit. You can use features to implement conceptually separate kernel Metadata descriptions such as pure configuration fragments, simple patches, complex features, and kernel types. Kernel types define general kernel features and policy to be reused in the BSPs. BSPs define hardware-specific features and aggregate them with kernel types to form the final description of what will be assembled and built. While the kernel Metadata syntax does not enforce any logical separation of configuration fragments, patches, features or kernel types, best practices dictate a logical separation of these types of Metadata. The following Metadata file hierarchy is recommended: <base>/ bsp/ cfg/ features/ ktypes/ patches/ The bsp directory contains the BSP descriptions. The remaining directories all contain "features". Separating bsp from the rest of the structure aids conceptualizing intended usage. Use these guidelines to help place your scc description files within the structure: If your file contains only configuration fragments, place the file in cfg. If your file contains only source-code fixes, place the file in patches. If your file encapsulates a major feature, often combining sources and configurations, place the file in features. If your file aggregates non-hardware configuration and patches in order to define a base kernel policy or major kernel type to be reused across multiple BSPs, place the file in ktypes. These distinctions can easily become blurred - especially as out-of-tree features slowly merge upstream over time. Also, remember that how the description files are placed is a purely logical organization and has no impact on the functionality of the kernel Metadata. There is no impact because all of cfg, features, patches, and ktypes, contain "features" as far as the kernel tools are concerned. Paths used in kernel Metadata files are relative to <base>, which is either FILESEXTRAPATHS if you are creating Metadata in recipe-space, or meta/cfg/kernel-cache/ if you are creating Metadata in-tree. Original text: The Yocto Project Linux kernel tools meta-data consists of three primary types of files: scc* description files, configuration fragments, and patches. The scc files define variables and include or otherwise reference any of the three file types. The description files are used to aggregate all types of meta-data into what ultimately describes the sources and the configuration required to build a Linux kernel tailored to a specific machine. The scc description files are used to define two fundamental types of meta-data: o Features o BSPs Features aggregate sources in the form of patches and configuration in the form of configuration fragments into a modular reusable unit. Features are used to implement conceptually separate meta-data descriptions like pure configuration fragments, simple patches, complex features, and kernel types (ktypes). Kernel types define general kernel features and policy to be reused in the BSPs. BSPs define hardware-specific features and aggregate them with kernel types to form the final description of what will be assembled and built. While the meta-data syntax does not enforce any logical separation of configuration fragments, patches, features or kernel types, best practices dictate a logical separation of these types of meta-data. The following meta-data file hierarchy is recommended: <base>/ bsp/ cfg/ features/ ktypes/ patches/ The bsp directory should contain the BSP descriptions, described in detail in 3.3.5. The remaining directories all contain "features"; the separation is meant to aid in conceptualizing their intended usage. A simple guide to determine where your scc description file should go is as follows. If it contains only configuration fragments, it belongs in cfg. If it contains only source-code fixes, it belongs in patches. If it encapsulates a major feature, often combining sources and configurations, it belongs in features. If it aggregates non-hardware configuration and patches in order to define a base kernel policy or major kernel type to be reused across multiple BSPs, it belongs in ktypes. The line between these can easily become blurred, especially as out-of-tree features are slowly merged upstream over time. Also remember that this is purely logical organization and has no impact on the functionality of the meta-data as all of cfg, features, patches, and ktypes, contain "features" as far as the Yocto Project Linux kernel tools are concerned. Paths used in meta-data files are relative to <base> which is either FILESEXTRAPATHS if you are creating meta-data in recipe-space (see 3.2.1), or meta/cfg/kernel-cache/ if you are creating meta-data in-tree (see 3.2.2). * scc stands for Series Configuration Control, but the naming has less significance in the current implementation of the tooling than it had in the past. Consider it to be a description file.
Configuration The simplest unit of kernel Metadata is the configuration-only feature. This feature consists of one or more Linux kernel configuration parameters in a configuration fragment file (.cfg) and an .scc file that describes the fragment. The Symmetric Multi-Processing (SMP) fragment included in the linux-yocto-3.4 Git repository consists of the following two files: cfg/smp.scc: define KFEATURE_DESCRIPTION "Enable SMP" kconf hardware smp.cfg cfg/smp.cfg: CONFIG_SMP=y CONFIG_SCHED_SMT=y You can find information on configuration fragment files in the "Creating Configuration Fragments" section of the Yocto Project Development Manual and in the "Generating Configuration Files" section earlier in this manual. KFEATURE_DESCRIPTION provides a short description of the fragment. Higher level kernel tools use this description. The kconf command is used to include the actual configuration fragment in an .scc file, and the "hardware" keyword identifies the fragment as being hardware enabling, as opposed to general policy, which would use the "non-hardware" keyword. The distinction is made for the benefit of the configuration validation tools, which warn you if a hardware fragment overrides a policy set by a non-hardware fragment. The description file can include multiple kconf statements, one per fragment. As described in the "Generating Configuration Files" section, you can use the following BitBake command to audit your configuration: $ bitbake linux-yocto -c kernel_configcheck -f Original text: The simplest unit of meta-data is the configuration-only feature. It consists of one or more Linux kernel configuration parameters in a .cfg file (as described in section XYZ) and an scc file describing the fragment. The SMP fragment included in the linux-yocto-3.4 git repository consists of the following two files: cfg/smp.scc: define KFEATURE_DESCRIPTION "Enable SMP" kconf hardware smp.cfg cfg/smp.cfg: CONFIG_SMP=y CONFIG_SCHED_SMT=y See 2.3.1 for details on creating configuration fragments. KFEATURE_DESCRIPTION provides a short description of the fragment, the primary use is for higher level tooling, such as the Yocto Project BSP Tools (TODO:Citation). The "kconf" command is used to include the actual configuration fragment in an scc file, and the "hardware" keyword identifies the fragment as being hardware enabling, as opposed to general policy (which would use the keyword "non-hardware"). The distinction is made for the benefit of the configuration validation tools which will warn you if a hardware fragment overrides a policy set by a non-hardware fragment. As described in 2.3.1, the following bitbake command can be used to audit your configuration: $ bitbake linux-yocto -c kernel_configcheck -f The description file can include multiple kconf statements, one per fragment.
Patches Patch descriptions are very similar to configuration fragment descriptions, which are described in the previous section. However, instead of a .cfg file, these descriptions work with source patches. A typical patch includes a description file and the patch itself: patches/mypatch.scc: patch mypatch.patch patches/mypatch.patch: <typical-patch> You can create the typical .patch file using diff -Nurp or git format-patch. The description file can include multiple patch statements, one per patch. Original text: Patches are described in a very similar way to configuration fragments (see 3.3.1). Instead of a .cfg file, they work with source patches. A typical patch includes a description file and the patch itself: patches/mypatch.scc: patch mypatch.patch patches/mypatch.patch: <typical patch created with 'diff -Nurp' or 'git format-patch'> The description file can include multiple patch statements, one per patch.
Features Features are a combination of configuration fragments and patches. Or, more accurately, configuration fragments and patches are simple forms of a feature, which is a more complex metadata type. In addition to the kconf and patch commands, features often aggregate description files with the include command. A hypothetical example of a feature description file might look like the following: features/myfeature.scc define KFEATURE_DESCRIPTION "Enable myfeature" patch 0001-myfeature-core.patch patch 0002-myfeature-interface.patch include cfg/myfeature_dependency.scc kconf non-hardware myfeature.cfg Features are typically less granular than configuration fragments and are more likely than configurations fragments and patches to be the types of things you will want to specify in the KERNEL_FEATURES variable of the Linux kernel recipe. See the "Using Kernel Metadata in a Recipe" section earlier in the manual. Original text: Features are a combination of configuration fragments and patches, or, more accurately, configuration fragments and patches are simple forms of a feature, a more complex meta-data type. In addition to the kconf and patch commands, features often aggregate description files with the include command. A hypothetical example of a feature description file might look like the following: features/myfeature.scc define KFEATURE_DESCRIPTION "Enable myfeature" patch 0001-myfeature-core.patch patch 0002-myfeature-interface.patch include cfg/myfeature_dependency.scc kconf non-hardware myfeature.cfg Features are typically less granular than configuration fragments and are more likely than configurations fragments and patches to be the types of things you will want to specify in the KERNEL_FEATURES variable of the Linux kernel recipe (see 3.1).
Kernel Types Kernel types, or ktypes, are used to aggregate all non-hardware configuration fragments together with any patches you want to use for all Linux kernel builds of the specified ktype. In short, ktypes are where you define a high-level kernel policy. Syntactically, however, they are no different than features as described in the previous section. The ktype is selected by the LINUX_KERNEL_TYPE variable in the recipe. See the "Using Kernel Metadata in a Recipe" section for more information. By way of example, the linux-yocto-3.4 tree defines three ktypes: standard, tiny, and preempt-rt. standard: Includes the generic Linux kernel policy of the Yocto Project linux-yocto kernel recipes. This includes things like which file systems, which networking options, which core kernel features, and which debugging and tracing options are supported. preempt-rt: Applies the PREEMPT_RT patches and the configuration options required to build a real-time Linux kernel. It inherits from standard. tiny: Independent from the standard configuration and defines a bare minimum configuration meant to serve as a base for very small Linux kernels. Tiny does not currently include any source changes, but it might in the future. The standard kernel type is defined by standard.scc: # Include this kernel type fragment to get the standard features and # configuration values. # Include all standard features include standard-nocfg.scc kconf non-hardware standard.cfg # individual cfg block section include cfg/fs/devtmpfs.scc include cfg/fs/debugfs.scc include cfg/fs/btrfs.scc include cfg/fs/ext2.scc include cfg/fs/ext3.scc include cfg/fs/ext4.scc include cfg/net/ipv6.scc include cfg/net/ip_nf.scc include cfg/net/ip6_nf.scc include cfg/net/bridge.scc As with any scc file, a ktype definition can aggregate other scc files with the include command, or directly pull in configuration fragments and patches with the kconf and patch commands, respectively. It is not strictly necessary to create a ktype scc file. The BSP file can define the ktype implicitly with a define KTYPE myktype line. See the next section for more information. Original text: Kernel types, or ktypes, are used to aggregate all non-hardware configuration fragments together with any patches you want to use for all Linux kernel builds of the specified ktype. In short, ktypes are where you define a high-level kernel policy. Syntactically, however, they are no different than features (see 3.3.3). preempt-rt, and tiny. The ktype is selected by the LINUX_KERNEL_TYPE variable in the recipe (see 3.1). By way of example, the linux-yocto-3.4 tree defines three ktypes: standard, tiny, and preempt-rt. The standard kernel type includes the generic Linux kernel policy of the Yocto Project linux-yocto kernel recipes. This includes things like which filesystems, which networking options, which core kernel features, and which debugging and tracing optoins are supported. The preempt-rt kernel type applies the PREEMPT_RT patches and the configuration options required to build a real-time Linux kernel. It inherits from standard. The tiny kernel type is independent from the standard configuration and defines a bare minimum configuration meant to serve as a base for very small Linux kernels. Tiny does not currently include any source changes, but it may in the future. The standard ktype is defined by standard.scc: # Include this kernel type fragment to get the standard features and # configuration values. # Include all standard features include standard-nocfg.scc kconf non-hardware standard.cfg # individual cfg block section include cfg/fs/devtmpfs.scc include cfg/fs/debugfs.scc include cfg/fs/btrfs.scc include cfg/fs/ext2.scc include cfg/fs/ext3.scc include cfg/fs/ext4.scc include cfg/net/ipv6.scc include cfg/net/ip_nf.scc include cfg/net/ip6_nf.scc include cfg/net/bridge.scc As with any scc file, a ktype definition can aggregate other scc files with the include command, or directly pull in configuration fragments and patches with the kconf and patch commands, respectively. Note: It is not strictly necessary to create a ktype scc file. The BSP file can define the ktype implicitly with a "define KTYPE myktype" line. See 3.3.5.
BSP Descriptions BSP descriptions combine kernel types with hardware-specific features. The hardware specific portion is typically defined independently, and then aggregated with each supported kernel type. Consider a simple example: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 kconf mybsp.cfg Every BSP description should include the definition of the KMACHINE, KTYPE, and KARCH variables. These variables allow the build-system to identify this description as meeting the criteria set by the recipe being built. This particular description can be said to support the "mybsp" machine for the "standard" kernel type and the "i386" architecture. Be aware that there is no hard link between the KTYPE and a ktype description file. If you do not have kernel types defined in your metadata, you only need to ensure that the recipe LINUX_KERNEL_TYPE and the KTYPE here match. Future versions of the tooling make the specification of KTYPE in the BSP optional. If you did want to separate your kernel policy from your hardware configuration, you could do so by specifying a kernel type, such as "standard" (see 3.3.4) and including that description in the BSP description. You might also have multiple hardware configurations that you aggregate into a single hardware description file which you could include here, rather than referencing a single .cfg file. Consider the following: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc include mybsp.scc In the above example, standard.scc aggregates all the configuration fragments, patches, and features that make up your standard kernel policy whereas mybsp.scc aggregates all those necessary to support the hardware available on the mybsp machine. For information on how to break a complete .config into the various, see the "Generating Configuration Files" section. Many real-world examples are more complex. Like any other scc file, BSP descriptions can aggregate features. Consider the Fish River Island II (fri2) BSP definitions from the linux-yocto-3.4 repository: fri2.scc: kconf hardware fri2.cfg include cfg/x86.scc include features/eg20t/eg20t.scc include cfg/dmaengine.scc nclude features/ericsson-3g/f5521gw.scc include features/power/intel.scc include cfg/efi.scc include features/usb/ehci-hcd.scc include features/usb/ohci-hcd.scc include features/iwlwifi/iwlwifi.scc The fri2.scc description file includes a hardware configuration fragment (fri2.cfg) specific to the fri2 BSP as well as several more general configuration fragments and features enabling hardware found on the fri2. This description is then included in each of the three machine-ktype descriptions (standard, preempt-rt, and tiny). Consider the fri2 standard description: fri2-standard.scc: define KMACHINE fri2 define KTYPE standard define KARCH i386 include ktypes/standard/standard.scc branch fri2 git merge emgd-1.14 include fri2.scc # Extra fri2 configs above the minimal defined in fri2.scc include cfg/efi-ext.scc include features/drm-emgd/drm-emgd.scc include cfg/vesafb.scc # default policy for standard kernels include cfg/usb-mass-storage.scc The "include fri2.scc" line about midway through the file defines all hardware enablement common to the BSP for all kernel types. Including the statement significantly reduces duplication. This description introduces a few more variables and commands worthy of further discussion. Notice the "branch" command, which is used to create a machine-specific branch into which source changes can be applied. With this branch set up, the git merge command uses Git to merge in a feature branch "emgd-1.14". This could also be handled with the patch command, but for commonly used features such as this, feature branches can be a convenient mechanism. See the "Feature Branches" section for more information. Now consider the Fish River Island 2 tiny (fri2-tiny) BSP description: fri2-tiny.scc: define KMACHINE fri2 define KTYPE tiny define KARCH i386 include ktypes/tiny/tiny.scc branch fri2 include fri2.scc As you might expect, the tiny description includes quite a bit less. In fact, it includes only the minimal policy defined by the tiny ktype and the hardware-specific configuration required for boot and the most basic functionality of the system as defined in the base fri2 description file. Notice again the three critical variables: KMACHINE, KTYPE, and KARCH. Of these, only the KTYPE has changed. It is now set to "tiny". Original text: BSP descriptions combine kernel types (see 3.3.4) with hardware-specific features (see 3.3.3). The hardware specific portion is typically defined independently, and then aggregated with each supported kernel type. Consider a simple example: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 kconf mybsp.cfg Every BSP description should include the definition of the KMACHINE, KTYPE, and KARCH variables. These variables allow the build-system to identify this description as meeting the criteria set by the recipe being built. This particular description can be said to support the "mybsp" machine for the "standard" kernel type and the "i386" architecture. Note that there is no hard link between the KTYPE and a ktype description file. If you do not have kernel types defined in your meta-data, you only need to ensure that the recipe LINUX_KERNEL_TYPE and the KTYPE here match. NOTE: future versions of the tooling make the specification of KTYPE in the BSP optional. If you did want to separate your kernel policy from your hardware configuration, you could do so by specifying a kernel type, such as "standard" (see 3.3.4) and including that description in the BSP description. You might also have multiple hardware configurations that you aggregate into a single hardware description file which you could include here, rather than referencing a single .cfg file. Consider the following: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc include mybsp.scc In the above example standard.scc aggregates all the configuration fragments, patches, and features that make up your standard kernel policy whereas mybsp.scc aggregates all those necessary to support the hardware available on the mybsp machine. For information on how to break a complete .config into the various fragments, see 2.3.1. Many real-world examples are more complex. Like any other scc file, BSP descriptions can aggregate features. Consider the Fish River Island II (fri2) BSP definitions from the linux-yocto-3.4 repository: fri2.scc: kconf hardware fri2.cfg include cfg/x86.scc include features/eg20t/eg20t.scc include cfg/dmaengine.scc include features/ericsson-3g/f5521gw.scc include features/power/intel.scc include cfg/efi.scc include features/usb/ehci-hcd.scc include features/usb/ohci-hcd.scc include features/iwlwifi/iwlwifi.scc The fri2.scc description file includes a hardware configuration fragment (fri2.cfg) specific to the fri2 BSP as well as several more general configuration fragments and features enabling hardware found on the fri2. This description is then included in each of the three machine-ktype descriptions (standard, preempt-rt, and tiny). Consider the fri2 standard description: fri2-standard.scc: define KMACHINE fri2 define KTYPE standard define KARCH i386 include ktypes/standard/standard.scc branch fri2 git merge emgd-1.14 include fri2.scc # Extra fri2 configs above the minimal defined in fri2.scc include cfg/efi-ext.scc include features/drm-emgd/drm-emgd.scc include cfg/vesafb.scc # default policy for standard kernels include cfg/usb-mass-storage.scc Note the "include fri2.scc" line about midway through the file. By defining all hardware enablement common to the BSP for all kernel types, duplication is significantly reduced. This description introduces a few more variables and commands worthy of further discussion. Note the "branch" command which is used to create a machine-specific branch into which source changes can be applied. With this branch set up, the "git merge" command uses the git SCM to merge in a feature branch "emgd-1.14". This could also be handled with the patch command, but for commonly used features such as this, feature branches can be a convenient mechanism (see 3.5). Next consider the fri2 tiny description: fri2-tiny.scc: define KMACHINE fri2 define KTYPE tiny define KARCH i386 include ktypes/tiny/tiny.scc branch fri2 include fri2.scc As you might expect, the tiny description includes quite a bit less. In fact, it includes only the minimal policy defined by the tiny ktype and the hardware-specific configuration required for boot and the most basic functionality of the system as defined in the base fri2 description file. Note again the three critical variables: KMACHINE, KTYPE, and KARCH. Of these, only the KTYPE has changed, now set to "tiny".
Machine Branches The "Using Kernel Metadata in a Recipe" section introduced the KBRANCH variable, which defines the source branch to use from the Linux kernel Git repository you are using. Many linux-yocto-custom derived recipes will be using Linux kernel sources with only a single branch: "master". However, when you are working with multiple boards and architectures, you are likely to run into the situation where a series of patches are needed for one board to boot. Sometimes, these patches are works-in-progress or fundamentally wrong, yet still necessary for specific boards. In these situations, you most likely do not want to include these patches in every kernel you build. You have a couple of options. First, you could encapsulate these patches in a feature description and only include them in the BSP description for the board(s) that require them. For more information, see the "Patches" and "BSP Descriptions" sections. Alternatively, you can create a branch in your Linux kernel sources and apply the patches there. You can then specify this new branch as the KBRANCH to use for this board. You can do this in the recipe with the KBRANCH variable: KBRANCH = "mynewbranch" or in the BSP description using the "branch" command: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc branch mynewbranch include mybsp.scc The approach you take, feature or branch, is entirely up to you and depends on what works best for your development model. If you are actively working on board support, you may find that working within a branch is more practical than trying to continually reintegrate your patches into a feature. On the other hand, if you are simply reusing some patches from an external tree and are not working on them, you may find the encapsulated feature to be appropriate as it does not require the additional complexity of branching in your Linux kernel sources. If you are supporting multiple boards and architectures and find yourself with numerous branches, you might consider using a hierarchical branching system similar to what the linux-yocto Linux kernel repositories use: <common>/<ktype>/<machine> If you had two ktypes, standard and small for instance, and three machines, your Git tree might look like this: common/base common/standard/base common/standard/machine_a common/standard/machine_b common/standard/machine_c common/small/base common/small/machine_a This organization can help clarify the relationship of the branches to each other. In this case, "common/standard/machine_a" would include everything in "common/base" and "common/standard/base". The "standard" and "small" branches add sources specific to those kernel types that for whatever reason are not appropriate for the other branches. The "base" branches are an artifact of the way Git manages its data internally on the filesystem: it will not allow you to use "common/standard" and "common/standard/machine_a" because it would have to create a file and a directory named "standard". Original text: Section 3.1 introduced the KBRANCH variable which defines the source branch to use from the Linux kernel git repository you are using. Many linux-yocto-custom derived recipes will be using Linux kernel sources with only a single branch: "master". However, when you are working with multiple boards and architectures, you are likely to run into the situation where a series of patches are needed for one board to boot. Sometimes these patches are works in progress or fundamentally wrong, yet still necessary for specific boards. In these situations, you most likely do not want to include these patches in every kernel you build. You have a couple of options. First, you could encapsulate these patches in a feature description and only include them in the BSP description for the board(s) that require them (see 3.3.2 and 3.3.5). Alternatively, you can create a branch in your Linux kernel sources and apply the patches there. You can then specify this new branch as the KBRANCH to use for this board. You can do this in the recipe with the KBRANCH variable: KBRANCH = "mynewbranch" or in the BSP description using the "branch" command: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc branch mynewbranch include mybsp.scc The decision of which approach to take, feature or branch, is entirely up to you and depends on what works best for your development model. If you are actively working on board support, you may find that working within a branch is more practical than trying to continually reintegrate your patches into a feature. On the other hand, if you are simply reusing some patches from an external tree and are not working on them, you may find the encapsulated feature to be appropriate as it does not require the additional complexity of branching in your Linux kernel sources. If you are supporting multiple boards and architectures and find yourself with numerous branches, you might consider using a hierarchical branching system similar to what the linux-yocto Linux kernel repositories use: <common>/<ktype>/<machine> If you had two ktypes, standard and small for instance, and three machines, your git tree might look like this: common/base common/standard/base common/standard/machine_a common/standard/machine_b common/standard/machine_c common/small/base common/small/machine_a This organization can help clarify the relationship of the branches to each other. In this case, "common/standard/machine_a" would include everything in "common/base" and "common/standard/base". The "standard" and "small" branches add sources specific to those kernel types that for whatever reason are not appropriate for the other branches. Note: The "base" branches are an artifact of the way git manages its data internally on the filesystem: it will not allow you to use "common/standard" and "common/standard/machine_a" because it would have to create a file and a directory named "standard".
Feature Branches During active development a new feature, it can be more efficient to work with that feature as a branch, rather than as a set of patches which have to be regularly updated. The Yocto Project Linux kernel tools provide for this with the git merge command. To merge a feature branch into a BSP, insert the git merge command after any branch commands: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc branch mynewbranch git merge myfeature include mybsp.scc Original text: During active development a new feature, it can be more efficient to work with that feature as a branch, rather than as a set of patches which have to be regularly updated. The Yocto Project Linux kernel tools provide for this with the "git merge" command. To merge a feature branch into a BSP, insert the "git merge" command after any branch commands: mybsp.scc: define KMACHINE mybsp define KTYPE standard define KARCH i386 include standard.scc branch mynewbranch git merge myfeature include mybsp.scc
SCC Reference This section provides short descriptions for the commands you can use from an .scc: branch [ref]: Creates a new branch relative to the current branch (typically ${KTYPE}) using the currently checked-out branch, or "ref" if specified. TODO: Bruce, we need to clarify the "relative to the current branch" bit. define: Defines variables, such as KMACHINE, KTYPE, KARCH, and KFEATURE_DESCRIPTION. include SCC_FILE: Includes an scc file in the current file. It will be parsed as if inserted inline. kconf [hardware|non-hardware] CFG_FILE: Queues a configuration fragment for merging into the final Linux .config file. git merge GIT_BRANCH: Merges the feature branch into the current branch. patch PATCH_FILE: Applies the patch to the current Git branch. Original text: * branch [ref] Create a new branch relative to the current branch (typically ${KTYPE}) using the currently checked-out branch, or "ref" if specified. TODO: Bruce, we need to clarify the "relative to the current branch" bit. * define Define variables, such as KMACHINE, KTYPE, KARCH, and KFEATURE_DESCRIPTION. * include SCC_FILE Include an scc file in the current file. It will be parsed as if inserted inline. * kconf [hardware|non-hardware] CFG_FILE Queue a configuration fragment for merging into the final Linux .config file. * merge (or "git merge") GIT_BRANCH Merge the feature branch into the current branch. * patch PATCH_FILE Apply the patch to the current git branch.