%poky; ] > Working with Advanced Metadata In addition to configuration fragments and patches, the Yocto Project kernel tools support rich metadata that you can use to define complex policies and BSP support. The purpose of the metadata and the tools to manage it, known as the kern-tools (kern-tools-native_git.bb), is to assist in managing the complexity of the configuration and sources in support of multiple Board Support Packages (BSPs) and Linux kernel types. In particular, the kernel tools allow you to specify only what you must, and nothing more. Where a complete Linux kernel .config includes all the automatically selected CONFIG options, the configuration fragments only need to contain the highest level visible CONFIG options as presented by the Linux kernel menuconfig system. This reduces your maintenance effort and allows you to further separate your configuration in ways that make sense for your project. A common split is policy and hardware. For example, all your kernels might support the proc and sys filesystems, but only specific boards will require sound, USB, or specific drivers. Specifying these individually allows you to aggregate them together as needed, but maintain them in only one place. Similar logic applies to source changes. Original Text: In addition to configuration fragments and patches, the Yocto Project kernel tools support rich metadata which you can use to define complex policies and BSP support. The purpose of the metadata and the tools to manage it, known as the kern-tools (kern-tools-native_git.bb), is to assist in managing the complexity of the configuration and sources in support of multiple BSPs and Linux kernel types. In particular, the kernel tools allow you to specify only what you must, and nothing more. Where a complete Linux kernel .config includes all the automatically selected CONFIG options, the configuration fragments only need to contain the highest level visible CONFIG options as presented by the Linux kernel menuconfig system. This reduces your maintenance effort and allows you to further separate your configuration in ways that make sense for your project. A common split is policy and hardware. For example, all your kernels may support the proc and sys filesystems, but only specific boards will require sound, usb, or specific drivers. Specifying these individually allows you to aggregate them together as needed, but maintain them in only one place. Similar logic applies to source changes.
Using Metadata in a Recipe The metadata provided with any linux-yocto style Linux kernel sources must define a BSP that corresponds to the definition laid out in the recipe. A BSP consists of an aggregation of kernel policy and hardware specific feature enablement. This can be influenced from within the recipe. Every linux-yocto style recipe must define the following variable: KMACHINE KMACHINE is typically set to the same value as used within the recipe-space BSP definition, such as "routerstationpro" or "fri2". However, multiple BSPs can reuse the same KMACHINE name if they are built using the same BSP description. See section 3.3.5 for more information. The meta-intel "fri2" and "fri2-noemgd" are good examples of such a situation where each specifies KMACHINE as "fri2". They may optionally define the following variables: KBRANCH KERNEL_FEATURES KBRANCH_DEFAULT LINUX_KERNEL_TYPE KBRANCH_DEFAULT defines the default source branch within the Linux kernel source repository to be used to build the Linux kernel. It is used as the default value for KBRANCH which may define an alternate branch, typically with a machine override, such as: KBRANCH_fri2 = "standard/fri2" Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master". LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the configuration and defaults to "standard" if you do not specify otherwise. Together with KMACHINE, this defines the search arguments used by the Yocto Project Linux kernel tools to find the appropriate description within the metadata with which to build out the sources and configuration. The linux-yocto recipes define "standard", "tiny", and "preempt-rt" kernel types. See section 3.3.4 for more inforation on kernel types. During the build, the kern-tools will search for the BSP description file that most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the recipe. It will use the first BSP description it finds matching both variables. Failing that it will issue a warning such as the following: WARNING: Can't find any BSP hardware or required configuration fragments. WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and meta/cfg/broken/fri2-broken/required_frags.txt in directory: meta/cfg/broken/fri2-broken In this example, KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE was set to "broken". It will then search first for the KMACHINE and then for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will use the sources from the KBRANCH and any configuration specified in the SRC_URI. KERNEL_FEATURES can be used to include features (configuration fragments, patches, or both) that are not already included by the KMACHINE and LINUX_KERNEL_TYPE combination. To include a feature specified as "features/netfilter.scc" for example, specify: KERNEL_FEATURES += "features/netfilter.scc" To include a feature called "cfg/sound.scc" just for the qemux86 machine, specify: KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc" The value of the entries in KERNEL_FEATURES are dependent on their location within the metadata itself. The examples here are taken from the linux-yocto-3.4 repository where "features" and "cfg" are subdirectories of the metadata directory. For details, see section 3.3. The processing of the these variables has evolved some between the 0.9 and 1.3 releases of the Yocto Project and associated kern-tools sources. The descriptions in this section are accurate for 1.3 and later releases of the Yocto Project. Original Text. The metadata provided with any linux-yocto style Linux kernel sources must define a BSP that corresponds to the definition laid out in the recipe. A BSP consists of an aggregation of kernel policy and hardware specific feature enablement. This can be influenced from within the recipe. Every linux-yocto style recipe must define the following variables: KMACHINE KMACHINE is typically set to the same value as used within the recipe-space BSP definition, such as "routerstationpro" or "fri2". However, multiple BSPs can reuse the same KMACHINE name if they are built using the same BSP description (see 3.3.5). The meta-intel "fri2" and "fri2-noemgd" are good examples of such a situation where each specifies KMACHINE as "fri2". They may optionally define the following variables: KBRANCH KERNEL_FEATURES KBRANCH_DEFAULT LINUX_KERNEL_TYPE KBRANCH_DEFAULT defines the default source branch within the Linux kernel source repository to be used to build the Linux kernel. It is used as the default value for KBRANCH which may define an alternate branch, typically with a machine override, such as: KBRANCH_fri2 = "standard/fri2" Unless you specify otherwise, KBRANCH_DEFAULT is initialized to "master". LINUX_KERNEL_TYPE defines the kernel type to be used in assembling the configuration and defaults to "standard" if you do not specify otherwise. Together with KMACHINE, this defines the search arguments used by the Yocto Project Linux kernel tools to find the appropriate description within the metadata with which to build out the sources and configuration. The linux-yocto recipes define "standard", "tiny", and "preempt-rt" kernel types. See 3.3.4 for more inforation on kernel types. During the build, the kern-tools will search for the BSP description file that most closely matches the KMACHINE and LINUX_KERNEL_TYPE passed in from the recipe. It will use the first BSP description it finds matching both variables. Failing that it will issue a warning such as the following: WARNING: Can't find any BSP hardware or required configuration fragments. WARNING: Looked at meta/cfg/broken/fri2-broken/hdw_frags.txt and meta/cfg/broken/fri2-broken/required_frags.txt in directory: meta/cfg/broken/fri2-broken In this example KMACHINE was set to "fri2-broken" and LINUX_KERNEL_TYPE was set to "broken". It will then search first for the KMACHINE and then for the LINUX_KERNEL_TYPE. If it cannot find a partial match, it will use the sources from the KBRANCH and any configuration specified in the SRC_URI. KERNEL_FEATURES can be used to include features (configuration fragments, patches, or both) that are not already included by the KMACHINE and LINUX_KERNEL_TYPE combination. To include a feature specified as "features/netfilter.scc" for example, specify: KERNEL_FEATURES += "features/netfilter.scc" To include a feature called "cfg/sound.scc" just for the qemux86 machine, specify: KERNEL_FEATURES_append_qemux86 = "cfg/sound.scc" The value of the entries in KERNEL_FEATURES are dependent on their location within the metadata itself. The examples here are taken from the linux-yocto-3.4 repository where "features" and "cfg" are subdirectories of the metadata directory. For details, see 3.3. Note: The processing of the these variables has evolved some between the 0.9 and 1.3 releases of the Yocto Project and associated kern-tools sources. The above is accurate for 1.3 and later releases of the Yocto Project.
Metadata Location This metadata can be defined along with the Linux kernel recipe (recipe-space) as partially described in the "Modifying an Existing Recipe" section as well as within the Linux kernel sources themselves (in-tree). Where you choose to store the metadata depends on what you want to do and how you intend to work. If you are unfamiliar with the Linux kernel and only wish to apply a config and possibly a couple of patches provided to you by others, you may find the recipe-space mechanism to be easier to work with. This is also a good approach if you are working with Linux kernel sources you do not control or if you just don't want to maintain a Linux kernel git repository on your own. If you are doing active kernel development and are already maintaining a Linux kernel git repository of your own, you may find it more convenient to work with the metadata in the same repository as the Linux kernel sources. This can make iterative development of the Linux kernel more efficient outside of the BitBake environment. Regardless of where the meta-data is stored, the syntax as described in the following sections applies equally. Original Text: This meta-data can be defined along with the Linux kernel recipe (recipe-space) as partially described in section 2.2 as well as within the Linux kernel sources themselves (in-tree). Where you choose to store the meta-data depends on what you want to do and how you intend to work. If you are unfamiliar with the Linux kernel and only wish to apply a config and possibly a couple of patches provided to you by others, you may find the recipe-space mechanism to be easier to work with. This is also a good approach if you are working with Linux kernel sources you do not control or if you just don't want to maintain a Linux kernel git repository on your own. If you are doing active kernel development and are already maintaining a Linux kernel git repository of your own, you may find it more convenient to work with the meta-data in the same repository as the Linux kernel sources. This can make iterative development of the Linux kernel more efficient outside of the bitbake environment. Regardless of where the meta-data is stored, the syntax as described in the following sections applies equally.
Recipe-Space Metadata When stored in recipe-space, the metadata files reside in a directory hierarchy below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a linux-yocto or linux-yocto-custom derived Linux kernel recipe. See the "Modifying an Existing Recipe" section for more information. By way of example, a trivial tree of metadata stored in recipe-space within a BSP layer might look like the following: meta/ `-- recipes-kernel `-- linux `-- linux-yocto |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg When the metadata is stored in recipe-space, you must take steps to ensure BitBake has the necessary information to decide which files to fetch and when they need to be fetched again. It is only necessary to specify the .scc files on the SRC_URI. BitBake parses them and fetches any files referenced in the .scc files by the include, patch, or kconf commands. Because of this, it is necessary to bump the recipe PR value when changing the content of files not explicitly listed in the SRC_URI. Original text: When stored in recipe-space, the meta-data files reside in a directory hierarchy below FILESEXTRAPATHS, which is typically set to ${THISDIR}/${PN} for a linux-yocto or linux-yocto-custom derived Linux kernel recipe. See 2.2. By way of example, a trivial tree of meta-data stored in recipe-space within a BSP layer might look like the following: meta/ `-- recipes-kernel `-- linux `-- linux-yocto |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg When the meta-data is stored in recipe-space, you must take steps to ensure bitbake has the necessary information to decide which files to fetch and when they need to be fetched again. It is only necessary to specify the .scc files on the SRC_URI; bitbake will parse them and fetch any files referenced in the .scc files by the include, patch, or kconf commands. Because of this, it is necessary to bump the recipe PR value when changing the content of files not explicitly listed in the SRC_URI.
In-Tree Metadata When stored in-tree, the metadata files reside in the "meta" directory of the Linux kernel sources. They may be present in the same branch as the sources, such as "master", or in their own orphan branch, typically named "meta". An orphan branch in Git is a branch with unique history and content to the other branches in the repository. This is useful to track metadata changes independently from the sources of the Linux kernel, while still keeping them together in the same repository. For the purposes of this document, we will discuss all in-tree metadata as residing below the meta/cfg/kernel-cache directory. By way of example, a trivial tree of metadata stored in a custom Linux kernel Git repository might look like the following: meta/ `-- cfg `-- kernel-cache |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg To use a specific branch for the metadata, specify the branch in the KMETA variable in your Linux kernel recipe, for example: KMETA = "meta" To use the same branch as the sources, set KMETA to the empty string: KMETA = "" If you are working with your own sources and want to create an orphan meta branch, you can do so using the following commands from within your Linux kernel Git repository: $ git checkout --orphan meta $ git rm -rf . $ git commit --allow-empty -m "Create orphan meta branch" Original text: When stored in-tree, the meta-data files reside in the "meta" directory of the Linux kernel sources. They may be present in the same branch as the sources, such as "master", or in their own orphan branch, typically named "meta". An orphan branch in git is a branch with unique history and content to the other branches in the repository. This is useful to track meta-data changes independently from the sources of the Linux kernel, while still keeping them together in the same repository. For the purposes of this document, we will discuss all in-tree meta-data as residing below the "meta/cfg/kernel-cache" directory. By way of example, a trivial tree of meta-data stored in a custom Linux kernel git repository might look like the following: meta/ `-- cfg `-- kernel-cache |-- bsp-standard.scc |-- bsp.cfg `-- standard.cfg To use a specific branch for the meta-data, specify the branch in the KMETA variable in your Linux kernel recipe, for example: KMETA = "meta" To use the same branch as the sources, set KMETA to the empty string: KMETA = "" If you are working with your own sources and want to create an orphan meta branch, you can do so using the following commands from within your Linux kernel git repository: $ git checkout --orphan meta $ git rm -rf . $ git commit --allow-empty -m "Create orphan meta branch"
Metadata Syntax The Yocto Project Linux kernel tools metadata consists of three primary types of files: scc scc stands for Series Configuration Control, but the naming has less significance in the current implementation of the tooling than it had in the past. Consider it to be a description file. description files, configuration fragments, and patches. The scc files define variables and include or otherwise reference any of the three file types. The description files are used to aggregate all types of metadata into what ultimately describes the sources and the configuration required to build a Linux kernel tailored to a specific machine. The scc description files are used to define two fundamental types of metadata: Features Board Support Packages (BSPs) Features aggregate sources in the form of patches and configuration in the form of configuration fragments into a modular reusable unit. Features are used to implement conceptually separate metadata descriptions like pure configuration fragments, simple patches, complex features, and kernel types (ktypes). Kernel types define general kernel features and policy to be reused in the BSPs. BSPs define hardware-specific features and aggregate them with kernel types to form the final description of what will be assembled and built. While the metadata syntax does not enforce any logical separation of configuration fragments, patches, features or kernel types, best practices dictate a logical separation of these types of meta-data. The following metadata file hierarchy is recommended: <base>/ bsp/ cfg/ features/ ktypes/ patches/ The bsp directory should contain the BSP descriptions, described in detail in section 3.3.5. The remaining directories all contain "features"; the separation is meant to aid in conceptualizing their intended usage. A simple guide to determine where your scc description file should go is as follows. If it contains only configuration fragments, it belongs in cfg. If it contains only source-code fixes, it belongs in patches. If it encapsulates a major feature, often combining sources and configurations, it belongs in features. If it aggregates non-hardware configuration and patches in order to define a base kernel policy or major kernel type to be reused across multiple BSPs, it belongs in ktypes. The lines between these can easily become blurred, especially as out-of-tree features are slowly merged upstream over time. Also remember that this is purely logical organization and has no impact on the functionality of the metadata as all of cfg, features, patches, and ktypes, contain "features" as far as the Yocto Project Linux kernel tools are concerned. Paths used in metadata files are relative to <base>, which is either FILESEXTRAPATHS if you are creating metadata in recipe-space as described in section "Recipe-Space Metadata", or meta/cfg/kernel-cache/ if you are creating metadata in-tree as described in the "In-Tree Metadata" section. Original text: The Yocto Project Linux kernel tools meta-data consists of three primary types of files: scc* description files, configuration fragments, and patches. The scc files define variables and include or otherwise reference any of the three file types. The description files are used to aggregate all types of meta-data into what ultimately describes the sources and the configuration required to build a Linux kernel tailored to a specific machine. The scc description files are used to define two fundamental types of meta-data: o Features o BSPs Features aggregate sources in the form of patches and configuration in the form of configuration fragments into a modular reusable unit. Features are used to implement conceptually separate meta-data descriptions like pure configuration fragments, simple patches, complex features, and kernel types (ktypes). Kernel types define general kernel features and policy to be reused in the BSPs. BSPs define hardware-specific features and aggregate them with kernel types to form the final description of what will be assembled and built. While the meta-data syntax does not enforce any logical separation of configuration fragments, patches, features or kernel types, best practices dictate a logical separation of these types of meta-data. The following meta-data file hierarchy is recommended: <base>/ bsp/ cfg/ features/ ktypes/ patches/ The bsp directory should contain the BSP descriptions, described in detail in 3.3.5. The remaining directories all contain "features"; the separation is meant to aid in conceptualizing their intended usage. A simple guide to determine where your scc description file should go is as follows. If it contains only configuration fragments, it belongs in cfg. If it contains only source-code fixes, it belongs in patches. If it encapsulates a major feature, often combining sources and configurations, it belongs in features. If it aggregates non-hardware configuration and patches in order to define a base kernel policy or major kernel type to be reused across multiple BSPs, it belongs in ktypes. The line between these can easily become blurred, especially as out-of-tree features are slowly merged upstream over time. Also remember that this is purely logical organization and has no impact on the functionality of the meta-data as all of cfg, features, patches, and ktypes, contain "features" as far as the Yocto Project Linux kernel tools are concerned. Paths used in meta-data files are relative to <base> which is either FILESEXTRAPATHS if you are creating meta-data in recipe-space (see 3.2.1), or meta/cfg/kernel-cache/ if you are creating meta-data in-tree (see 3.2.2). * scc stands for Series Configuration Control, but the naming has less significance in the current implementation of the tooling than it had in the past. Consider it to be a description file.