From c8839099506ee9c694f5b9da3ef6cc7cf0b2c142 Mon Sep 17 00:00:00 2001 From: Michael Opdenacker Date: Fri, 2 Dec 2022 15:50:48 +0100 Subject: SPDX and CVE documentation updates Backporting from master the ones that are applicable to kirkstone. (From yocto-docs rev: c10d65ef3bbdf4fe3abc03e3aef3d4ca8c2ad87f) Signed-off-by: Michael Opdenacker Signed-off-by: Mikko Rapeli Signed-off-by: Richard Purdie --- documentation/ref-manual/classes.rst | 96 ++++++++++++++++++++++++++++- documentation/ref-manual/terms.rst | 28 +++++++++ documentation/ref-manual/variables.rst | 109 ++++++++++++++++++++++++++++++++- 3 files changed, 229 insertions(+), 4 deletions(-) (limited to 'documentation/ref-manual') diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index a6dafe8f90..6e73c5d4fb 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst @@ -361,6 +361,32 @@ authors used. Both build methods inherit the ``cpan-base`` class for basic Perl support. +.. _ref-classes-create-spdx: + +``create-spdx.bbclass`` +======================= + +The :ref:`create-spdx ` class provides support for +automatically creating :term:`SPDX` :term:`SBOM` documents based upon image +and SDK contents. + +This class is meant to be inherited globally from a configuration file:: + + INHERIT += "create-spdx" + +The toplevel :term:`SPDX` output file is generated in JSON format as a +``IMAGE-MACHINE.spdx.json`` file in ``tmp/deploy/images/MACHINE/`` inside the +:term:`Build Directory`. There are other related files in the same directory, +as well as in ``tmp/deploy/spdx``. + +The exact behaviour of this class, and the amount of output can be controlled +by the :term:`SPDX_ARCHIVE_PACKAGED`, :term:`SPDX_ARCHIVE_SOURCES` and +:term:`SPDX_INCLUDE_SOURCES` variables. + +See the description of these variables and the +":ref:`dev-manual/common-tasks:creating a software bill of materials`" +section in the Yocto Project Development Manual for more details. + .. _ref-classes-cross: ``cross.bbclass`` @@ -396,14 +422,62 @@ discussion on these cross-compilation tools. ``cve-check.bbclass`` ===================== -The ``cve-check`` class looks for known CVEs (Common Vulnerabilities -and Exposures) while building an image. This class is meant to be +The :ref:`cve-check ` class looks for known CVEs (Common Vulnerabilities +and Exposures) while building with BitBake. This class is meant to be inherited globally from a configuration file:: INHERIT += "cve-check" +To filter out obsolete CVE database entries which are known not to impact software from Poky and OE-Core, +add following line to the build configuration file:: + + include cve-extra-exclusions.inc + You can also look for vulnerabilities in specific packages by passing -``-c cve_check`` to BitBake. You will find details in the +``-c cve_check`` to BitBake. + +After building the software with Bitbake, CVE check output reports are available in ``tmp/deploy/cve`` +and image specific summaries in ``tmp/deploy/images/*.cve`` or ``tmp/deploy/images/*.json`` files. + +When building, the CVE checker will emit build time warnings for any detected +issues which are in the state ``Unpatched``, meaning that CVE issue seems to affect the software component +and version being compiled and no patches to address the issue are applied. Other states +for detected CVE issues are: ``Patched`` meaning that a patch to address the issue is already +applied, and ``Ignored`` meaning that the issue can be ignored. + +The ``Patched`` state of a CVE issue is detected from patch files with the format +``CVE-ID.patch``, e.g. ``CVE-2019-20633.patch``, in the :term:`SRC_URI` and using +CVE metadata of format ``CVE: CVE-ID`` in the commit message of the patch file. + +If the recipe lists the ``CVE-ID`` in :term:`CVE_CHECK_IGNORE` variable, then the CVE state is reported +as ``Ignored``. Multiple CVEs can be listed separated by spaces. Example:: + + CVE_CHECK_IGNORE += "CVE-2020-29509 CVE-2020-29511" + +If CVE check reports that a recipe contains false positives or false negatives, these may be +fixed in recipes by adjusting the CVE product name using :term:`CVE_PRODUCT` and :term:`CVE_VERSION` variables. +:term:`CVE_PRODUCT` defaults to the plain recipe name :term:`BPN` which can be adjusted to one or more CVE +database vendor and product pairs using the syntax:: + + CVE_PRODUCT = "flex_project:flex" + +where ``flex_project`` is the CVE database vendor name and ``flex`` is the product name. Similarly +if the default recipe version :term:`PV` does not match the version numbers of the software component +in upstream releases or the CVE database, then the :term:`CVE_VERSION` variable can be used to set the +CVE database compatible version number, for example:: + + CVE_VERSION = "2.39" + +Any bugs or missing or incomplete information in the CVE database entries should be fixed in the CVE database +via the `NVD feedback form `__. + +Users should note that security is a process, not a product, and thus also CVE checking, analyzing results, +patching and updating the software should be done as a regular process. The data and assumptions +required for CVE checker to reliably detect issues are frequently broken in various ways. +These can only be detected by reviewing the details of the issues and iterating over the generated reports, +and following what happens in other Linux distributions and in the greater open source community. + +You will find some more details in the ":ref:`dev-manual/common-tasks:checking for vulnerabilities`" section in the Development Tasks Manual. @@ -1975,6 +2049,22 @@ When inherited by a recipe, the ``perlnative`` class supports using the native version of Perl built by the build system rather than using the version provided by the build host. +.. _ref-classes-pypi: + +``pypi.bbclass`` +================ + +The :ref:`pypi ` class sets variables appropriately for recipes that build +Python modules from `PyPI `__, the Python Package Index. +By default it determines the PyPI package name based upon :term:`BPN` +(stripping the "python-" or "python3-" prefix off if present), however in +some cases you may need to set it manually in the recipe by setting +:term:`PYPI_PACKAGE`. + +Variables set by the :ref:`pypi ` class include :term:`SRC_URI`, :term:`SECTION`, +:term:`HOMEPAGE`, :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX` +and :term:`CVE_PRODUCT`. + .. _ref-classes-python_flit_core: ``python_flit_core.bbclass`` diff --git a/documentation/ref-manual/terms.rst b/documentation/ref-manual/terms.rst index cba514c345..a9c84fe3a5 100644 --- a/documentation/ref-manual/terms.rst +++ b/documentation/ref-manual/terms.rst @@ -323,6 +323,23 @@ universal, the list includes them just in case: :term:`build host` and other components, that can work on specific hardware. + :term:`SBOM` + This term means *Software Bill of Materials*. When you distribute + software, it offers a description of all the components you used, + their corresponding licenses, their dependencies, the changes that were + applied and the known vulnerabilities that were fixed. + + This can be used by the recipients of the software to assess + their exposure to license compliance and security vulnerability issues. + + See the :wikipedia:`Software Supply Chain ` + article on Wikipedia for more details. + + The OpenEmbedded Build System can generate such documentation for your + project, in :term:`SPDX` format, based on all the metadata it used to + build the software images. See the ":ref:`dev-manual/common-tasks:creating + a software bill of materials`" section of the Development Tasks manual. + :term:`Source Directory` This term refers to the directory structure created as a result of creating a local copy of the ``poky`` Git @@ -383,6 +400,17 @@ universal, the list includes them just in case: ":ref:`overview-manual/development-environment:repositories, tags, and branches`" section in the Yocto Project Overview and Concepts Manual. + :term:`SPDX` + This term means *Software Package Data Exchange*, and is used as a open + standard for providing a *Software Bill of Materials* (:term:`SBOM`). + This standard is developed through a `Linux Foundation project + `__ and is used by the OpenEmbedded Build System to + provide an :term:`SBOM` associated to each a software image. + + For details, see Wikipedia's :wikipedia:`SPDX page ` + and the ":ref:`dev-manual/common-tasks:creating a software bill of materials`" + section of the Development Tasks manual. + :term:`Task` A per-recipe unit of execution for BitBake (e.g. :ref:`ref-tasks-compile`, diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 47462233c9..c182478a31 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst @@ -1469,15 +1469,31 @@ system and gives an overview of their function and contents. # This is windows only issue. CVE_CHECK_IGNORE += "CVE-2020-15523" + :term:`CVE_CHECK_SHOW_WARNINGS` + Specifies whether or not the :ref:`cve-check ` + class should generate warning messages on the console when unpatched + CVEs are found. The default is "1", but you may wish to set it to "0" if + you are already examining/processing the logs after the build has + completed and thus do not need the warning messages. + :term:`CVE_CHECK_SKIP_RECIPE` The list of package names (:term:`PN`) for which CVEs (Common Vulnerabilities and Exposures) are ignored. + :term:`CVE_DB_UPDATE_INTERVAL` + Specifies the CVE database update interval in seconds, as used by + ``cve-update-db-native``. The default value is "86400" i.e. once a day + (24*60*60). If the value is set to "0" then the update will be forced + every time. Alternatively, a negative value e.g. "-1" will disable + updates entirely. + :term:`CVE_PRODUCT` In a recipe, defines the name used to match the recipe name against the name in the upstream `NIST CVE database `__. - The default is ${:term:`BPN`}. If it does not match the name in the NIST CVE + The default is ${:term:`BPN`} (except for recipes that inherit the + :ref:`pypi ` class where it is set based upon + :term:`PYPI_PACKAGE`). If it does not match the name in the NIST CVE database or matches with multiple entries in the database, the default value needs to be changed. @@ -1492,6 +1508,18 @@ system and gives an overview of their function and contents. CVE_PRODUCT = "vendor:package" + :term:`CVE_VERSION` + In a recipe, defines the version used to match the recipe version + against the version in the `NIST CVE database `__ + when usign :ref:`cve-check `. + + The default is ${:term:`PV`} but if recipes use custom version numbers + which do not map to upstream software component release versions and the versions + used in the CVE database, then this variable can be used to set the + version number for :ref:`cve-check `. Example:: + + CVE_VERSION = "2.39" + :term:`CVSDIR` The directory in which files checked out under the CVS system are stored. @@ -6089,6 +6117,14 @@ system and gives an overview of their function and contents. :term:`PV` is the default value of the :term:`PKGV` variable. + :term:`PYPI_PACKAGE` + When inheriting the :ref:`pypi ` class, specifies the + `PyPI `__ package name to be built. The default value + is set based upon :term:`BPN` (stripping any "python-" or "python3-" + prefix off if present), however for some packages it will need to be set + explicitly if that will not match the package name (e.g. where the + package name has a prefix, underscores, uppercase letters etc.) + :term:`PYTHON_ABI` When used by recipes that inherit the :ref:`setuptools3 ` class, denotes the @@ -7058,6 +7094,77 @@ system and gives an overview of their function and contents. You can specify only a single URL in :term:`SOURCE_MIRROR_URL`. + :term:`SPDX_ARCHIVE_PACKAGED` + This option allows to add to :term:`SPDX` output compressed archives + of the files in the generated target packages. + + Such archives are available in + ``tmp/deploy/spdx/MACHINE/packages/packagename.tar.zst`` + under the :term:`Build Directory`. + + Enable this option as follows:: + + SPDX_ARCHIVE_PACKAGED = "1" + + According to our tests on release 4.1 "langdale", building + ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this + option multiplied the size of the ``tmp/deploy/spdx`` directory by a + factor of 13 (+1.6 GiB for this image), compared to just using the + :ref:`create-spdx ` class with no option. + + Note that this option doesn't increase the size of :term:`SPDX` + files in ``tmp/deploy/images/MACHINE``. + + :term:`SPDX_ARCHIVE_SOURCES` + This option allows to add to :term:`SPDX` output compressed archives + of the sources for packages installed on the target. It currently + only works when :term:`SPDX_INCLUDE_SOURCES` is set. + + This is one way of fulfilling "source code access" license + requirements. + + Such source archives are available in + ``tmp/deploy/spdx/MACHINE/recipes/recipe-packagename.tar.zst`` + under the :term:`Build Directory`. + + Enable this option as follows:: + + SPDX_INCLUDE_SOURCES = "1" + SPDX_ARCHIVE_SOURCES = "1" + + According to our tests on release 4.1 "langdale", building + ``core-image-minimal`` for the ``qemux86-64`` machine, enabling + these options multiplied the size of the ``tmp/deploy/spdx`` + directory by a factor of 11 (+1.4 GiB for this image), + compared to just using the :ref:`create-spdx ` + class with no option. + + Note that using this option only marginally increases the size + of the :term:`SPDX` output in ``tmp/deploy/images/MACHINE/`` + (+ 0.07\% with the tested image), compared to just enabling + :term:`SPDX_INCLUDE_SOURCES`. + + :term:`SPDX_INCLUDE_SOURCES` + This option allows to add a description of the source files used to build + the host tools and the target packages, to the ``spdx.json`` files in + ``tmp/deploy/spdx/MACHINE/recipes/`` under the :term:`Build Directory`. + As a consequence, the ``spdx.json`` files under the ``by-namespace`` and + ``packages`` subdirectories in ``tmp/deploy/spdx/MACHINE`` are also + modified to include references to such source file descriptions. + + Enable this option as follows:: + + SPDX_INCLUDE_SOURCES = "1" + + According to our tests on release 4.1 "langdale", building + ``core-image-minimal`` for the ``qemux86-64`` machine, enabling + this option multiplied the total size of the ``tmp/deploy/spdx`` + directory by a factor of 3 (+291 MiB for this image), + and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in + ``tmp/deploy/images/MACHINE`` by a factor of 130 (+15 MiB for this + image), compared to just using the + :ref:`create-spdx ` class with no option. + :term:`SPDXLICENSEMAP` Maps commonly used license names to their SPDX counterparts found in ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP` -- cgit v1.2.3-54-g00ecf