diff options
author | Michael Opdenacker <michael.opdenacker@bootlin.com> | 2022-12-02 15:50:48 +0100 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2022-12-07 15:04:15 +0000 |
commit | c8839099506ee9c694f5b9da3ef6cc7cf0b2c142 (patch) | |
tree | 2ec7cac1be39f8765f1ebca1e19c1cdbe21af205 /documentation | |
parent | be95d5f93abf2bce807f33fa0d9ae38e49fdbcce (diff) | |
download | poky-c8839099506ee9c694f5b9da3ef6cc7cf0b2c142.tar.gz |
SPDX and CVE documentation updates
Backporting from master the ones that are applicable to kirkstone.
(From yocto-docs rev: c10d65ef3bbdf4fe3abc03e3aef3d4ca8c2ad87f)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation')
-rw-r--r-- | documentation/conf.py | 1 | ||||
-rw-r--r-- | documentation/dev-manual/common-tasks.rst | 291 | ||||
-rw-r--r-- | documentation/ref-manual/classes.rst | 96 | ||||
-rw-r--r-- | documentation/ref-manual/terms.rst | 28 | ||||
-rw-r--r-- | documentation/ref-manual/variables.rst | 109 |
5 files changed, 442 insertions, 83 deletions
diff --git a/documentation/conf.py b/documentation/conf.py index a7cdf415f8..203b85dc7e 100644 --- a/documentation/conf.py +++ b/documentation/conf.py | |||
@@ -106,6 +106,7 @@ extlinks = { | |||
106 | 'oe_wiki': ('https://www.openembedded.org/wiki%s', None), | 106 | 'oe_wiki': ('https://www.openembedded.org/wiki%s', None), |
107 | 'oe_layerindex': ('https://layers.openembedded.org%s', None), | 107 | 'oe_layerindex': ('https://layers.openembedded.org%s', None), |
108 | 'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None), | 108 | 'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None), |
109 | 'wikipedia': ('https://en.wikipedia.org/wiki/%s', None), | ||
109 | } | 110 | } |
110 | 111 | ||
111 | # Intersphinx config to use cross reference with Bitbake user manual | 112 | # Intersphinx config to use cross reference with Bitbake user manual |
diff --git a/documentation/dev-manual/common-tasks.rst b/documentation/dev-manual/common-tasks.rst index 65f94432ab..1b91ada0a1 100644 --- a/documentation/dev-manual/common-tasks.rst +++ b/documentation/dev-manual/common-tasks.rst | |||
@@ -11426,39 +11426,6 @@ layers (recipes, configuration files, and so forth) enables you to meet | |||
11426 | your requirements to include the scripts to control compilation as well | 11426 | your requirements to include the scripts to control compilation as well |
11427 | as any modifications to the original source. | 11427 | as any modifications to the original source. |
11428 | 11428 | ||
11429 | Providing spdx files | ||
11430 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
11431 | |||
11432 | The spdx module has been integrated to a layer named meta-spdxscanner. | ||
11433 | meta-spdxscanner provides several kinds of scanner. If you want to enable | ||
11434 | this function, you have to follow the following steps: | ||
11435 | |||
11436 | 1. Add meta-spdxscanner layer into ``bblayers.conf``. | ||
11437 | |||
11438 | 2. Refer to the README in meta-spdxscanner to setup the environment (e.g, | ||
11439 | setup a fossology server) needed for the scanner. | ||
11440 | |||
11441 | 3. Meta-spdxscanner provides several methods within the bbclass to create spdx files. | ||
11442 | Please choose one that you want to use and enable the spdx task. You have to | ||
11443 | add some config options in ``local.conf`` file in your :term:`Build | ||
11444 | Directory`. Here is an example showing how to generate spdx files | ||
11445 | during bitbake using the fossology-python.bbclass:: | ||
11446 | |||
11447 | # Select fossology-python.bbclass. | ||
11448 | INHERIT += "fossology-python" | ||
11449 | # For fossology-python.bbclass, TOKEN is necessary, so, after setup a | ||
11450 | # Fossology server, you have to create a token. | ||
11451 | TOKEN = "eyJ0eXAiO..." | ||
11452 | # The fossology server is necessary for fossology-python.bbclass. | ||
11453 | FOSSOLOGY_SERVER = "http://xx.xx.xx.xx:8081/repo" | ||
11454 | # If you want to upload the source code to a special folder: | ||
11455 | FOLDER_NAME = "xxxx" //Optional | ||
11456 | # If you don't want to put spdx files in tmp/deploy/spdx, you can enable: | ||
11457 | SPDX_DEPLOY_DIR = "${DEPLOY_DIR}" //Optional | ||
11458 | |||
11459 | For more usage information refer to :yocto_git:`the meta-spdxscanner repository | ||
11460 | </meta-spdxscanner/>`. | ||
11461 | |||
11462 | Compliance Limitations with Executables Built from Static Libraries | 11429 | Compliance Limitations with Executables Built from Static Libraries |
11463 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | 11430 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
11464 | 11431 | ||
@@ -11499,21 +11466,92 @@ the license from the fetched source:: | |||
11499 | Checking for Vulnerabilities | 11466 | Checking for Vulnerabilities |
11500 | ============================ | 11467 | ============================ |
11501 | 11468 | ||
11502 | Vulnerabilities in images | 11469 | Vulnerabilities in Poky and OE-Core |
11503 | ------------------------- | 11470 | ----------------------------------- |
11504 | 11471 | ||
11505 | The Yocto Project has an infrastructure to track and address unfixed | 11472 | The Yocto Project has an infrastructure to track and address unfixed |
11506 | known security vulnerabilities, as tracked by the public | 11473 | known security vulnerabilities, as tracked by the public |
11507 | `Common Vulnerabilities and Exposures (CVE) <https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures>`__ | 11474 | :wikipedia:`Common Vulnerabilities and Exposures (CVE) <Common_Vulnerabilities_and_Exposures>` |
11508 | database. | 11475 | database. |
11509 | 11476 | ||
11510 | To know which packages are vulnerable to known security vulnerabilities, | 11477 | The Yocto Project maintains a `list of known vulnerabilities |
11511 | add the following setting to your configuration:: | 11478 | <https://autobuilder.yocto.io/pub/non-release/patchmetrics/>`__ |
11479 | for packages in Poky and OE-Core, tracking the evolution of the number of | ||
11480 | unpatched CVEs and the status of patches. Such information is available for | ||
11481 | the current development version and for each supported release. | ||
11482 | |||
11483 | Security is a process, not a product, and thus at any time, a number of security | ||
11484 | issues may be impacting Poky and OE-Core. It is up to the maintainers, users, | ||
11485 | contributors and anyone interested in the issues to investigate and possibly fix them by | ||
11486 | updating software components to newer versions or by applying patches to address them. | ||
11487 | It is recommended to work with Poky and OE-Core upstream maintainers and submit | ||
11488 | patches to fix them, see ":ref:`dev-manual/common-tasks:submitting a change to the yocto project`" for details. | ||
11489 | |||
11490 | Vulnerability check at build time | ||
11491 | --------------------------------- | ||
11492 | |||
11493 | To enable a check for CVE security vulnerabilities using :ref:`cve-check <ref-classes-cve-check>` in the specific image | ||
11494 | or target you are building, add the following setting to your configuration:: | ||
11512 | 11495 | ||
11513 | INHERIT += "cve-check" | 11496 | INHERIT += "cve-check" |
11514 | 11497 | ||
11515 | This way, at build time, BitBake will warn you about known CVEs | 11498 | The CVE database contains some old incomplete entries which have been |
11516 | as in the example below:: | 11499 | deemed not to impact Poky or OE-Core. These CVE entries can be excluded from the |
11500 | check using build configuration:: | ||
11501 | |||
11502 | include conf/distro/include/cve-extra-exclusions.inc | ||
11503 | |||
11504 | With this CVE check enabled, BitBake build will try to map each compiled software component | ||
11505 | recipe name and version information to the CVE database and generate recipe and | ||
11506 | image specific reports. These reports will contain: | ||
11507 | |||
11508 | - metadata about the software component like names and versions | ||
11509 | |||
11510 | - metadata about the CVE issue such as description and NVD link | ||
11511 | |||
11512 | - for each software component, a list of CVEs which are possibly impacting this version | ||
11513 | |||
11514 | - status of each CVE: ``Patched``, ``Unpatched`` or ``Ignored`` | ||
11515 | |||
11516 | The status ``Patched`` means that a patch file to address the security issue has been | ||
11517 | applied. ``Unpatched`` status means that no patches to address the issue have been | ||
11518 | applied and that the issue needs to be investigated. ``Ignored`` means that after | ||
11519 | analysis, it has been deemed to ignore the issue as it for example affects | ||
11520 | the software component on a different operating system platform. | ||
11521 | |||
11522 | After a build with CVE check enabled, reports for each compiled source recipe will be | ||
11523 | found in ``build/tmp/deploy/cve``. | ||
11524 | |||
11525 | For example the CVE check report for the ``flex-native`` recipe looks like:: | ||
11526 | |||
11527 | $ cat poky/build/tmp/deploy/cve/flex-native | ||
11528 | LAYER: meta | ||
11529 | PACKAGE NAME: flex-native | ||
11530 | PACKAGE VERSION: 2.6.4 | ||
11531 | CVE: CVE-2016-6354 | ||
11532 | CVE STATUS: Patched | ||
11533 | CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read. | ||
11534 | CVSS v2 BASE SCORE: 7.5 | ||
11535 | CVSS v3 BASE SCORE: 9.8 | ||
11536 | VECTOR: NETWORK | ||
11537 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354 | ||
11538 | |||
11539 | LAYER: meta | ||
11540 | PACKAGE NAME: flex-native | ||
11541 | PACKAGE VERSION: 2.6.4 | ||
11542 | CVE: CVE-2019-6293 | ||
11543 | CVE STATUS: Ignored | ||
11544 | CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service. | ||
11545 | CVSS v2 BASE SCORE: 4.3 | ||
11546 | CVSS v3 BASE SCORE: 5.5 | ||
11547 | VECTOR: NETWORK | ||
11548 | MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293 | ||
11549 | |||
11550 | For images, a summary of all recipes included in the image and their CVEs is also | ||
11551 | generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found | ||
11552 | in the ``tmp/deploy/images`` directory for each compiled image. | ||
11553 | |||
11554 | At build time CVE check will also throw warnings about ``Unpatched`` CVEs:: | ||
11517 | 11555 | ||
11518 | WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log | 11556 | WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log |
11519 | WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log | 11557 | WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log |
@@ -11522,21 +11560,46 @@ It is also possible to check the CVE status of individual packages as follows:: | |||
11522 | 11560 | ||
11523 | bitbake -c cve_check flex libarchive | 11561 | bitbake -c cve_check flex libarchive |
11524 | 11562 | ||
11525 | Note that OpenEmbedded-Core keeps a list of known unfixed CVE issues which can | 11563 | Fixing CVE product name and version mappings |
11526 | be ignored. You can pass this list to the check as follows:: | 11564 | -------------------------------------------- |
11565 | |||
11566 | By default, :ref:`cve-check <ref-classes-cve-check>` uses the recipe name :term:`BPN` as CVE | ||
11567 | product name when querying the CVE database. If this mapping contains false positives, e.g. | ||
11568 | some reported CVEs are not for the software component in question, or false negatives like | ||
11569 | some CVEs are not found to impact the recipe when they should, then the problems can be | ||
11570 | in the recipe name to CVE product mapping. These mapping issues can be fixed by setting | ||
11571 | the :term:`CVE_PRODUCT` variable inside the recipe. This defines the name of the software component in the | ||
11572 | upstream `NIST CVE database <https://nvd.nist.gov/>`__. | ||
11527 | 11573 | ||
11528 | bitbake -c cve_check libarchive -R conf/distro/include/cve-extra-exclusions.inc | 11574 | The variable supports using vendor and product names like this:: |
11529 | 11575 | ||
11530 | Enabling vulnerabily tracking in recipes | 11576 | CVE_PRODUCT = "flex_project:flex" |
11531 | ---------------------------------------- | ||
11532 | 11577 | ||
11533 | The :term:`CVE_PRODUCT` variable defines the name used to match the recipe name | 11578 | In this example the vendor name used in the CVE database is ``flex_project`` and the |
11534 | against the name in the upstream `NIST CVE database <https://nvd.nist.gov/>`__. | 11579 | product is ``flex``. With this setting the ``flex`` recipe only maps to this specific |
11580 | product and not products from other vendors with same name ``flex``. | ||
11535 | 11581 | ||
11536 | Editing recipes to fix vulnerabilities | 11582 | Similarly, when the recipe version :term:`PV` is not compatible with software versions used by |
11537 | -------------------------------------- | 11583 | the upstream software component releases and the CVE database, these can be fixed using |
11584 | the :term:`CVE_VERSION` variable. | ||
11585 | |||
11586 | Note that if the CVE entries in the NVD database contain bugs or have missing or incomplete | ||
11587 | information, it is recommended to fix the information there directly instead of working | ||
11588 | around the issues possibly for a long time in Poky and OE-Core side recipes. Feedback to | ||
11589 | NVD about CVE entries can be provided through the `NVD contact form <https://nvd.nist.gov/info/contact-form>`__. | ||
11590 | |||
11591 | Fixing vulnerabilities in recipes | ||
11592 | --------------------------------- | ||
11593 | |||
11594 | If a CVE security issue impacts a software component, it can be fixed by updating to a newer | ||
11595 | version of the software component or by applying a patch. For Poky and OE-Core master branches, updating | ||
11596 | to a newer software component release with fixes is the best option, but patches can be applied | ||
11597 | if releases are not yet available. | ||
11538 | 11598 | ||
11539 | To fix a given known vulnerability, you need to add a patch file to your recipe. Here's | 11599 | For stable branches, it is preferred to apply patches for the issues. For some software |
11600 | components minor version updates can also be applied if they are backwards compatible. | ||
11601 | |||
11602 | Here is an example of fixing CVE security issues with patch files, | ||
11540 | an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`:: | 11603 | an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`:: |
11541 | 11604 | ||
11542 | SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \ | 11605 | SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \ |
@@ -11548,31 +11611,21 @@ an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`:: | |||
11548 | file://fix-CVE-2020-22033-CVE-2020-22019.patch \ | 11611 | file://fix-CVE-2020-22033-CVE-2020-22019.patch \ |
11549 | file://fix-CVE-2021-33815.patch \ | 11612 | file://fix-CVE-2021-33815.patch \ |
11550 | 11613 | ||
11551 | The :ref:`cve-check <ref-classes-cve-check>` class defines two ways of | 11614 | A good practice is to include the CVE identifier in both the patch file name |
11552 | supplying a patch for a given CVE. The first | 11615 | and inside the patch file commit message using the format:: |
11553 | way is to use a patch filename that matches the below pattern:: | ||
11554 | |||
11555 | cve_file_name_match = re.compile(".*([Cc][Vv][Ee]\-\d{4}\-\d+)") | ||
11556 | |||
11557 | As shown in the example above, multiple CVE IDs can appear in a patch filename, | ||
11558 | but the :ref:`cve-check <ref-classes-cve-check>` class will only consider | ||
11559 | the last CVE ID in the filename as patched. | ||
11560 | |||
11561 | The second way to recognize a patched CVE ID is when a line matching the | ||
11562 | below pattern is found in any patch file provided by the recipe:: | ||
11563 | 11616 | ||
11564 | cve_match = re.compile("CVE:( CVE\-\d{4}\-\d+)+") | 11617 | CVE: CVE-2020-22033 |
11565 | 11618 | ||
11566 | This allows a single patch file to address multiple CVE IDs at the same time. | 11619 | CVE checker will then capture this information and change the CVE status to ``Patched`` |
11620 | in the generated reports. | ||
11567 | 11621 | ||
11568 | Of course, another way to fix vulnerabilities is to upgrade to a version | 11622 | If analysis shows that the CVE issue does not impact the recipe due to configuration, platform, |
11569 | of the package which is not impacted, typically a more recent one. | 11623 | version or other reasons, the CVE can be marked as ``Ignored`` using the :term:`CVE_CHECK_IGNORE` variable. |
11570 | The NIST database knows which versions are vulnerable and which ones | 11624 | As mentioned previously, if data in the CVE database is wrong, it is recommend to fix those |
11571 | are not. | 11625 | issues in the CVE database directly. |
11572 | 11626 | ||
11573 | Last but not least, you can choose to ignore vulnerabilities through | 11627 | Recipes can be completely skipped by CVE check by including the recipe name in |
11574 | the :term:`CVE_CHECK_SKIP_RECIPE` and :term:`CVE_CHECK_IGNORE` | 11628 | the :term:`CVE_CHECK_SKIP_RECIPE` variable. |
11575 | variables. | ||
11576 | 11629 | ||
11577 | Implementation details | 11630 | Implementation details |
11578 | ---------------------- | 11631 | ---------------------- |
@@ -11589,24 +11642,104 @@ file. The found CVE IDs are also considered as patched. | |||
11589 | Then, the code looks up all the CVE IDs in the NIST database for all the | 11642 | Then, the code looks up all the CVE IDs in the NIST database for all the |
11590 | products defined in :term:`CVE_PRODUCT`. Then, for each found CVE: | 11643 | products defined in :term:`CVE_PRODUCT`. Then, for each found CVE: |
11591 | 11644 | ||
11592 | - If the package name (:term:`PN`) is part of | 11645 | - If the package name (:term:`PN`) is part of |
11593 | :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as patched. | 11646 | :term:`CVE_CHECK_SKIP_RECIPE`, it is considered as ``Patched``. |
11594 | 11647 | ||
11595 | - If the CVE ID is part of :term:`CVE_CHECK_IGNORE`, it is | 11648 | - If the CVE ID is part of :term:`CVE_CHECK_IGNORE`, it is |
11596 | considered as patched too. | 11649 | set as ``Ignored``. |
11597 | 11650 | ||
11598 | - If the CVE ID is part of the patched CVE for the recipe, it is | 11651 | - If the CVE ID is part of the patched CVE for the recipe, it is |
11599 | already considered as patched. | 11652 | already considered as ``Patched``. |
11600 | 11653 | ||
11601 | - Otherwise, the code checks whether the recipe version (:term:`PV`) | 11654 | - Otherwise, the code checks whether the recipe version (:term:`PV`) |
11602 | is within the range of versions impacted by the CVE. If so, the CVE | 11655 | is within the range of versions impacted by the CVE. If so, the CVE |
11603 | is considered as unpatched. | 11656 | is considered as ``Unpatched``. |
11604 | 11657 | ||
11605 | The CVE database is stored in :term:`DL_DIR` and can be inspected using | 11658 | The CVE database is stored in :term:`DL_DIR` and can be inspected using |
11606 | ``sqlite3`` command as follows:: | 11659 | ``sqlite3`` command as follows:: |
11607 | 11660 | ||
11608 | sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462 | 11661 | sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462 |
11609 | 11662 | ||
11663 | When analyzing CVEs, it is recommended to: | ||
11664 | |||
11665 | - study the latest information in `CVE database <https://nvd.nist.gov/vuln/search>`__. | ||
11666 | |||
11667 | - check how upstream developers of the software component addressed the issue, e.g. | ||
11668 | what patch was applied, which upstream release contains the fix. | ||
11669 | |||
11670 | - check what other Linux distributions like `Debian <https://security-tracker.debian.org/tracker/>`__ | ||
11671 | did to analyze and address the issue. | ||
11672 | |||
11673 | - follow security notices from other Linux distributions. | ||
11674 | |||
11675 | - follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for | ||
11676 | discussions and advance notifications of CVE bugs and software releases with fixes. | ||
11677 | |||
11678 | Creating a Software Bill of Materials | ||
11679 | ===================================== | ||
11680 | |||
11681 | Once you are able to build an image for your project, once the licenses for | ||
11682 | each software component are all identified (see | ||
11683 | ":ref:`dev-manual/common-tasks:working with licenses`") and once vulnerability | ||
11684 | fixes are applied (see ":ref:`dev-manual/common-tasks:checking | ||
11685 | for vulnerabilities`"), the OpenEmbedded build system can generate | ||
11686 | a description of all the components you used, their licenses, their dependencies, | ||
11687 | the changes that were applied and the known vulnerabilities that were fixed. | ||
11688 | |||
11689 | This description is generated in the form of a *Software Bill of Materials* | ||
11690 | (:term:`SBOM`), using the :term:`SPDX` standard. | ||
11691 | |||
11692 | When you release software, this is the most standard way to provide information | ||
11693 | about the Software Supply Chain of your software image and SDK. The | ||
11694 | :term:`SBOM` tooling is often used to ensure open source license compliance by | ||
11695 | providing the license texts used in the product which legal departments and end | ||
11696 | users can read in standardized format. | ||
11697 | |||
11698 | :term:`SBOM` information is also critical to performing vulnerability exposure | ||
11699 | assessments, as all the components used in the Software Supply Chain are listed. | ||
11700 | |||
11701 | The OpenEmbedded build system doesn't generate such information by default. | ||
11702 | To make this happen, you must inherit the | ||
11703 | :ref:`create-spdx <ref-classes-create-spdx>` class from a configuration file:: | ||
11704 | |||
11705 | INHERIT += "create-spdx" | ||
11706 | |||
11707 | You then get :term:`SPDX` output in JSON format as an | ||
11708 | ``IMAGE-MACHINE.spdx.json`` file in ``tmp/deploy/images/MACHINE/`` inside the | ||
11709 | :term:`Build Directory`. | ||
11710 | |||
11711 | This is a toplevel file accompanied by an ``IMAGE-MACHINE.spdx.index.json`` | ||
11712 | containing an index of JSON :term:`SPDX` files for individual recipes, together | ||
11713 | with an ``IMAGE-MACHINE.spdx.tar.zst`` compressed archive containing all such | ||
11714 | files. | ||
11715 | |||
11716 | The :ref:`create-spdx <ref-classes-create-spdx>` class offers options to include | ||
11717 | more information in the output :term:`SPDX` data, such as adding compressed | ||
11718 | archives of the files in the generated target packages | ||
11719 | (:term:`SPDX_ARCHIVE_PACKAGED`), adding a description of the source files | ||
11720 | handled by the target recipes (:term:`SPDX_INCLUDE_SOURCES`) and adding archives | ||
11721 | of these source files themselves (:term:`SPDX_ARCHIVE_SOURCES`). | ||
11722 | |||
11723 | Though the toplevel :term:`SPDX` output is available in | ||
11724 | ``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary | ||
11725 | generated files are available in ``tmp/deploy/spdx/MACHINE`` too, such as: | ||
11726 | |||
11727 | - The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst`` | ||
11728 | archive. | ||
11729 | |||
11730 | - Compressed archives of the files in the generated target packages, | ||
11731 | in ``packages/packagename.tar.zst`` (when :term:`SPDX_ARCHIVE_PACKAGED` | ||
11732 | is set). | ||
11733 | |||
11734 | - Compressed archives of the source files used to build the host tools | ||
11735 | and the target packages in ``recipes/recipe-packagename.tar.zst`` | ||
11736 | (when :term:`SPDX_ARCHIVE_SOURCES` is set). Those are needed to fulfill | ||
11737 | "source code access" license requirements. | ||
11738 | |||
11739 | See the `tools page <https://spdx.dev/resources/tools/>`__ on the :term:`SPDX` | ||
11740 | project website for a list of tools to consume and transform the :term:`SPDX` | ||
11741 | data generated by the OpenEmbedded build system. | ||
11742 | |||
11610 | Using the Error Reporting Tool | 11743 | Using the Error Reporting Tool |
11611 | ============================== | 11744 | ============================== |
11612 | 11745 | ||
diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst index a6dafe8f90..6e73c5d4fb 100644 --- a/documentation/ref-manual/classes.rst +++ b/documentation/ref-manual/classes.rst | |||
@@ -361,6 +361,32 @@ authors used. | |||
361 | Both build methods inherit the ``cpan-base`` class for basic Perl | 361 | Both build methods inherit the ``cpan-base`` class for basic Perl |
362 | support. | 362 | support. |
363 | 363 | ||
364 | .. _ref-classes-create-spdx: | ||
365 | |||
366 | ``create-spdx.bbclass`` | ||
367 | ======================= | ||
368 | |||
369 | The :ref:`create-spdx <ref-classes-create-spdx>` class provides support for | ||
370 | automatically creating :term:`SPDX` :term:`SBOM` documents based upon image | ||
371 | and SDK contents. | ||
372 | |||
373 | This class is meant to be inherited globally from a configuration file:: | ||
374 | |||
375 | INHERIT += "create-spdx" | ||
376 | |||
377 | The toplevel :term:`SPDX` output file is generated in JSON format as a | ||
378 | ``IMAGE-MACHINE.spdx.json`` file in ``tmp/deploy/images/MACHINE/`` inside the | ||
379 | :term:`Build Directory`. There are other related files in the same directory, | ||
380 | as well as in ``tmp/deploy/spdx``. | ||
381 | |||
382 | The exact behaviour of this class, and the amount of output can be controlled | ||
383 | by the :term:`SPDX_ARCHIVE_PACKAGED`, :term:`SPDX_ARCHIVE_SOURCES` and | ||
384 | :term:`SPDX_INCLUDE_SOURCES` variables. | ||
385 | |||
386 | See the description of these variables and the | ||
387 | ":ref:`dev-manual/common-tasks:creating a software bill of materials`" | ||
388 | section in the Yocto Project Development Manual for more details. | ||
389 | |||
364 | .. _ref-classes-cross: | 390 | .. _ref-classes-cross: |
365 | 391 | ||
366 | ``cross.bbclass`` | 392 | ``cross.bbclass`` |
@@ -396,14 +422,62 @@ discussion on these cross-compilation tools. | |||
396 | ``cve-check.bbclass`` | 422 | ``cve-check.bbclass`` |
397 | ===================== | 423 | ===================== |
398 | 424 | ||
399 | The ``cve-check`` class looks for known CVEs (Common Vulnerabilities | 425 | The :ref:`cve-check <ref-classes-cve-check>` class looks for known CVEs (Common Vulnerabilities |
400 | and Exposures) while building an image. This class is meant to be | 426 | and Exposures) while building with BitBake. This class is meant to be |
401 | inherited globally from a configuration file:: | 427 | inherited globally from a configuration file:: |
402 | 428 | ||
403 | INHERIT += "cve-check" | 429 | INHERIT += "cve-check" |
404 | 430 | ||
431 | To filter out obsolete CVE database entries which are known not to impact software from Poky and OE-Core, | ||
432 | add following line to the build configuration file:: | ||
433 | |||
434 | include cve-extra-exclusions.inc | ||
435 | |||
405 | You can also look for vulnerabilities in specific packages by passing | 436 | You can also look for vulnerabilities in specific packages by passing |
406 | ``-c cve_check`` to BitBake. You will find details in the | 437 | ``-c cve_check`` to BitBake. |
438 | |||
439 | After building the software with Bitbake, CVE check output reports are available in ``tmp/deploy/cve`` | ||
440 | and image specific summaries in ``tmp/deploy/images/*.cve`` or ``tmp/deploy/images/*.json`` files. | ||
441 | |||
442 | When building, the CVE checker will emit build time warnings for any detected | ||
443 | issues which are in the state ``Unpatched``, meaning that CVE issue seems to affect the software component | ||
444 | and version being compiled and no patches to address the issue are applied. Other states | ||
445 | for detected CVE issues are: ``Patched`` meaning that a patch to address the issue is already | ||
446 | applied, and ``Ignored`` meaning that the issue can be ignored. | ||
447 | |||
448 | The ``Patched`` state of a CVE issue is detected from patch files with the format | ||
449 | ``CVE-ID.patch``, e.g. ``CVE-2019-20633.patch``, in the :term:`SRC_URI` and using | ||
450 | CVE metadata of format ``CVE: CVE-ID`` in the commit message of the patch file. | ||
451 | |||
452 | If the recipe lists the ``CVE-ID`` in :term:`CVE_CHECK_IGNORE` variable, then the CVE state is reported | ||
453 | as ``Ignored``. Multiple CVEs can be listed separated by spaces. Example:: | ||
454 | |||
455 | CVE_CHECK_IGNORE += "CVE-2020-29509 CVE-2020-29511" | ||
456 | |||
457 | If CVE check reports that a recipe contains false positives or false negatives, these may be | ||
458 | fixed in recipes by adjusting the CVE product name using :term:`CVE_PRODUCT` and :term:`CVE_VERSION` variables. | ||
459 | :term:`CVE_PRODUCT` defaults to the plain recipe name :term:`BPN` which can be adjusted to one or more CVE | ||
460 | database vendor and product pairs using the syntax:: | ||
461 | |||
462 | CVE_PRODUCT = "flex_project:flex" | ||
463 | |||
464 | where ``flex_project`` is the CVE database vendor name and ``flex`` is the product name. Similarly | ||
465 | if the default recipe version :term:`PV` does not match the version numbers of the software component | ||
466 | in upstream releases or the CVE database, then the :term:`CVE_VERSION` variable can be used to set the | ||
467 | CVE database compatible version number, for example:: | ||
468 | |||
469 | CVE_VERSION = "2.39" | ||
470 | |||
471 | Any bugs or missing or incomplete information in the CVE database entries should be fixed in the CVE database | ||
472 | via the `NVD feedback form <https://nvd.nist.gov/info/contact-form>`__. | ||
473 | |||
474 | Users should note that security is a process, not a product, and thus also CVE checking, analyzing results, | ||
475 | patching and updating the software should be done as a regular process. The data and assumptions | ||
476 | required for CVE checker to reliably detect issues are frequently broken in various ways. | ||
477 | These can only be detected by reviewing the details of the issues and iterating over the generated reports, | ||
478 | and following what happens in other Linux distributions and in the greater open source community. | ||
479 | |||
480 | You will find some more details in the | ||
407 | ":ref:`dev-manual/common-tasks:checking for vulnerabilities`" | 481 | ":ref:`dev-manual/common-tasks:checking for vulnerabilities`" |
408 | section in the Development Tasks Manual. | 482 | section in the Development Tasks Manual. |
409 | 483 | ||
@@ -1975,6 +2049,22 @@ When inherited by a recipe, the ``perlnative`` class supports using the | |||
1975 | native version of Perl built by the build system rather than using the | 2049 | native version of Perl built by the build system rather than using the |
1976 | version provided by the build host. | 2050 | version provided by the build host. |
1977 | 2051 | ||
2052 | .. _ref-classes-pypi: | ||
2053 | |||
2054 | ``pypi.bbclass`` | ||
2055 | ================ | ||
2056 | |||
2057 | The :ref:`pypi <ref-classes-pypi>` class sets variables appropriately for recipes that build | ||
2058 | Python modules from `PyPI <https://pypi.org/>`__, the Python Package Index. | ||
2059 | By default it determines the PyPI package name based upon :term:`BPN` | ||
2060 | (stripping the "python-" or "python3-" prefix off if present), however in | ||
2061 | some cases you may need to set it manually in the recipe by setting | ||
2062 | :term:`PYPI_PACKAGE`. | ||
2063 | |||
2064 | Variables set by the :ref:`pypi <ref-classes-pypi>` class include :term:`SRC_URI`, :term:`SECTION`, | ||
2065 | :term:`HOMEPAGE`, :term:`UPSTREAM_CHECK_URI`, :term:`UPSTREAM_CHECK_REGEX` | ||
2066 | and :term:`CVE_PRODUCT`. | ||
2067 | |||
1978 | .. _ref-classes-python_flit_core: | 2068 | .. _ref-classes-python_flit_core: |
1979 | 2069 | ||
1980 | ``python_flit_core.bbclass`` | 2070 | ``python_flit_core.bbclass`` |
diff --git a/documentation/ref-manual/terms.rst b/documentation/ref-manual/terms.rst index cba514c345..a9c84fe3a5 100644 --- a/documentation/ref-manual/terms.rst +++ b/documentation/ref-manual/terms.rst | |||
@@ -323,6 +323,23 @@ universal, the list includes them just in case: | |||
323 | :term:`build host<Build Host>` and other components, that can | 323 | :term:`build host<Build Host>` and other components, that can |
324 | work on specific hardware. | 324 | work on specific hardware. |
325 | 325 | ||
326 | :term:`SBOM` | ||
327 | This term means *Software Bill of Materials*. When you distribute | ||
328 | software, it offers a description of all the components you used, | ||
329 | their corresponding licenses, their dependencies, the changes that were | ||
330 | applied and the known vulnerabilities that were fixed. | ||
331 | |||
332 | This can be used by the recipients of the software to assess | ||
333 | their exposure to license compliance and security vulnerability issues. | ||
334 | |||
335 | See the :wikipedia:`Software Supply Chain <Software_supply_chain>` | ||
336 | article on Wikipedia for more details. | ||
337 | |||
338 | The OpenEmbedded Build System can generate such documentation for your | ||
339 | project, in :term:`SPDX` format, based on all the metadata it used to | ||
340 | build the software images. See the ":ref:`dev-manual/common-tasks:creating | ||
341 | a software bill of materials`" section of the Development Tasks manual. | ||
342 | |||
326 | :term:`Source Directory` | 343 | :term:`Source Directory` |
327 | This term refers to the directory structure | 344 | This term refers to the directory structure |
328 | created as a result of creating a local copy of the ``poky`` Git | 345 | created as a result of creating a local copy of the ``poky`` Git |
@@ -383,6 +400,17 @@ universal, the list includes them just in case: | |||
383 | ":ref:`overview-manual/development-environment:repositories, tags, and branches`" | 400 | ":ref:`overview-manual/development-environment:repositories, tags, and branches`" |
384 | section in the Yocto Project Overview and Concepts Manual. | 401 | section in the Yocto Project Overview and Concepts Manual. |
385 | 402 | ||
403 | :term:`SPDX` | ||
404 | This term means *Software Package Data Exchange*, and is used as a open | ||
405 | standard for providing a *Software Bill of Materials* (:term:`SBOM`). | ||
406 | This standard is developed through a `Linux Foundation project | ||
407 | <https://spdx.dev/>`__ and is used by the OpenEmbedded Build System to | ||
408 | provide an :term:`SBOM` associated to each a software image. | ||
409 | |||
410 | For details, see Wikipedia's :wikipedia:`SPDX page <Software_Package_Data_Exchange>` | ||
411 | and the ":ref:`dev-manual/common-tasks:creating a software bill of materials`" | ||
412 | section of the Development Tasks manual. | ||
413 | |||
386 | :term:`Task` | 414 | :term:`Task` |
387 | A per-recipe unit of execution for BitBake (e.g. | 415 | A per-recipe unit of execution for BitBake (e.g. |
388 | :ref:`ref-tasks-compile`, | 416 | :ref:`ref-tasks-compile`, |
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst index 47462233c9..c182478a31 100644 --- a/documentation/ref-manual/variables.rst +++ b/documentation/ref-manual/variables.rst | |||
@@ -1469,15 +1469,31 @@ system and gives an overview of their function and contents. | |||
1469 | # This is windows only issue. | 1469 | # This is windows only issue. |
1470 | CVE_CHECK_IGNORE += "CVE-2020-15523" | 1470 | CVE_CHECK_IGNORE += "CVE-2020-15523" |
1471 | 1471 | ||
1472 | :term:`CVE_CHECK_SHOW_WARNINGS` | ||
1473 | Specifies whether or not the :ref:`cve-check <ref-classes-cve-check>` | ||
1474 | class should generate warning messages on the console when unpatched | ||
1475 | CVEs are found. The default is "1", but you may wish to set it to "0" if | ||
1476 | you are already examining/processing the logs after the build has | ||
1477 | completed and thus do not need the warning messages. | ||
1478 | |||
1472 | :term:`CVE_CHECK_SKIP_RECIPE` | 1479 | :term:`CVE_CHECK_SKIP_RECIPE` |
1473 | The list of package names (:term:`PN`) for which | 1480 | The list of package names (:term:`PN`) for which |
1474 | CVEs (Common Vulnerabilities and Exposures) are ignored. | 1481 | CVEs (Common Vulnerabilities and Exposures) are ignored. |
1475 | 1482 | ||
1483 | :term:`CVE_DB_UPDATE_INTERVAL` | ||
1484 | Specifies the CVE database update interval in seconds, as used by | ||
1485 | ``cve-update-db-native``. The default value is "86400" i.e. once a day | ||
1486 | (24*60*60). If the value is set to "0" then the update will be forced | ||
1487 | every time. Alternatively, a negative value e.g. "-1" will disable | ||
1488 | updates entirely. | ||
1489 | |||
1476 | :term:`CVE_PRODUCT` | 1490 | :term:`CVE_PRODUCT` |
1477 | In a recipe, defines the name used to match the recipe name | 1491 | In a recipe, defines the name used to match the recipe name |
1478 | against the name in the upstream `NIST CVE database <https://nvd.nist.gov/>`__. | 1492 | against the name in the upstream `NIST CVE database <https://nvd.nist.gov/>`__. |
1479 | 1493 | ||
1480 | The default is ${:term:`BPN`}. If it does not match the name in the NIST CVE | 1494 | The default is ${:term:`BPN`} (except for recipes that inherit the |
1495 | :ref:`pypi <ref-classes-pypi>` class where it is set based upon | ||
1496 | :term:`PYPI_PACKAGE`). If it does not match the name in the NIST CVE | ||
1481 | database or matches with multiple entries in the database, the default | 1497 | database or matches with multiple entries in the database, the default |
1482 | value needs to be changed. | 1498 | value needs to be changed. |
1483 | 1499 | ||
@@ -1492,6 +1508,18 @@ system and gives an overview of their function and contents. | |||
1492 | 1508 | ||
1493 | CVE_PRODUCT = "vendor:package" | 1509 | CVE_PRODUCT = "vendor:package" |
1494 | 1510 | ||
1511 | :term:`CVE_VERSION` | ||
1512 | In a recipe, defines the version used to match the recipe version | ||
1513 | against the version in the `NIST CVE database <https://nvd.nist.gov/>`__ | ||
1514 | when usign :ref:`cve-check <ref-classes-cve-check>`. | ||
1515 | |||
1516 | The default is ${:term:`PV`} but if recipes use custom version numbers | ||
1517 | which do not map to upstream software component release versions and the versions | ||
1518 | used in the CVE database, then this variable can be used to set the | ||
1519 | version number for :ref:`cve-check <ref-classes-cve-check>`. Example:: | ||
1520 | |||
1521 | CVE_VERSION = "2.39" | ||
1522 | |||
1495 | :term:`CVSDIR` | 1523 | :term:`CVSDIR` |
1496 | The directory in which files checked out under the CVS system are | 1524 | The directory in which files checked out under the CVS system are |
1497 | stored. | 1525 | stored. |
@@ -6089,6 +6117,14 @@ system and gives an overview of their function and contents. | |||
6089 | 6117 | ||
6090 | :term:`PV` is the default value of the :term:`PKGV` variable. | 6118 | :term:`PV` is the default value of the :term:`PKGV` variable. |
6091 | 6119 | ||
6120 | :term:`PYPI_PACKAGE` | ||
6121 | When inheriting the :ref:`pypi <ref-classes-pypi>` class, specifies the | ||
6122 | `PyPI <https://pypi.org/>`__ package name to be built. The default value | ||
6123 | is set based upon :term:`BPN` (stripping any "python-" or "python3-" | ||
6124 | prefix off if present), however for some packages it will need to be set | ||
6125 | explicitly if that will not match the package name (e.g. where the | ||
6126 | package name has a prefix, underscores, uppercase letters etc.) | ||
6127 | |||
6092 | :term:`PYTHON_ABI` | 6128 | :term:`PYTHON_ABI` |
6093 | When used by recipes that inherit the | 6129 | When used by recipes that inherit the |
6094 | :ref:`setuptools3 <ref-classes-setuptools3>` class, denotes the | 6130 | :ref:`setuptools3 <ref-classes-setuptools3>` class, denotes the |
@@ -7058,6 +7094,77 @@ system and gives an overview of their function and contents. | |||
7058 | 7094 | ||
7059 | You can specify only a single URL in :term:`SOURCE_MIRROR_URL`. | 7095 | You can specify only a single URL in :term:`SOURCE_MIRROR_URL`. |
7060 | 7096 | ||
7097 | :term:`SPDX_ARCHIVE_PACKAGED` | ||
7098 | This option allows to add to :term:`SPDX` output compressed archives | ||
7099 | of the files in the generated target packages. | ||
7100 | |||
7101 | Such archives are available in | ||
7102 | ``tmp/deploy/spdx/MACHINE/packages/packagename.tar.zst`` | ||
7103 | under the :term:`Build Directory`. | ||
7104 | |||
7105 | Enable this option as follows:: | ||
7106 | |||
7107 | SPDX_ARCHIVE_PACKAGED = "1" | ||
7108 | |||
7109 | According to our tests on release 4.1 "langdale", building | ||
7110 | ``core-image-minimal`` for the ``qemux86-64`` machine, enabling this | ||
7111 | option multiplied the size of the ``tmp/deploy/spdx`` directory by a | ||
7112 | factor of 13 (+1.6 GiB for this image), compared to just using the | ||
7113 | :ref:`create-spdx <ref-classes-create-spdx>` class with no option. | ||
7114 | |||
7115 | Note that this option doesn't increase the size of :term:`SPDX` | ||
7116 | files in ``tmp/deploy/images/MACHINE``. | ||
7117 | |||
7118 | :term:`SPDX_ARCHIVE_SOURCES` | ||
7119 | This option allows to add to :term:`SPDX` output compressed archives | ||
7120 | of the sources for packages installed on the target. It currently | ||
7121 | only works when :term:`SPDX_INCLUDE_SOURCES` is set. | ||
7122 | |||
7123 | This is one way of fulfilling "source code access" license | ||
7124 | requirements. | ||
7125 | |||
7126 | Such source archives are available in | ||
7127 | ``tmp/deploy/spdx/MACHINE/recipes/recipe-packagename.tar.zst`` | ||
7128 | under the :term:`Build Directory`. | ||
7129 | |||
7130 | Enable this option as follows:: | ||
7131 | |||
7132 | SPDX_INCLUDE_SOURCES = "1" | ||
7133 | SPDX_ARCHIVE_SOURCES = "1" | ||
7134 | |||
7135 | According to our tests on release 4.1 "langdale", building | ||
7136 | ``core-image-minimal`` for the ``qemux86-64`` machine, enabling | ||
7137 | these options multiplied the size of the ``tmp/deploy/spdx`` | ||
7138 | directory by a factor of 11 (+1.4 GiB for this image), | ||
7139 | compared to just using the :ref:`create-spdx <ref-classes-create-spdx>` | ||
7140 | class with no option. | ||
7141 | |||
7142 | Note that using this option only marginally increases the size | ||
7143 | of the :term:`SPDX` output in ``tmp/deploy/images/MACHINE/`` | ||
7144 | (+ 0.07\% with the tested image), compared to just enabling | ||
7145 | :term:`SPDX_INCLUDE_SOURCES`. | ||
7146 | |||
7147 | :term:`SPDX_INCLUDE_SOURCES` | ||
7148 | This option allows to add a description of the source files used to build | ||
7149 | the host tools and the target packages, to the ``spdx.json`` files in | ||
7150 | ``tmp/deploy/spdx/MACHINE/recipes/`` under the :term:`Build Directory`. | ||
7151 | As a consequence, the ``spdx.json`` files under the ``by-namespace`` and | ||
7152 | ``packages`` subdirectories in ``tmp/deploy/spdx/MACHINE`` are also | ||
7153 | modified to include references to such source file descriptions. | ||
7154 | |||
7155 | Enable this option as follows:: | ||
7156 | |||
7157 | SPDX_INCLUDE_SOURCES = "1" | ||
7158 | |||
7159 | According to our tests on release 4.1 "langdale", building | ||
7160 | ``core-image-minimal`` for the ``qemux86-64`` machine, enabling | ||
7161 | this option multiplied the total size of the ``tmp/deploy/spdx`` | ||
7162 | directory by a factor of 3 (+291 MiB for this image), | ||
7163 | and the size of the ``IMAGE-MACHINE.spdx.tar.zst`` in | ||
7164 | ``tmp/deploy/images/MACHINE`` by a factor of 130 (+15 MiB for this | ||
7165 | image), compared to just using the | ||
7166 | :ref:`create-spdx <ref-classes-create-spdx>` class with no option. | ||
7167 | |||
7061 | :term:`SPDXLICENSEMAP` | 7168 | :term:`SPDXLICENSEMAP` |
7062 | Maps commonly used license names to their SPDX counterparts found in | 7169 | Maps commonly used license names to their SPDX counterparts found in |
7063 | ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP` | 7170 | ``meta/files/common-licenses/``. For the default :term:`SPDXLICENSEMAP` |