summaryrefslogtreecommitdiffstats
path: root/meta/lib
Commit message (Collapse)AuthorAgeFilesLines
* gdbserver : add selftestYogesh Tyagi2022-08-101-0/+122
| | | | | | | | | | | | | | | The test runs gdbserver on qemu and connects the gdb client from host over TCP. It builds a cross gdb on the host and compiles the program to be debugged on the target, launches the gdbserver and tries to connect cross gdb to it. [YOCTO #13996] (From OE-Core rev: 37164f7e39eea3a1e594d8306d2569868438ba93) Signed-off-by: Yogesh Tyagi <yogesh.tyagi@intel.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/wic: Tweak test case to not depend on kernel sizeRichard Purdie2022-08-081-1/+1
| | | | | | | | | | The current test assumes the kernel size leaves a certain amount of whitespace in the output. Improve this constraint so a slightly larger kernel doesn't fail the test. (From OE-Core rev: bd60c44bef4a1b5d3c8fe77a9e6d3a8f43b0dea4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa devtool: Add tests to cover devtool handling of various git URL stylesThomas Roos2022-08-081-1/+33
| | | | | | | | | | | Add two test cases for git URL styles that trigger reformat_git_url. [YOCTO #11394] (From OE-Core rev: 5593439a5efbb53fc46099650ae86943751b0c4e) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/reproducible: Exclude rust/rust-dbg for now until we can fixRichard Purdie2022-08-081-0/+2
| | | | | | | | | | | | There looks to be a reproducibility issue left in one of the rust libraries. It doesn't appear to be a string issue but some binary problem. Disable rust from the reproducibility testing until we can get to the bottom of the issue (allowing wider testing of all the other improvements). (From OE-Core rev: a261333f6591ea94afc567dee04a2e3c6d5059cf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/sstate: Ensure tests are deterministicRichard Purdie2022-08-071-14/+12
| | | | | | | | | | | | | glob.glob() depends on the order of files on disk and selecting [0] is race prone. We should cover all the nativesdk files so rework the function to do this. Spotted as some oe-selftests failed, some passed and it raised a question of why! (From OE-Core rev: 8818478420a5c73b1dc1710774545f7e984307da) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk/rust: Fix file deletion for multilib SDKsRichard Purdie2022-08-011-2/+2
| | | | | | | | | | We need to use shutil.rmtree here since removedirs() only covers directories. Make the exception for specific too to make errors easier to catch. (From OE-Core rev: 9d2a661e46123a2292f7887658e6fa54923dbcc0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* create-spdx: Fix supplier fieldMihai Lindner2022-07-281-1/+1
| | | | | | | | | | | The correct field name is "supplier" according to SPDX schema. The "supplier" field translates to "PackageSupplier", but that's for tag-value format. (From OE-Core rev: ca8db0e0a2860ac1e3f537471fa71b43c3be0a58) Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/meta_ide: add a test for running SDK tests directly in a yocto buildAlexander Kanavin2022-07-281-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's been a recent discussion about how we can make the Yocto SDK experience better [1]. One of the ideas was to eliminate the SDK as a separate artefact altogether and simply provide everything that the SDK and eSDKs do directly in a yocto build. This does not mean that people have to 'learn Yocto', but rather that the integrators should provide a well-functioning sstate cache infrastructure (same as with minimal eSDK, really), and a few wrapper scripts for setting up the build and the SDK environment that run layer setup and bitbake behind the scenes. [1] https://lists.openembedded.org/g/openembedded-architecture/topic/thoughts_on_the_esdk/90990557 So without further ado, here's how you get a 'SDK' without building one: 1. Set up all the needed layers and a yocto build directory. 2. Run: $ bitbake meta-ide-support $ bitbake -c populate_sysroot gtk+3 (or any other target or native item that the application developer would need) $ bitbake populate-sysroots 3. Set up the SDK environment: . tmp/deploy/images/qemux86-64/environment-setup-core2-64-poky-linux (adjust accordingly) Et voila! The Unix environment is now set up to use the cross-toolchain from Yocto, exactly as in the SDK. And devtool/bitbake are available to extend it, exactly as in the eSDK. Theare are numerous benefits here: no need to produce, test, distribute and maintain separate SDK artifacts. No two separate environments for the yocto build and the SDK. Less code paths where things can go wrong. Less awkward, gigantic tarballs. Less SDK update headaches: 'updating the SDK' simply means updating the yocto layers with git fetch or layer management tooling. Built-in SDK extensibility: just run bitbake again to add more things to the sysroot, or add layers if even more things are required. How is this tested? Exactly same as the regular SDK: $ bitbake -c testsdk meta-ide-support This runs the same toolchain tests from meta/lib/oeqa/sdk/cases as the regular sdk testing does. (From OE-Core rev: 5c845d7f4ea6ae7ba18ed43180dad28775cace31) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: allow epoxy/galculator tests to run in esdk and direct yocto buildsAlexander Kanavin2022-07-282-2/+4
| | | | | | | | | | Other tests already have similar tweaks. (From OE-Core rev: 3134d19ba15bb783389c40617d5e2b568c7cd81c) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: add a test class for running SDK tests directly in a Yocto buildAlexander Kanavin2022-07-281-0/+43
| | | | | | | | | | | This is a simpler version of the same class in testsdk.py, as it does not need to unpack and set up the SDK, and can proceed to the tests straight away. (From OE-Core rev: be21c62e5bd96164aab9e01168b7a43c6de44c17) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toolchain-scripts.bbclass: adjust toolchain_create_tree_env_script to better ↵Alexander Kanavin2022-07-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | replicate (e)SDK Specifically: 1. Place the environment setup file into $B and not into $TMPDIR, so that the recipe using the class can itself better decide what to do with the file. 2. Use global, unified sysroots (provided through build-sysroots recipe) and not recipe-specific ones, as this allows flexible on-the-fly management of what libraries are available to build applications, without having to modify any recipes, similar to eSDK 'extensible' part. This also requires adjustment of the sstate sametune_samegsigs test, as meta-ide-support becomes dependent on $MACHINE (unified sysroots have it in their paths) and needs to be excluded from the test. 3. Add a few missing settings that have been added to SDK environment files. 4. Add a snippet to the environment setup file that also runs the relocation scripts. In regular SDKs this is executed by the SDK installer, in direct SDK we can do it when setting up the environment. (From OE-Core rev: db5dfd78ae441201778b1175f4fb9a3eba994899) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: Add basic rust cargo testOtavio Salvador2022-07-283-0/+42
| | | | | | | | | | | Add a QA test to the SDK to test that a basic cargo build works. [RP: Tweaked to work for multilibs and updated to match toolchain changes] (From OE-Core rev: d0cfe587bc897e79ef01805cc9a42fbca28c883c) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/gotoolchain: set CGO_ENABLED=1Ross Burton2022-07-251-0/+1
| | | | | | | | | | | | | | | In cross-compiles CGO_ENABLED=1 needs to be set explicitly, as otherwise Go refuses to use it even if CC is already set. This fixes the selftest on setups where the host and the SDK target don't have matching architectures. [ YOCTO #14859 ] (From OE-Core rev: 19be072619d39267df44f23c4c8b64f3808f6148) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/gotoolchain: put writable files in the Go module cacheRoss Burton2022-07-251-6/+1
| | | | | | | | | | | By default 'go mod' creates read-only files, but that just complicates things. Add -modcacherw to make the cache read/write, so it can be cleaned up without needing to chmod. (From OE-Core rev: 7ff30e0d9fe8527cbc2f8ca84e0300fdc84663b6) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: rename git.py to intercept.pyRoss Burton2022-07-252-3/+3
| | | | | | | | | | | | By naming this test class git.py, any attempt to import GitPython (as needed by oelib.buildhistory) failed. As this class exercises the intercepts, rename it to intercept.py. (From OE-Core rev: d557cbbf86767bc2ebf2beb3d70af3b3ca5e0529) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* zlib: remove historical movement of libz.so to /libRoss Burton2022-07-251-5/+4
| | | | | | | | | | | | | Historically there's been a split between /lib for early boot and /usr/lib for everything else, but with modern systems this split is meaningless and incomplete. If a minimal system for early boot is needed, it should be a full minimal system in a initramfs. [RP: Fixed up selftest to match] (From OE-Core rev: 990073dfc167354b4af41db83ac46c18b1aa99d5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager/ipk: do not pipe stderr to stdoutShruthi Ravichandran2022-07-251-10/+13
| | | | | | | | | | | | | | | | Some opkg commands print an error during cleanup when the tmp_dir does not exist and an attempt is made to delete it. The error messages are harmless and the opkg commands eventually succeed. When these commands are run and stderr is piped to stdout, the error messages may clobber the stdout and cause unexpected results while parsing the output of the command. Therefore, when parsing the output of a command, do not pipe stderr to stdout. Instead, capture stderr and stdout separately, and upon success, send stderr to bb.note(). (From OE-Core rev: f2167ae80258253eb47a5b148546b265320284cc) Signed-off-by: Shruthi Ravichandran <shruthi.ravichandran@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base/reproducible: Change Source Date Epoch generation methodsRichard Purdie2022-07-231-1/+0
| | | | | | | | | | | | | | | | | | | | | | | In bc, we found that there are files newer than Changelog (e.g. scan.l) which means after install runs, the timestamp is clamped to SDE which means "bitbake bc -C compile" would then try and regenerate generated files and complain flex was missing. Rather than assuming Changelog/NEWS are magic for dates, drop that scan method and scan all unpacked files. This shouldn't be that slow as they'd be in the disk cache already after an unpack. For bc, this changes SDE by about two minutes and avoids the problems rebuilding. Add a version comment to the task definition as changes in python library code won't trigger a rebuild and we need one here. (From OE-Core rev: 32dda0ad91a9e7946351c897578b4c97ae142341) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime: add test that the kernel has CONFIG_PREEMPT_RT enabledRoss Burton2022-07-201-0/+17
| | | | | | | | | | | This is the absolute bare minimum for testing the RT patches, but it does mean we if we build and boot a RT kernel we can verify that it is what we expect. (From OE-Core rev: 0301d5845115d09299f87683b3efa46f3b4c7be9) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* sstatesig: Include all dependencies in SPDX task signaturesJoshua Watt2022-07-191-0/+9
| | | | | | | | | | | | | | | | | SDPX generation involves looking through BB_TASKDEPDATA for dependencies, then linking to the generated documents for those dependencies. These document links use a checksum to validate the document, which means that if a upstream document changes, all downstream documents must be regenerated to get the new checksum, otherwise the compendium of documents produced by the build will have broken links; therefore all dependent task should be included in the signature (even from "ABI safe" recipes). (From OE-Core rev: 5fe543b9ceec971cf0297ff0ae3b0ccc4703cece) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe/utils: remove the ThreadedPoolJose Quaresma2022-07-151-64/+0
| | | | | | | | | | | | | | | The ThreadedPool in OE-core is mainly because python2 doesn't have threaded pools but python2 is dead for some time now and python3 have a ThreadPoolExecutor. The only local in OE-core where this ThreadedPool is in use is on the sstate.bbclass that is ported to the python3 ThreadPoolExecutor. (From OE-Core rev: aa8fd5e7c2a1125895accfd55ce9320819a10959) Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* selftest/runtime_test/virgl: Disable for all almalinuxRichard Purdie2022-07-151-1/+1
| | | | | | | | | We can't support vgem on RHEL derived distros so disable this test for all almalinux hosts rather than specific versions. (From OE-Core rev: e921f3c1b917072e4c5a110c7dfeeadd2e571bde) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: introduce UBOOT_MKIMAGE_KERNEL_TYPEMing Liu2022-07-151-1/+3
| | | | | | | | | | | | | | | | Sometimes an end user might want to choose another kernel type argument for uboot-mkimage other than "kernel", for instance: "kernel_noload". Let's introduce a variable UBOOT_MKIMAGE_KERNEL_TYPE to support that, and it could be used by BSP layers as well. (From OE-Core rev: e288686e97de1265eeeaf452141e1473867efb1b) (From OE-Core rev: 4eb7bbcc2f08b25387a15b7e4a89ef199783c973) Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/sstatetests: Update test to work with bitbake changesRichard Purdie2022-07-151-0/+2
| | | | | | | | | | | | | | Bitbake is dropping the DL_DIR fallback for local file urls and ensuring local urls are fetchable. This test fails as it copies the meta directory of COREBASE but not scripts and nativesdk-qemu-helper references runqemu from there which doesn't exist in the copied data. Tweak to symlink scripts into position in the copied metadata which avoids the now fatal parsing error. (From OE-Core rev: 5a5199943de5df9a4d44277d07f4313642c34b3a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa: test_invalid_recipe_src_uri expect parse time errorPaulo Neves2022-07-151-3/+1
| | | | | | | | | | | | | Before this commit the test expected a runtime error but the logic of the fetcher changed so that files not found for checksumming cause a fatal interruption of the parse, thuse never reaching the runtime error this test expected. This commit aligns with the bitbake changes to the fetcher. (From OE-Core rev: 8b506bfbdd18dfdb411080f69ef5c0d416b3f2e0) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: devtool: test modify git recipe building from a subdirPaul Eggleton2022-07-141-17/+97
| | | | | | | | | | | | | | Add a test that verifies that devtool modify + devtool finish do the right thing on a recipe that fetches from git and sets S to point to a subdirectory of the source tree. We have a few examples among the core recipes, dos2unix is a convenient one so let's use that. (The test first verifies that that is still true in case the recipe is changed in future.) (From OE-Core rev: a84d9ed14173b0bf467ea78dff4f0f7bae0bc082) Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish: handle patching when S points to subdir of a git repoPaul Eggleton2022-07-141-2/+7
| | | | | | | | | | | | If devtool finish needs to create a patch and have it applied to the sources for a recipe where S points to a subdirectory of the sources, then the patch needs to be applied at the root of the repo i.e. we need to add a patchdir= parameter to the SRC_URI entry. (From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f) Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* patch: handle if S points to a subdirectory of a git repoPaul Eggleton2022-07-141-4/+4
| | | | | | | | | | | | | | | | | If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to a subdirectory of the checked out sources, then we were erroneously initialising the subdirectory as its own git repo. Check if the returned top-level repo directory is a subdirectory of WORKDIR and do not run initialise the source directory if that is the case. (This was a regression introduced with OE-Core revision 6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test that verified the behaviour.) (From OE-Core rev: 9cca53a2bcbf6809615ce5626c86c6ee481a7a76) Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/bbtests: Update message lookup for test_git_unpack_nonetwork_failAlexandre Belloni2022-07-081-1/+1
| | | | | | | | | bitbake's output changed, update the test (From OE-Core rev: 7e64b63f96dd1d71e263e7bbbe6591e51e98395a) Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sysroot.py: Check bitbake return statusPaulo Neves2022-07-081-0/+5
| | | | | | | | | | | bitbake ran but we incorrectly did not assert the exit status needs to be non 0. Now all sysroot tests commands expected to fail are verified to do so. (From OE-Core rev: 5fe8c14f50d414e768588cef0675d8ef296ced77) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: Add cmdline_shebang_wrapper util.Paulo Neves2022-07-011-0/+11
| | | | | | | | | | | | Useful to work around shebang relocation issues, where shebangs are too long or have arguments in them, thus preventing them from using the /usr/bin/env shebang. (From OE-Core rev: 6edc1fffcbe1405d8c309a75643d7d6cd9a92848) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Test staged .la and .pc filesPaulo Neves2022-07-011-0/+32
| | | | | | | | | | | | These files are checked by qa_check_staged but there was no test cases for whether the tests actually worked. Now there are. (From OE-Core rev: 2a96719a201cb7b8db774718adec89dbd7e1aec3) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest: Add test for shebang overflowPaulo Neves2022-07-011-0/+10
| | | | | | | | | | | | | Make sure we do not stage any executable with a bigger shebang than 128. Fixes [1] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 (From OE-Core rev: 280f68528c93b5ffab888c99190accf59e807a3f) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool/devtool: Fix python egg whitespace issues in PACKAGECONFIGThomas Roos2022-07-011-1/+14
| | | | | | | | | | | | | | | | | | | Substitute expressions or whitespace from python egg requires.txt when generating PACKAGECONFIG Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements. Recipetool parses this information to generate the PACKAGECONFIG. These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG. This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable. Also adding an oe-selftest for this. [YOCTO #14446] (From OE-Core rev: a854d95a79e64f3f82abfa4cc1daec750abf4249) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildperf/base.py: skip reduced_proc_pressure directoryAryaman Gupta2022-06-301-1/+1
| | | | | | | | | | | | | | | | Buildperf was designed to skip all non-directories under buildstats, i.e. proc log files. With the /proc/pressure stats being collected in a new directory, the following error was seen: meta/lib/oeqa/buildperf/base.py", line 392, in split_nevr n_e_v, revision = nevr.rsplit('-', 1) ValueError: not enough values to unpack (expected 2, got 1) Add an additional check to skip the reduced_proc_pressure directory. (From OE-Core rev: d3a25dd7aa63ab98c8bb931b4b3bc61807806ed3) Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildstats.py: close /proc/pressure/cpu file descriptorAryaman Gupta2022-06-301-2/+2
| | | | | | | | | | | | Use python 'with' symantics to ensure that the /proc/pressure/cpu file descriptor used in SystemStats init is closed. Previously, this would lead to a single file descriptor being leaked. For example: ResourceWarning: unclosed file <_io.BufferedReader name='/proc/pressure/cpu'> (From OE-Core rev: 643653160cd77d346cdc9b9ec25c7212c7dfe176) Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildstats.bbclass: correct sampling of system statsAryaman Gupta2022-06-291-4/+19
| | | | | | | | | | | | | | | | | | | | | | | | The last time of sampling would be updated within the SystemStats class but not re-recorded into the datastore, leading to multiple samples being collected in the same second in the sample function of buildstats.py. Fix this to collect and store only one sample per second within a certain tolerance to deal with variation in the arrival time. This fix elimates the spikiness of sampled data, in cases where the difference between the current and the last sample is taken. Previously, since many samples per second were recorded, certain types of data would result in a very small elapsed time and hence a small numerical difference. For example, the CPU usage from /proc/stat is a running total of usage and taking the difference between data collected 0.1 seconds apart would result in usage appearing lower than it actually was. (From OE-Core rev: 0e2df45ab066bb4ad2c4f8622ee9c1a8ecdea9cb) Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com> Signed-off-by: Randy MacLeod <randy.macleod@windriver.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildstats.py: enable collection of /proc/pressure dataAryaman Gupta2022-06-291-10/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Linux pressure monitoring system helps determine when system resources are being overutilized by measuring how contended the CPU, IO and memory are. This information can be found under /proc/pressure/ which contains 3 files - cpu, memory and io. In each of the files, the format is as follows: some avg10=70.24 avg60=68.52 avg300=69.91 total=3559632828 full avg10=57.59 avg60=58.06 avg300=60.38 total=3300487258 The "some" state of a given resource represents when one or more tasks are delayed on that resource whereas the "full" state represents when all the tasks are delayed. Currently, we only collect data from the "some" state but the "full" data can simply be appended to the log files if neccessary. The "avg10", "avg60" and "avg300" fields represent the average percentage of time runnable tasks were delayed in the last 10, 60 or 300 seconds respectively. The "total" field represents the total time, in microseconds, that some runnable task was delayed on a resource. More information can be found at: https://www.kernel.org/doc/html/latest/accounting/psi.html and in the source code under kernel/sched/psi.c This commit adds functionality to collect and log the "some" CPU, memory and IO pressure. The "avg10", "avg60" and "avg300" fields are logged without change. In place of the "total" field, the difference between the current "total" and the previous sample's "total" is logged, allowing the measurement of pressure in between each polling interval, as was done for /proc/stat data. The log files are stored in: <build_name>/tmp/buildstats/<build_time>/reduced_proc_pressure/{cpu,io,memory}.log mirroring the directory structure of /proc/pressure. If the /proc/pressure directory does not exist or the resource files can't be read/opened, the reduced_proc_pressure directory is not created. (From OE-Core rev: 061931520b8baa7f3a03bf466aa9ec8bf995bc14) Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com> Signed-off-by: Randy MacLeod <randy.macleod@windriver.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake.conf: Change -dev RDEPENDS to RRECOMMENDSRichard Purdie2022-06-281-1/+1
| | | | | | | | | | | Switch the default DEPENDS for ${PN}-dev to be a RRECOMMENDS instead. This takes advantage of a change to complmentary package globbing to not follow RRECOMMENDS and means and SDK for an image with both openssh and dropbear compoments will now build successfully. (From OE-Core rev: 6f28420ab0e8f2ab5eb06326024777a40aded0a6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package_manager: Change complementary package handling to not include soft ↵Ross Burton2022-06-284-8/+14
| | | | | | | | | | | | | | | | | | | | | | dependencies We've some long standing bugs where the RDEPENDS from -dev packages causes problems, e.g. dropbear and openssh components on an image working fine together but then the SDK failing to build as the main openssh and dropbear packages conflict with each other (pulled in by openssh-dev and dropbear-dev). We propose changing the behavour of complementary package installation to ignore RRECOMMENDS. If we then change the ${PN}-dev dependency on ${PN} to a RRECOMMENDS, we can avoid many of the issues people run into yet still have the desired behaviour of ${PN}-dev pulling in ${PN}. This therefore changes the package manager code so that it doesn't follow RRECOMMENDS for completementary package globs. [RP: Added deb support] (From OE-Core rev: b44b0b9294675f89aa51ff84f532664f4c479677) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib:npm_registry: initial checkinEnrico Scholz2022-06-281-0/+169
| | | | | | | | | | | | | | | | | | | | Helper module to: - generate meta information from package.json content. This data has a format as provided by https://registry.npmjs.org - put this meta information and the corresponding tarball in the nodejs cache. This uses an external, nodejs version specific helper script (oe-npm-cache) shipped in oe-meta To avoid further nodejs version dependencies, future versions of this module might omit the caching completely and serve meta information and tarball by an http server. (From OE-Core rev: 6cd5886ad05fee704e8a5892bd370c360c8c3b54) Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/runtime/scp: Disable scp test for dropbearRichard Purdie2022-06-251-1/+1
| | | | | | | | | | | | | | | Fedora is switching to use sftp as the backend for scp. This means the scp test fails on Fedora 36 hosts with a dropbear target as dropbear doesn't support sftp. This change is in the upstream openssh code, other distros have not yet changed the default but probably will follow. The easiest way to resolve test failures in dropbear images is to stop testing this against dropbear as it is no longer expected to work and will likely spread as the change filters through other distros. (From OE-Core rev: a71fc7d455400f406b0d607be712a1133fe91166) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/sdk: drop the nativesdk-python 2.x testAlexander Kanavin2022-06-251-11/+0
| | | | | | | | | | Python 2.x has been EOL for a while, and so this test never runs. (From OE-Core rev: b687627e9cffb8123c156413f55ea1929f1a7831) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oeqa/selftest/cve_check: add tests for Ignored and partial reportsMarta Rybczynska2022-06-171-0/+82
| | | | | | | | | | | Add testcases for partial reports with CVE_CHECK_REPORT_PATCHED and Ignored CVEs. (From OE-Core rev: 3f7639b90004973782a2e74925fd2e9a764c1090) Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* weston.py: added xwayland testssuesens2022-06-151-0/+8
| | | | | | | | | | | | Test xwayland support if DISTRO_FEATURE X11 is enabled Test checked if xserver listening on display (From OE-Core rev: c8dea13bd65f1695c79eba26b092d411a1699b19) Signed-off-by: ssuesens <sebastian.suesens@baslerweb.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/create-spdx: Add SPDX_PRETTY optionJoshua Watt2022-06-151-2/+2
| | | | | | | | | | | Adds an option to make the SPDX more human-readable (at the expense of a larger files) (From OE-Core rev: 4799594b26f77ed259dc661bf077519b338390c8) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oescripts: change compare logic in OEListPackageconfigTestsMingli Yu2022-06-141-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When multilib enabled and add layers/meta-openembedded/meta-oe in conf/bblayers.conf, it reports below error when run oe-selftest. $ oe-selftest -r oescripts [snip] [20:36:33-0700] 2022-05-16 03:36:33,494 - oe-selftest - INFO - RESULTS - oescripts.OEListPackageconfigTests.test_packageconfig_flags_option_flags: FAILED (585.37s) [snip] It is because the output of "list-packageconfig-flags.py -f" as below: $ ../scripts/contrib/list-packageconfig-flags.py -f [snip] qt lib32-pinentry lib32-wxwidgets nativesdk-pinentry pinentry pinentry-native wxwidgets wxwidgets-native secret lib32-pinentry nativesdk-pinentry pinentry pinentry-native [snip] But the check logic as below: class OEListPackageconfigTests(OEScriptTests): #oe-core.scripts.List_all_the_PACKAGECONFIG's_flags def check_endlines(self, results, expected_endlines): for line in results.output.splitlines(): for el in expected_endlines: if line.split() == el.split(): expected_endlines.remove(el) break def test_packageconfig_flags_option_flags(self): results = runCmd('%s/contrib/list-packageconfig-flags.py -f' % self.scripts_dir) expected_endlines = [] expected_endlines.append("PACKAGECONFIG FLAG RECIPE NAMES") expected_endlines.append("qt nativesdk-pinentry pinentry pinentry-native") expected_endlines.append("secret nativesdk-pinentry pinentry pinentry-native") self.check_endlines(results, expected_endlines) And the test will fail as line.split() doesn't equal el.split() as line.split() is ['lib32-pinentry', 'lib32-wxwidgets', 'nativesdk-pinentry', 'pinentry', 'pinentry-native', 'wxwidgets', 'wxwidgets-native'] and el.split() is ['nativesdk-pinentry', 'pinentry', 'pinentry-native']. So change the compare logic to fix the gap. (From OE-Core rev: 239f22847bcae0cb31769adb0a42b5440173a7c5) Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* base/buildhistory/image-buildinfo: Use common buildcfg functionRichard Purdie2022-06-111-0/+7
| | | | | | | | Reduce code duplication to a common function in oe.buildcfg. (From OE-Core rev: 0b191a1df4c3722defb09dde0c16b1d9e7fe7ef6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildcfg: Drop unused svn revision functionRichard Purdie2022-06-111-11/+0
| | | | | | | | | This isn't used anywhere and everyone is using git now, the functions are now hardcoded as such too. Drop this function. (From OE-Core rev: 09ba96a2d7fa02c7a82758bbf4109b04ffca2c55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/buildcfg: Share common clean/dirty layer functionRichard Purdie2022-06-111-1/+14
| | | | | | | | | The comments even say this was copy/paste code. Move to a shared library function. (From OE-Core rev: ac3de2f850a418673b87e1c454970cb099e191b0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-29 21:03:53 +0000