summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: tests.codeparser: add tests for shell expansionsAntonin Godard3 hours1-0/+40
| | | | | | | | | | | Tests quotes around `` and $() expansions, nested and multiple expansions, and that escaped quotes are treated as characters by the parser. (Bitbake rev: 610c078474cb9e0a1cb1fabf1411aff6f747b3f3) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: remove redundant list conversionAntonin Godard3 hours1-1/+1
| | | | | | | (Bitbake rev: bd256c4efc1150405c1b57057dcbb6e95c57b9af) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: support shell substitutions in quotesAntonin Godard3 hours1-7/+22
| | | | | | | | | | | | | | | | | | | | | | | | | The current shell substitution mechanism only works without quotes. For example: var1=$(cmd1 ...) Will work and add `cmd1` to the correspondind `run.do_*` file. However, although quite common, this syntax is not supported: var1="$(cmd1 ...)" This commit adds this feature by adding a step to process_words() to check whether we are dealing with quotes first, and by iterating on what's between them to detect new shell substitution candidates. These candidates are tested and parsed like before in the next step. The original `part` being part of the candidates means the syntax var1=$(cmd1 ...) is still valid. (Bitbake rev: 64bf65d3aa85be511348cf1c13ebacbe97286180) Signed-off-by: Antonin Godard <antoningodard@pm.me> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: add bitbake selftestsMichael Opdenacker3 hours2-0/+388
| | | | | | | | | | | | Run them with "bitbake-selftest prserv.tests" (Bitbake rev: c396def1527faddcaf899d8b26f5b996a81f8fa8) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: import simplificationMichael Opdenacker3 hours3-65/+6
| | | | | | | | | | | | | | | | | | | | Simplify the importone() hook: - to make it independent from the "history" mode which is client specific. - remove the "history" parameter - we want all values to be imported for binary reproducibility purposes. - using the store_value() function (which warrants you don't save the same value twice and doesn't write when you're using a read-only server) is enough. (Bitbake rev: 44cb707ce9229e29573594d582a4ff8c6e757ef6) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: store_value() improvementsMichael Opdenacker3 hours1-6/+16
| | | | | | | | | | | | | | | | | | | Add a test_checksum_value() to test whether a (version, pkgarch, checksum, value) entry already exists in the database. This is used to protect the store_value() function from an error when trying to store a duplicate entry in the database. Also check whether the current database is open in read-only mode. (Bitbake rev: 79fc48cfacd9c5cac729ac8bcb70a0686fc2a17b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: avoid possible race condition in database codeMichael Opdenacker3 hours1-14/+11
| | | | | | | | | | | | | | | Remove a possible race condition by allowing a read-only server to create the PR table anyway. This avoids a failure if both a read-only and read-write server try to access an empty database at the same time. (Bitbake rev: ffc7f2b904237e6ec1bf9ddc020d12432b047893) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Suggested-by: Joshua Watt <jpewhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: enable database sharingMichael Opdenacker3 hours2-180/+151
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | sqlite3 can allow multiple processes to access the database simultaneously, but it must be opened correctly. The key change is that the database is no longer opened in "exclusive" mode (defaulting to shared mode). In addition, the journal is set to "WAL" mode, as this is the most efficient for dealing with simultaneous access between different processes. In order to keep the database performance, synchronous mode is set to "off". The WAL journal will protect against incomplete transactions in any given client, however the database will not be protected against unexpected power loss from the OS (which is a fine trade off for performance, and also the same as the previous implementation). The use of a database cursor enabled to remove the _execute() wrapper. The cursor automatically makes sure that the query happens in an atomic transaction and commits when finished. This also removes the need for a "dirty" flag for the database and for explicit database syncing, which simplifies the code. (Bitbake rev: a0b740a4b73260a50295a3ca602e5c18a9ca8bc2) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: add "upstream" server supportMichael Opdenacker3 hours5-125/+338
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a PRSERVER_UPSTREAM variable that makes the local PR server connect to an "upstream" one. This makes it possible to implement local fixes to an upstream package (revision "x", in a way that gives the local update priority (revision "x.y"). Update the calculation of the new revisions to support the case when prior revisions are not integers, but have an "x.y..." format." Set the comments in the handle_get_pr() function in serv.py for details about the calculation of the local revision. This is done by going on supporting the "history" mode that wasn't used so far (revisions can return to a previous historical value), in addition to the default "no history" mode (revisions can never decrease). Rather than storing the history mode in the database table itself (i.e. "PRMAIN_hist" and "PRMAIN_nohist"), the history mode is now passed through the client requests. As a consequence, the table name is now "PRMAIN", which is incompatible with what was generated before, but avoids confusion if we kept the "PRMAIN_nohist" name for both "history" and "no history" modes. Update the server version to "2.0.0". (Bitbake rev: 5ed7aada422b65e81c25cd8de360546470688ab8) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: move code from __init__ to bitbake-prservMichael Opdenacker3 hours2-14/+8
| | | | | | | | | | | | This script was the only user of this code. (Bitbake rev: ae5270e7b569c676def35a30c6f0b3a37106378f) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: declare "max_package_pr" client hookMichael Opdenacker3 hours1-1/+1
| | | | | | | | | | | | Add missing declaration for the max_package_pr client hook (Bitbake rev: c26447748c77bebb7d993ddfcaeec055c7b86966) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add the proof of concept implementationAlexander Kanavin3 hours1-0/+310
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For the rationale and design guidelines please see this message: https://lists.openembedded.org/g/openembedded-architecture/message/1913 Left out for now but will be done later: - config fragment support - tests - documentation - official configuration repository 1. If you don't know where to start, list available configurations, and pick one: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup list Available configurations: poky-alex Poky reference distribution, with alex fixes poky-kirkstone Poky reference distribution, kirkstone long term support release (supported until April 2026) === 2. Then build is initialized this way: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup init poky-alex Initializing build in /home/alex/builds/poky-alex Run /home/alex/builds/poky-alex/build/build.sh to build using this configuration. === Note: 'init' sub-command can also take a path or a URL with a configuration file directly. You can see how those files look like here: https://github.com/kanavin/bitbake-setup-configurations 3. The above message refers to a one-liner shell script that would build the targets specified in the chosen configuration: === alex@Zen2:/srv/work/alex/bitbake$ cat /home/alex/builds/poky-alex/build/build.sh . /home/alex/builds/poky-alex/build/init-build-env && bitbake core-image-minimal === 4. To check if the build configuration needs to be updated, run: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup status ~/builds/poky-alex/ Configuration has not changed. === If the configuration has changed, you will see the difference: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup status ~/builds/poky-alex/ Top level configuration has changed: --- /home/alex/builds/poky-alex/config/poky-alex.conf.json 2024-04-18 13:42:54.312460823 +0200 +++ /home/alex/builds/poky-alex/config-tmp-b413az6s/poky-alex.conf.json 2024-04-18 13:50:42.635433203 +0200 @@ -7,7 +7,7 @@ "uri": "git://git.yoctoproject.org/poky-contrib" } }, - "rev": "akanavin/sstate-for-all" + "rev": "akanavin/sstate-for-all-and-everyone" }, "path": "poky" } === If the configuration has not changed, but layer revisions referred to it have (for example if the configuration specifies a tip of a branch), you will see that too: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup status ~/builds/poky-alex/ Layer repository git://git.yoctoproject.org/poky-contrib checked out into /home/alex/builds/poky-alex/layers/poky updated revision akanavin/sstate-for-all from 6b842ba55f996b27c900e3de78ceac8cb3b1c492 to aeb73e29379fe6007a8adc8d94c1ac18a93e68de === 5. If the configuration has changed, you can bring it in sync with: === alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup update ~/builds/poky-alex/ Layer repository git://git.yoctoproject.org/poky-contrib checked out into /home/alex/builds/poky-alex/layers/poky updated revision akanavin/sstate-for-all from 6b842ba55f996b27c900e3de78ceac8cb3b1c492 to aeb73e29379fe6007a8adc8d94c1ac18a93e68de ... (skip git output) Existing build directory renamed to /home/alex/builds/poky-alex/build-backup.20240418135458 Run /home/alex/builds/poky-alex/build/build.sh to build using this configuration. === Note that it will also rename the existing build directory, and print changes in bitbake configuration (diff of content of build/conf/) if that has changed. I can't at the moment think of anything more clever that is also not much more brittle or complex to implement, but open to suggestions. 6. To make it easier to review the code, please also review the data it's operating on: === alex@Zen2:/srv/work/alex/bitbake$ ls ~/.bitbake-setup/ cache configurations downloads alex@Zen2:/srv/work/alex/bitbake$ ls ~/builds/poky-alex/ build build-backup.20240418135458 config config-upstream.json layers === (Bitbake rev: c945815bfdf3835c692094d0a1502690a407d741) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure generateTaskDepTreeData fails for NoProviderHEADmasterRichard Purdie6 hours1-3/+3
| | | | | | | | | | If an invalid provider is requested, error out early rather than trying to build partial runqueue data structures as the taskdep UI will have exited after seeing the bad provider. (Bitbake rev: a478087998cb794cc4e31189b3ce07973d3949bc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Improve/fix cache invalidation via mtimeRichard Purdie18 hours1-4/+8
| | | | | | | | | | | | | | | | | | We have been seeing obscure failures in devtool, particularly on newer autobuilder workers where it appears the cache is assumed to be valid when it shouldn't be. We're using the 'seconds' granulation mtime field which is not really a good way of telling if a file has changed. We can switch to the "ns" version which is better however also add in inode number and size as precautions. We already have all this data and tuples are fast so there isn't really any cost to do so. This hopefully fixes [YOCTO #15318]. (Bitbake rev: d9e5d313c79500e3c70ab9c3239b6b2180194f67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc/client: Fix websockets minimum version for python 3.10Richard Purdie31 hours1-1/+3
| | | | | | | | | | | | python 3.10 support is only available in websockets 10.0 and later: https://github.com/python-websockets/websockets/commit/08d8011132ba038b3f6c4d591189b57af4c9f147 Update the version for this case. This avoids failures on Ubuntu 22.04. (Bitbake rev: 0e4767c4a880408750e1a6855270c5a4eef8383d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: update to version 2.9.1Richard Purdie9 days2-2/+2
| | | | | | | | This allow the use of new siggen API (Bitbake rev: e53503546990adeab67b6d044fcce59dc5a3f455) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Check websockets versionjoshua Watt9 days1-0/+22
| | | | | | | | | | | Checks that the minimum version of the websockets module is present, and if not raises an ImportError. This allows the user to get earlier feedback if using websockets is going to succeed (Bitbake rev: 330ea6914aad65dc8b34c986c44779820c392f03) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle ImportError for websocketsjoshua Watt9 days1-1/+2
| | | | | | | | | | | | Handles ImportError when creating a hash equivalence to ping the server. This notifies user earlier with a more precise error if websockets can't be used, and also prevents passing a known bad upstream value to the local server (Bitbake rev: aa80b3cfc5d16dfba13ca7fb9b78bae179ce3b74) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen/runqueue: Report which dependencies affect the taskhashjoshua Watt9 days2-4/+6
| | | | | | | | | | | | | Report which task dependencies in BB_TASKDEPDATA are included in the taskhash. This allows tasks to identify which tasks dependencies may change without the task re-running. Knowing this information is important for tasks that want to transfer information from dependencies (such as SPDX) (Bitbake rev: a313b4f07727e8187526157ba039911c3f73dd46) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/svn: Fix mirroring issue with svnKari Sivonen9 days2-1/+12
| | | | | | | | | | | | Add return false to supports_checksum for svn fetcher which fhis fixes MIRROR usage for svn uris. Also add a testcase. [YOCTO #15473] (Bitbake rev: 21cfc7ae9a19f39ac8904e1c3466e7e499ac523f) Signed-off-by: Kari Sivonen <kari.sivonen@live.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Handle conflict between cwd and cleandirsRichard Purdie9 days1-0/+2
| | | | | | | | | | If the cwd of the task is also a cleandirs, you would see warnings from bitbake about being unable to obtain cwd during the task execution. Tweak the code to detect this and avoid the warnings. (Bitbake rev: 6c7fd60c10955b0f23f64b25b5b4e154eb22a8f8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/crate: add upstream latest version check functionAlexander Kanavin9 days2-0/+25
| | | | | | | | | | | This is actually rather easy: crate web API provides a json with all the versions, for example: https://crates.io/api/v1/crates/cargo-c/versions (Bitbake rev: f6c2755db9a1f88c8534193b420fa31d135945e6) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-layers: adapt force option to not use tinfoilSimone Weiß9 days2-6/+12
| | | | | | | | | | | | | | | | | | Fixes [YOCTO #15417] When a layer adds a new dependency after it was added to a conf, it can not be removed w/o this dependency in the setup. Even the dependent layer can not be added, as the tinfoil setup will fail. Adapt --force to not perform the tinfoil at all, the use will be at own risk, i.e. the added layers might not parse properly afterwards. This is not merged into the force option with -F as it even changes the loading of plugins from other layers and is hence even more invasive as force. Instead force can now be speciefied multiple times and is counted. (Bitbake rev: 541fa7f582133949563e65f2d43c4b16e873e5c1) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Use hash client to ping upstream serverjoshua Watt2024-05-021-4/+3
| | | | | | | | | | | | | | The cooker attempts to connect to the upstream hash equivalent server to warn the user early if it is misconfigured. However, this was making the assumption that it was a raw TCP connection and failed when attempting to use a websocket upstream server. Fix this by creating an hash client and using the ping API to check the server instead of using a raw socket. (Bitbake rev: 5e84c13a6c594ed34c341849806657ddda206714) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gcp: Add missing runfetchcmd importSven Schwermer2024-04-201-0/+1
| | | | | | | | | This adds the missing import. This bug was introduced with 1ab1d36c. (Bitbake rev: 97ffe14311407f6e705ec24b70870ab32f0637b9) Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.9.0 development version postreleaseRichard Purdie2024-04-162-2/+2
| | | | | | (Bitbake rev: 67a1aa8dbb3cb3a30fa7d697431ebb30323e4f28) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.8.0Richard Purdie2024-04-162-2/+2
| | | | | | (Bitbake rev: c86466d51e8ff14e57a734c1eec5bb651fdc73ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: client: Fix mode state errorsJoshua Watt2024-04-161-29/+10
| | | | | | | | | | | | | | | | | | Careful reading of the code can contrive cases where poorly timed ConnectionError's will result in the client mode being incorrectly reset to MODE_NORMAL when it should actual be a stream mode for the current command. Fix this by no longer attempting to restore the mode when the connection is setup. Instead, attempt to set the stream mode inside the send wrapper for the stream data, which means that it should always end up in the correct mode before continuing. Also, factor out the transition to normal mode into a invoke() override so it doesn't need to be specified over and over again. (Bitbake rev: 0cd276fd98eeca463518d4a42675fffb18d6b3de) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Capture SSL environment for hashserverJoshua Watt2024-04-161-24/+70
| | | | | | | | | | | | | | Now that the bitbake hash server supports SSL connections, we need to capture a few environment variables which can affect the ability to connect via SSL. Note that the variables are only put in place to affect the environment while actually invoking the server [RP: Tweak to use BB_ORIGENV as well] [RP: Tweak to handle os.environ restore correctly] (Bitbake rev: 0bacf6551821beb8915513b120ae672ae8eb1612) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Handle unclosed functions correctlyRichard Purdie2024-04-141-1/+7
| | | | | | | | | | | | | | | | | A function accidentally defined as: somefunction() { : } which is unclosed due to the space at the end, would currently silently cause breakage. Have the parser throw and error for this. [YOCTO #15470] (Bitbake rev: a7dce72da6be626734486808f1b731247697e638) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: remove unnecessary codeMichael Opdenacker2024-04-141-5/+1
| | | | | | | | | | | | | | In db.py, the ifnull() statement guarantees that the SQL request will return a value. It's therefore unnecessary to test the case when no value is found. (Bitbake rev: e4ae5177861c9a27e93e5a2d3a6c393baecd6416) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: correct error messageMichael Opdenacker2024-04-141-1/+1
| | | | | | | | | | | | | according to db.py, prserv.NotFoundError is returned here when adding a new value to the database failed (Bitbake rev: 4cc4069987edd14f51715dfaf0c6e1a3aa307106) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: remove redundant exception handlerMichael Opdenacker2024-04-141-2/+0
| | | | | | | | | | | | This exception handler is already present in db.py's get_value() code. (Bitbake rev: 2fd38b1bb685ec441f0eb0f28f3d84ba252ba90b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: add extra requestsMichael Opdenacker2024-04-143-1/+96
| | | | | | | | | | | | | | | | | | | | | | | | | Useful for connecting a PR server to an upstream one - "test-package" checks whether the specified package version and arch is known in the database. - "test-pr" checks a specified output hash is found in the database. Otherwise it returns 'None' instead of a new value. - "max-package-pr" returns the highest PR number for (version, arch) entries in the database, and None if not found Add new DB functions supporting the above, plus test_value() which tells whether a given value is available for the specified package and architecture. (Bitbake rev: 0f1474a30f741b760ca81c19dd1d8f3bd5647251) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: capitalization and spacing improvementsMichael Opdenacker2024-04-143-34/+34
| | | | | | | | | | | | | | Choosing only one style of capitalization Add extra space after some commas too Remove idle spaces (Bitbake rev: daad17bccec8cb98ef2fca4262641167500bd46e) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: include parse_address from hashservMichael Opdenacker2024-04-142-26/+24
| | | | | | | | | | | | | | Moving the code and related definitions from hashserv/__init__.py to asyncrpc/client.py, allowing this function to be used in other asyncrpc clients. (Bitbake rev: b67bb05e431414866b8e8c6a4c88d20b9cdb44a3) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Suggested-by: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: use self.logger instead of logger directlyMichael Opdenacker2024-04-141-4/+4
| | | | | | | | | | | | | | | | | In both the PRServerClient and PRClient objects. This aligns with what is done in hashserv/server.py and makes it possible to benefit from possible specializations of the logger in the corresponding super classes, instead of using always the global logger. (Bitbake rev: 5fc6d2b1a5db617e16c1eb9fbd25e821237611d8) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-prserv: replace deprecated optparse by argparseMichael Opdenacker2024-04-141-27/+55
| | | | | | | | | | | | | | | | | | | optparse is deprecated since Python 2.7 Note that this is neither supposed to change the options supported by bitbake-prserv nor the way they are interpreted. Note that in the "--help" output, long options are now reported for example as "--host HOST" instead of "--host=HOST" but both are equivalent anyway, as they already were with optparse. (Bitbake rev: 434cd00a9e5a8ef32088f1a587005adf910a92eb) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: use double quotes by defaultMichael Opdenacker2024-04-145-62/+62
| | | | | | | | | | | | To aligh with the hashserv code (Bitbake rev: 7a6999750791659eaffe49aabfbfba9f37f51913) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: simplify the PRServerClient() interfaceMichael Opdenacker2024-04-141-12/+12
| | | | | | | | | | | | | | | | serv.py: simplify the PRServerClient() interface by passing the server object instead of multiple arguments, and then retrieving the data through this object. This replicates what is done for ServerClient() in hashserv/server.py (Bitbake rev: d3be073218feb4d6e68a751832da4936da485dbc) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/user-manual: Add BB_LOADFACTOR_MAXRichard Purdie2024-04-121-0/+9
| | | | | | | | Document BB_LOADFACTOR_MAX which was recently added. (Bitbake rev: 833b76e9333e317cab5f17d6f7daaecc89c69547) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: Add section for variable contextSimone Weiß2024-04-112-0/+92
| | | | | | | | | | | | | | This is inspired by the same section in the yocto-docs. It aims to provide information in what contexts(recipes, .conf, bbclass,...) a variable is usually used. For that I tried to group similar variables, so that a short overview is given. This was inspired by [YOCTO #14072], but of course does not implement a warning if a variable is used in an unintended context. (Bitbake rev: 5ced476685376b1a32b7fdd9546f9b61c5962aa0) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftpsRob Woolley2024-04-111-1/+4
| | | | | | | | | | | | | | | Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Rob Woolley <rob.woolley@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-hashclient: Warn on bad .netrcJoshua Watt2024-03-261-0/+2
| | | | | | | | | If there is an error parsing .netrc, warn the user on stderr (Bitbake rev: 6366ea8d9c284d10bb8f4129004b55239d9022c0) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add support for hashserve credentialsJoshua Watt2024-03-231-5/+16
| | | | | | | | | | Adds support for hashserver credentials to be specified in the SignatureGenerator (Bitbake rev: 741bef3755fde7bae1386aad575ea704d9fe0969) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Install Git LFS in local repository configDerek Erdmann2024-03-221-1/+1
| | | | | | | | | | | | | | | | | Git uses a lock file to prevent concurrent modifications to the global config, so if unpack tasks for different recipes try to run "git lfs install" simultaneously the operation can fail: error: could not lock config file /home/build/.gitconfig: File exists exit status 255 Run `git lfs install --force` to reset Git configuration. Adding "--local" sets the smudge and clean filters in the local repository's config instead of modifying the user's global config. (Bitbake rev: 328ca4de8422be514fa0d0c9e3cfd36bb9d3e9a7) Signed-off-by: Derek Erdmann <derek.erdmann@sonos.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Fix silent hang issue caused by unexpected stdout ↵Yang Xu2024-03-221-3/+9
| | | | | | | | | | | | | | | | | | | | | content This patch addresses an issue in bitbake-worker where stdout, reserved for status reporting, is improperly accessed by child processes. The problem occurs during the execution of parseRecipe, which calls anonymous functions. If these functions use print-like operations, they can inadvertently output data to stdout. This unexpected data can cause the runqueue to hang silently, if the stdout buffer is flushed before exec_task is executed. To prevent this, the patch redirects stdout to /dev/null and ensures it is flushed prior to the execution of exec_task. (Bitbake rev: 08f3e677d6af27a41a918aaa9da9c1c9b20a0b95) Signed-off-by: Yang Xu <yang.xu@mediatek.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: better estimate number of available cpusFelix Moessbauer2024-03-221-1/+4
| | | | | | | | | | | | | When running in a cgroup which is limited to a subset of cpus (via cpuset.cpus), cpu_count() should return the number of cpus that can be used instead of the number of cpus the system has. This also aligns the semantics with nproc. (Bitbake rev: a029bfe96c6542f178720c72a772b7ede9898118) Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix misleading "no output" msgViswanath Kraleti2024-03-221-1/+4
| | | | | | | | | | | When a command is run with a non-null log, errors are only output to the log and are not returned in the exception. In that case direct users to that logfile instead of telling the command had no output. (Bitbake rev: 944fe0a77932a5559e01ae6035c4bffa5185ea6a) Signed-off-by: Viswanath Kraleti <quic_vkraleti@quicinc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: handle URIs with single-valued query parametersRoss Burton2024-03-222-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Whilst typically the URI query is a list of key-value pairs, that's not actually required by the URI specification. For example: http://example.com/foo?bar is a valid query, but this will result in the fetcher raising an exception: File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__ self.query = self._param_str_split(urlp.query, "&") File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]: ValueError: not enough values to unpack (expected 2, got 1) In this case the query is just "bar", but the fetcher is trying to split it into a key-value pair. The URI object exposes the parsed query explicitly as a dictionary of key-value pairs, so we have to be a little creative here: if a value is None then it isn't a key-value pair, but a bare key. Fix this by handling elements without the deliminator in _param_str_split() (by assigning the value to None), and handle a None value when formatting the query in _param_str_join(). (Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-17 16:55:37 +0000