summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2/wget: Fix failure path for files that are empty or don't existRichard Purdie8 days1-9/+9
| | | | | | | | | | | | | | | When we intercepted the file download to a temp file, we broke the exist/size checks which need to happen before the rename. Correct the ordering. For some reason, python 3.12 exposes this problem in the selftests differently to previous versions. (Bitbake rev: 8714a02e13477a9d97858b3642e05f28247454b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c56bd9a9280378bc64c6a7fe6d7b70847e0b9e6d) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: tests/fetch: Tweak to work on Fedora40Richard Purdie8 days1-3/+3
| | | | | | | | | | | | | | On Fedora40, "localhost" sometimes resolves to ::1 and sometimes to 127.0.0.1 and python only binds to one of the addresses, leading to test failures. Use 127.0.0.1 explicitly to avoid problems of the name resolution, we're trying to test things other than the host networking. (Bitbake rev: 5fa596b3f2e49fbd144347b9800c76de72a29e28) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9adc6da42618f41bf0d6b558d62b2f3c13bedd61) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: fetch2/wget: Canonicalize DL_DIR paths for wget2 compatibilityRudolf J Streif8 days1-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some distributions (namely Fedora Core 40) have started replacing wget with wget2. There are some changes to wget2 that make it incompatible with wget: 1. ftp/ftps is not supported anymore 2. progress 'dot' is not yet supported 3. Relative paths in -P and -O are not correctly dealt with Item 1: Is already dealt with since Scarthgap by only adding the option --passive-ftp when the URL specifies ftp/sftp. While that won't help if ftp/sftp is actually required it at least does not break http/https downloads. Item 2: While not supported it at least does not break the operation. Item 3: If there are relative path components in -P or -O then wget2 only deals with them correctly if there is one, and only one, relative path component at the beginning of the path: -P ./downloads works -P ../downloads works -P ../../downloads does not work -P ./../downloads does not work -P /home/user/downloads/../downloads does not work In cases where there are more than one relative path component at the beginning of the path and/or one or more reltaive path component somewhere in the middle or end of the path, wget2 aborts with the message Internal error: Unexpected relative path: '<path>') Such can happen if DL_DIR includes relative path components e.g. DL_DIR = "${TOPDIR}/../../downloads". This patch canonicalizes DL_DIR before it is passed to wget. (Bitbake rev: 47678142e26bb76d1351886060deff5e75039bc9) Signed-off-by: Rudolf J Streif <rudolf.streif@ibeeto.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3e4208952b086adc510e78c1c5f9cf4550d79dc9) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: tests/fetch: Tweak test to match upstream repo url changeRichard Purdie2024-06-141-1/+1
| | | | | | | | | | Upstream changed their urls, update our test to match. (Bitbake rev: 0791e66a47185d6f202af6be0d39e36a8e41850a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dc391b86540ec5e0a0f1d811c776a22d443b1c06) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: siggen: Enable batching of unihash queriesJoshua Watt2024-06-061-3/+6
| | | | | | | | | | | Uses the batching API of the client to reduce the effect of latency when making multiple queries to the server (Bitbake rev: fdac31655fb5441139f70e50292dbb5774cfdafe) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: hashserv: client: Add batch stream APIJoshua Watt2024-06-062-9/+172
| | | | | | | | | | | | | | Changes the stream mode to do "batch" processing. This means that the sending and reciving of messages is done simultaneously so that messages can be sent as fast as possible without having to wait for each reply. This allows multiple messages to be in flight at once, reducing the effect of the round trip latency from the server. (Bitbake rev: f99a17023b972d0d90dccb111f983655af6ccb87) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: bb: Use namedtuple for Task dataJoshua Watt2024-06-062-18/+39
| | | | | | | | | | | | | Task dependency data is becoming unwieldy with the number of indices it contains. Convert it to use a named tuple instead, which allows members to be indexed by a named property or an index (which allows it to retain backward compatibility). (Bitbake rev: bc7c44affe8e3fa94e92c3bcb8ad85bf11963779) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: runqueue: Improve rehash get_unihash parallelismRichard Purdie2024-06-061-2/+10
| | | | | | | | | | Improve the rehash code to query unihashes in parallel since this is more efficient on slower links. (Bitbake rev: 682fb42420eebf18f8a98bd3992baf9034dac5ac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: runqueue: Process unihashes in parallel at initRichard Purdie2024-06-062-9/+10
| | | | | | | | | | | | | | | Improve the runqueue init code to call unihash queries in parallel since this is faster and more efficient, particularly on slower links with longer round trip times. The call to the function from cooker is unneeded since that function calls prepare() and hence this functionality will already have run, so drop that obsolete call. (Bitbake rev: e0486054c7a4c637446c18608e9983cc8dc4d7fe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: siggen/runqueue: Report which dependencies affect the taskhashjoshua Watt2024-06-062-4/+6
| | | | | | | | | | | | | | Report which task dependencies in BB_TASKDEPDATA are included in the taskhash. This allows tasks to identify which tasks dependencies may change without the task re-running. Knowing this information is important for tasks that want to transfer information from dependencies (such as SPDX) (Bitbake rev: 853423661779023763a87462b623b6e9ff2798b2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: runqueue: Allow rehash loop to exit in case of interruptsRichard Purdie2024-06-061-1/+3
| | | | | | | | | | | The initial hash serve loop exits in the case where interrupts are present but probably checks a bit too often. Tweak that and also allow the slow rehash loop to break on interrupt, improving bitbake Ctrl+C response. (Bitbake rev: ca2f63d84b90cbd99936c9d7815d912d9019a594) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: runqueue: Add timing warnings around slow loopsRichard Purdie2024-06-061-0/+22
| | | | | | | | | | | | | | With hashserve enabled, there are two slow paths/loops, one at initial runqueue generation and also during the rehash process when new outhashes are found. Add timing information at the hashserve log level for when these loops take longer than 30s or 60s overall. This will leave evidence in the logs when things are running particularly slowly. (Bitbake rev: 9ee503c79936b13f1d45f9e43211f77a528cdbfa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: fetch2/gcp: Add missing runfetchcmd importSven Schwermer2024-05-291-0/+1
| | | | | | | | | | This adds the missing import. This bug was introduced with 1ab1d36c. (Bitbake rev: b5159c0373e2e7d403aed16e096ad655f38b1fa7) Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: parse: Improve/fix cache invalidation via mtimeRichard Purdie2024-05-291-4/+8
| | | | | | | | | | | | | | | | | | | | We have been seeing obscure failures in devtool, particularly on newer autobuilder workers where it appears the cache is assumed to be valid when it shouldn't be. We're using the 'seconds' granulation mtime field which is not really a good way of telling if a file has changed. We can switch to the "ns" version which is better however also add in inode number and size as precautions. We already have all this data and tuples are fast so there isn't really any cost to do so. This hopefully fixes [YOCTO #15318]. (Bitbake rev: 2aa519fec30ecf0e55202d6421fbba2f0320e211) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d9e5d313c79500e3c70ab9c3239b6b2180194f67) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: cooker: Handle ImportError for websocketsjoshua Watt2024-05-291-1/+2
| | | | | | | | | | | | | | Handles ImportError when creating a hash equivalence to ping the server. This notifies user earlier with a more precise error if websockets can't be used, and also prevents passing a known bad upstream value to the local server (Bitbake rev: 93190565fd1251e8f47d9a6291739f8b8ed5ec87) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit aa80b3cfc5d16dfba13ca7fb9b78bae179ce3b74) Signed-off-by: Steve Sakoman <steve@sakoman.com>
* bitbake: cooker: Use hash client to ping upstream serverjoshua Watt2024-05-021-4/+3
| | | | | | | | | | | | | | The cooker attempts to connect to the upstream hash equivalent server to warn the user early if it is misconfigured. However, this was making the assumption that it was a raw TCP connection and failed when attempting to use a websocket upstream server. Fix this by creating an hash client and using the ping API to check the server instead of using a raw socket. (Bitbake rev: 8f90d10f9efc9a32e13f6bd031992aece79fe7cc) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump to version 2.8.0Richard Purdie2024-04-162-2/+2
| | | | | | (Bitbake rev: c86466d51e8ff14e57a734c1eec5bb651fdc73ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: client: Fix mode state errorsJoshua Watt2024-04-161-29/+10
| | | | | | | | | | | | | | | | | | Careful reading of the code can contrive cases where poorly timed ConnectionError's will result in the client mode being incorrectly reset to MODE_NORMAL when it should actual be a stream mode for the current command. Fix this by no longer attempting to restore the mode when the connection is setup. Instead, attempt to set the stream mode inside the send wrapper for the stream data, which means that it should always end up in the correct mode before continuing. Also, factor out the transition to normal mode into a invoke() override so it doesn't need to be specified over and over again. (Bitbake rev: 0cd276fd98eeca463518d4a42675fffb18d6b3de) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Capture SSL environment for hashserverJoshua Watt2024-04-161-24/+70
| | | | | | | | | | | | | | Now that the bitbake hash server supports SSL connections, we need to capture a few environment variables which can affect the ability to connect via SSL. Note that the variables are only put in place to affect the environment while actually invoking the server [RP: Tweak to use BB_ORIGENV as well] [RP: Tweak to handle os.environ restore correctly] (Bitbake rev: 0bacf6551821beb8915513b120ae672ae8eb1612) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Handle unclosed functions correctlyRichard Purdie2024-04-141-1/+7
| | | | | | | | | | | | | | | | | A function accidentally defined as: somefunction() { : } which is unclosed due to the space at the end, would currently silently cause breakage. Have the parser throw and error for this. [YOCTO #15470] (Bitbake rev: a7dce72da6be626734486808f1b731247697e638) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: remove unnecessary codeMichael Opdenacker2024-04-141-5/+1
| | | | | | | | | | | | | | In db.py, the ifnull() statement guarantees that the SQL request will return a value. It's therefore unnecessary to test the case when no value is found. (Bitbake rev: e4ae5177861c9a27e93e5a2d3a6c393baecd6416) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: correct error messageMichael Opdenacker2024-04-141-1/+1
| | | | | | | | | | | | | according to db.py, prserv.NotFoundError is returned here when adding a new value to the database failed (Bitbake rev: 4cc4069987edd14f51715dfaf0c6e1a3aa307106) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: remove redundant exception handlerMichael Opdenacker2024-04-141-2/+0
| | | | | | | | | | | | This exception handler is already present in db.py's get_value() code. (Bitbake rev: 2fd38b1bb685ec441f0eb0f28f3d84ba252ba90b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: add extra requestsMichael Opdenacker2024-04-143-1/+96
| | | | | | | | | | | | | | | | | | | | | | | | | Useful for connecting a PR server to an upstream one - "test-package" checks whether the specified package version and arch is known in the database. - "test-pr" checks a specified output hash is found in the database. Otherwise it returns 'None' instead of a new value. - "max-package-pr" returns the highest PR number for (version, arch) entries in the database, and None if not found Add new DB functions supporting the above, plus test_value() which tells whether a given value is available for the specified package and architecture. (Bitbake rev: 0f1474a30f741b760ca81c19dd1d8f3bd5647251) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: capitalization and spacing improvementsMichael Opdenacker2024-04-143-34/+34
| | | | | | | | | | | | | | Choosing only one style of capitalization Add extra space after some commas too Remove idle spaces (Bitbake rev: daad17bccec8cb98ef2fca4262641167500bd46e) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: include parse_address from hashservMichael Opdenacker2024-04-142-26/+24
| | | | | | | | | | | | | | Moving the code and related definitions from hashserv/__init__.py to asyncrpc/client.py, allowing this function to be used in other asyncrpc clients. (Bitbake rev: b67bb05e431414866b8e8c6a4c88d20b9cdb44a3) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Suggested-by: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: use self.logger instead of logger directlyMichael Opdenacker2024-04-141-4/+4
| | | | | | | | | | | | | | | | | In both the PRServerClient and PRClient objects. This aligns with what is done in hashserv/server.py and makes it possible to benefit from possible specializations of the logger in the corresponding super classes, instead of using always the global logger. (Bitbake rev: 5fc6d2b1a5db617e16c1eb9fbd25e821237611d8) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-prserv: replace deprecated optparse by argparseMichael Opdenacker2024-04-141-27/+55
| | | | | | | | | | | | | | | | | | | optparse is deprecated since Python 2.7 Note that this is neither supposed to change the options supported by bitbake-prserv nor the way they are interpreted. Note that in the "--help" output, long options are now reported for example as "--host HOST" instead of "--host=HOST" but both are equivalent anyway, as they already were with optparse. (Bitbake rev: 434cd00a9e5a8ef32088f1a587005adf910a92eb) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: use double quotes by defaultMichael Opdenacker2024-04-145-62/+62
| | | | | | | | | | | | To aligh with the hashserv code (Bitbake rev: 7a6999750791659eaffe49aabfbfba9f37f51913) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: simplify the PRServerClient() interfaceMichael Opdenacker2024-04-141-12/+12
| | | | | | | | | | | | | | | | serv.py: simplify the PRServerClient() interface by passing the server object instead of multiple arguments, and then retrieving the data through this object. This replicates what is done for ServerClient() in hashserv/server.py (Bitbake rev: d3be073218feb4d6e68a751832da4936da485dbc) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Tim Orling <ticotimo@gmail.com> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc/user-manual: Add BB_LOADFACTOR_MAXRichard Purdie2024-04-121-0/+9
| | | | | | | | Document BB_LOADFACTOR_MAX which was recently added. (Bitbake rev: 833b76e9333e317cab5f17d6f7daaecc89c69547) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: Add section for variable contextSimone Weiß2024-04-112-0/+92
| | | | | | | | | | | | | | This is inspired by the same section in the yocto-docs. It aims to provide information in what contexts(recipes, .conf, bbclass,...) a variable is usually used. For that I tried to group similar variables, so that a short overview is given. This was inspired by [YOCTO #14072], but of course does not implement a warning if a variable is used in an unintended context. (Bitbake rev: 5ced476685376b1a32b7fdd9546f9b61c5962aa0) Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com> Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: Make wget --passive-ftp option conditional on ftp/ftpsRob Woolley2024-04-111-1/+4
| | | | | | | | | | | | | | | Fedora 40 introduces wget2 as a drop-in replacement for wget. This rewrite does not currently have support for FTP. This causes the wget fetcher to fail complaining about an unrecognized option. Making --passive-ftp conditional based on the protocol used in the SRC_URI limits the scope of the problem. It also gives us an opportunity to build the older wget as a host tool. (Bitbake rev: f10e630fd7561746d835a4378e8777e78f56e44a) Signed-off-by: Rob Woolley <rob.woolley@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-hashclient: Warn on bad .netrcJoshua Watt2024-03-261-0/+2
| | | | | | | | | If there is an error parsing .netrc, warn the user on stderr (Bitbake rev: 6366ea8d9c284d10bb8f4129004b55239d9022c0) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Add support for hashserve credentialsJoshua Watt2024-03-231-5/+16
| | | | | | | | | | Adds support for hashserver credentials to be specified in the SignatureGenerator (Bitbake rev: 741bef3755fde7bae1386aad575ea704d9fe0969) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Install Git LFS in local repository configDerek Erdmann2024-03-221-1/+1
| | | | | | | | | | | | | | | | | Git uses a lock file to prevent concurrent modifications to the global config, so if unpack tasks for different recipes try to run "git lfs install" simultaneously the operation can fail: error: could not lock config file /home/build/.gitconfig: File exists exit status 255 Run `git lfs install --force` to reset Git configuration. Adding "--local" sets the smudge and clean filters in the local repository's config instead of modifying the user's global config. (Bitbake rev: 328ca4de8422be514fa0d0c9e3cfd36bb9d3e9a7) Signed-off-by: Derek Erdmann <derek.erdmann@sonos.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Fix silent hang issue caused by unexpected stdout ↵Yang Xu2024-03-221-3/+9
| | | | | | | | | | | | | | | | | | | | | content This patch addresses an issue in bitbake-worker where stdout, reserved for status reporting, is improperly accessed by child processes. The problem occurs during the execution of parseRecipe, which calls anonymous functions. If these functions use print-like operations, they can inadvertently output data to stdout. This unexpected data can cause the runqueue to hang silently, if the stdout buffer is flushed before exec_task is executed. To prevent this, the patch redirects stdout to /dev/null and ensures it is flushed prior to the execution of exec_task. (Bitbake rev: 08f3e677d6af27a41a918aaa9da9c1c9b20a0b95) Signed-off-by: Yang Xu <yang.xu@mediatek.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: better estimate number of available cpusFelix Moessbauer2024-03-221-1/+4
| | | | | | | | | | | | | When running in a cgroup which is limited to a subset of cpus (via cpuset.cpus), cpu_count() should return the number of cpus that can be used instead of the number of cpus the system has. This also aligns the semantics with nproc. (Bitbake rev: a029bfe96c6542f178720c72a772b7ede9898118) Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix misleading "no output" msgViswanath Kraleti2024-03-221-1/+4
| | | | | | | | | | | When a command is run with a non-null log, errors are only output to the log and are not returned in the exception. In that case direct users to that logfile instead of telling the command had no output. (Bitbake rev: 944fe0a77932a5559e01ae6035c4bffa5185ea6a) Signed-off-by: Viswanath Kraleti <quic_vkraleti@quicinc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: handle URIs with single-valued query parametersRoss Burton2024-03-222-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Whilst typically the URI query is a list of key-value pairs, that's not actually required by the URI specification. For example: http://example.com/foo?bar is a valid query, but this will result in the fetcher raising an exception: File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__ self.query = self._param_str_split(urlp.query, "&") File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]: ValueError: not enough values to unpack (expected 2, got 1) In this case the query is just "bar", but the fetcher is trying to split it into a key-value pair. The URI object exposes the parsed query explicitly as a dictionary of key-value pairs, so we have to be a little creative here: if a value is None then it isn't a key-value pair, but a bare key. Fix this by handling elements without the deliminator in _param_str_split() (by assigning the value to None), and handle a None value when formatting the query in _param_str_join(). (Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: support NO_COLORPeter A. Bigot2024-03-222-2/+2
| | | | | | | | | | | | | Red text on a black background can make it difficult for people with visual impairments to read the text of error messages. Respect the presence of a non-empty NO_COLOR environment variable as an indication the user doesn't want colorization to be automatically enabled. See: https://no-color.org/ (Bitbake rev: d9986c54cd3d67ed1f7cb636b17696c8d0d4db85) Signed-off-by: Peter A. Bigot <pab@pabigot.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch2/git: Escape parentheses in git src namePhilippe Rivest2024-03-221-1/+1
| | | | | | | | | | | | | | | | | | FIXES [YOCTO #15404] When using git fetcher on a repo with parentheses in its URL, the invocation of the git clone command will fail. The clone directory is not quoted thus the shell will return an error and won't execute the command. (Bitbake rev: b5624ee5643d881afa004571a096a189ab5389b5) Cc: Philippe Rivest <privest@genetec.com> (Bitbake rev: 12f9738577934ad7c99f0770f1392a9d6050e7d6) Signed-off-by: Philippe Rivest <technophil98@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: allow '=' in environment variable valuesdavid d zuhn2024-03-222-3/+3
| | | | | | | | | | | | Limit the split to key & value (2 items) instead of the n items one can get if there are '=' characters in the value. Fixes [YOCTO #15447] (Bitbake rev: 86315961829ab1d137a0265cc246c44d3929e1fb) Signed-off-by: david d zuhn <david.zuhn@sonos.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Fix bug where umask 0 was not being applied to a taskSava Jakovljev2024-03-221-1/+1
| | | | | | | | | | | | | | | | | | * In the current implementation, "umask" variable is initially set to None and overwritten with user-specified value. However, in the worker implementation, a faulty if clause would only check whether the variable contains a value that evaluates to True, and not whether the variable is defined, so the value of 0 would lead to umask not being changed. This bug makes it impossible to have a task set its umask to value 0, for any possible reason it may want to. Fix this bug by extending the condition checked in the worker implementation. (Bitbake rev: 19f9df6c750c592316a0fa18165b68636281fe3e) Signed-off-by: Sava Jakovljev <sjakovljev@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: improve descriptions of '-S printdiff'Alexander Kanavin2024-03-222-4/+7
| | | | | | | (Bitbake rev: becf88c2250a47102c8d36ad8b40839e0bfa9137) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: remove BB_ENV_PASSTHROUGH from preserved_envvars()Michael Opdenacker2024-03-051-1/+0
| | | | | | | | | | | | | preserved_envvars() is used when the BB_ENV_PASSTHROUGH environment variable is not set. Therefore, its code shouldn't return this variable. (Bitbake rev: 0a33b560233b983456178541603ab96fea22238b) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Reported-by: Robert P. J. Day <rpjday@crashcourse.ca> Tested-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv/serv: Fix a PID file removal race on prserv stopYoann Congal2024-03-031-1/+4
| | | | | | | | | | | | | | | | A race condition has happened where the exiting server removed the PID file between the existence check and the removal, resulting in a FileNotFoundError exception. The fix is to ignore the FileNotFoundError exception, the existence check is now redundant so remove it to simplify. Fixes [YOCTO #14341] (Bitbake rev: 40d00bf9308e0bf73a00134a99a012a292daa1c5) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Avoid clean upon failureRichard Purdie2024-03-032-2/+12
| | | | | | | | | | | | | | Currently when git fetches fail, it destroys all the existing local clone data. For large repositories this can introduce long build delays when for example, you just typo'd the git revision hash. The git fetcher should be able to recover most directories so when the fetch is for a git repo, avoid removing things unless clean is explicitly called (e.g. a -c cleanall task). (Bitbake rev: 1b3cd039fe19b24bd4be9a0202a98cdcbb0e9443) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: taskexp_ncurses: fix execution example in introductory commentUlrich Ölmann2024-02-281-1/+1
| | | | | | | | | | | | | | Drop the ".py" file extension from the "-u" option's argument that has been overlooked while applying the original patch (see [1]) to make the example work. While at it sort the recipes' names consistently with respect to what is found in the self-test examples below. [1] https://lore.kernel.org/bitbake-devel/6f2645a7c4db2ae149d387544d2b94209cfed3f4.camel@linuxfoundation.org/ (Bitbake rev: 1f4d517b7a0389f78d1f791135f8dc9120e9912b) Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: asyncrpc: Add support for server headersJoshua Watt2024-02-272-3/+46
| | | | | | | | | | | | Adds support for asyncrpc servers to send connection headers to clients on connection. Since this is a breaking protocol change, clients must opt-in to expect headers from the server, corresponding to a version bump in the client protocol. (Bitbake rev: 1cb2b8be6cc5269553f549285592e47b7d29db03) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-07-02 19:24:08 +0000