summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2/git: Add tag to shallow clone tarball nameRichard Purdie11 days1-2/+2
| | | | | | | | | | | It makes sense to allow the shallow clone mirror tarball name to include the tag name so that tags can be added to existing urls and the tarballs will be maintained correctly. The code already allows this to be done easily just by moving the tag handling code. (Bitbake rev: 68fce3be14e4dd801661f4ef302d229fb16a04b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Improve shallow clone tag fetchingRichard Purdie11 days1-1/+1
| | | | | | | | | | Currently, tags are fetched as just a name, which works but means they're not seen as tags by git commands like git describe. Instead, fetch them as refs/tags/XXX which means such commands then work correctly. (Bitbake rev: c1f30ad61f5e55beb377451887bbbc5cb569f2e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: allow overriding default unpack directory with a variableAlexander Kanavin2025-06-201-1/+1
| | | | | | | | | | | This will allow setting the variable in oe-core such that it matches S, which in turn will allow dropping S from almost all git:// fetching recipes, and not having to set it in newly written ones. (Bitbake rev: 378db0fdd95f8704fccd852452555bfddcad384d) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/az: Add sanity check and clarify documentationRobbin Van Damme2025-06-021-0/+4
| | | | | | | | | | | | AZ_SAS token should be prefixed with a question mark. Add a sanity check for this and fix the documentation. [YOCTO #15882] (Bitbake rev: 22011765202514600314732b97f1bb938e21f585) Signed-off-by: Robbin Van Damme <robbinvandamme@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Ensure a valid symlink in `PREMIRRORS` case when using ↵Stefan Koch2025-05-282-1/+16
| | | | | | | | | | | | shallow cloning - Since `ud.path` contains in that case the `PREMIRRORS` prefix path, this change ensures that a correct symlink is set up. (Bitbake rev: 37ed18e45aa17406162efc5ee3ddb2d6b33d07b9) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Move the `ensure_symlink()` function into the `FetchMethod` ↵Stefan Koch2025-05-281-19/+17
| | | | | | | | | class (Bitbake rev: d934763f91b9bda85f273a44f12a04a3ca19fc28) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: add support for .debs containing uncompressed data tarballsIsaac True2025-05-281-2/+2
| | | | | | | | | | | Some .deb files contain uncompressed data tarballs which do not have an additional file extension after `.tar`. Add support for such cases while preserving the existing behaviour. (Bitbake rev: e3834deb427ceb1d1ae9a96c1e0dec990d7f3805) Signed-off-by: Isaac True <isaac.true@emlix.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: fix shallow clone for tag containing slashChen Qi2025-05-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | If a tag contains slash, e.g., debian/5.22, then shallow clone fails because it's using a wrong ref. To reproduce the issue, add the following lines in local.conf: BB_GIT_SHALLOW = "1" BB_GENERATE_SHALLOW_TARBALLS = "1" And then run 'bitbake debianutils -c fetch'. What the original os.path.basename(ref) wanted to do is to remove the strings such as refs/heads/. So we do it explitly to fix this issue. Fixes: [YOCTO #15862] (Bitbake rev: c6d6999f1ed01e7445b8f177a888038edacf555c) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix LFS object checkout in submodulesPhilip Lorenz2025-05-081-6/+5
| | | | | | | | | | | | | | | Skipping smudging prevents the LFS objects from replacing their placeholder files when `git submodule update` actually checks out the target revision in the submodule. Smudging cannot happen earlier as the clone stored in `.git/modules` is bare. This should be fine as long as all LFS objects are available in the download cache (which they are after the other fixes are applied). (Bitbake rev: d270e33a07c50bb9c08861cf9a6dc51e1fd2d874) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix incorrect lfs parametrization for submodulesPhilip Lorenz2025-05-081-1/+1
| | | | | | | | | | | The existing code would pass `True` or `False` to the git fetcher. As the fetcher expects `lfs` to be set to `1` this always lead to LFS fetching being disabled. (Bitbake rev: 5e487a5a096400271ed1e29b0df72903f2304e49) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Use git-lfs fetch to download objectsPhilip Lorenz2025-05-081-24/+1
| | | | | | | | | | | | | | | | | | | Its not clear which exact git-lfs versions failed to work with bare repositories, however git-lfs 2.13.2 which is shipped by Debian 10 (i.e. the oldest supported distribution by scarthgap) shows no issue when fetching into a bare repository. Switch to git-lfs fetch which in turn eliminates issues seen when using the gitsm fetcher with submodules utilizing lfs. In these scenarios, fetching of LFS objects did not actually happen as the gitsm fetcher parametrizes the to be fetched repositories with `bareclone=1` which in turn means that the target revision was never checked out (and therefore no LFS objects were fetched). (Bitbake rev: 501d3317df5780a5b0e3c2562b26ed276c1dbe90) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Simplify git LFS detectionPhilip Lorenz2025-05-081-11/+1
| | | | | | | | | | | | Its unclear why this function does not operate on the desired source revision to begin with (which really should be the decider on whether a particular source revision uses LFS or not). Simplify the decision logic by always checking the `.gitattributes` file of the target revision. (Bitbake rev: b3faa0ce5c0a6945f26b5b303a7f38c00d132397) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Check for git-lfs existence before using itPhilip Lorenz2025-05-081-8/+18
| | | | | | | | | | | | | So far, existence of `git-lfs` was only checked during unpacking. As the binary is also used in earlier steps also check for its existence there. Additionally, factor out the LFS existence check into a dedicated function and call it wherever git-lfs is used for the first time. (Bitbake rev: 5818367db9b261b7e07c347d38044e6cba8f9727) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Clean up no longer used name parameterPhilip Lorenz2025-04-291-4/+4
| | | | | | | | | There's no need to pass `name` when it is no longer used. (Bitbake rev: b132d35dee643e270e3e6dd536dcc90334a0111c) Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: correct typo in comment and stylePeter Marko2025-04-101-3/+3
| | | | | | | | | | | | Correct typo Fash -> Fast. With it also delete some trailing spaces which are unnecessary and editors are trying to remove when changing anything in the file. (Bitbake rev: b96040a0a066344527f0bc6ca4d09cda6b9d2f11) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: always fetch lfs when creating shallow tarballPeter Marko2025-04-101-3/+2
| | | | | | | | | | | | | | | | | | Before fast shallow tarballs the shallow tarball with lfs worked only if the original .../downloads/git2/... directory existed. Once it was not there, do_unpack on shallow tarball without lfs files failed due to disabled network. It was fixed for fast shallow tarballs and this commit fixes the other scenarios, too. This can happen when such shallow tarball is put to mirror or if someone does cleanup to reclaim disk space. (Bitbake rev: 4e6fb31a88448cdacf4e9f84cf0a8a8035f84d63) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch: Fix BB_FETCH_PREMIRRORONLY for git mirror tarballsJulian Haller2025-04-101-0/+4
| | | | | | | | | | | | | When invoking the original git fetcher after downloading a mirror tarball, BB_FETCH_PREMIRRORONLY is ignored. This leads to git fetch commands targeting the upstream source being executed silently. Ensure setting BB_NO_NETWORK before invoking the original fetcher. While this was only observed for git, setting this in general for all fetcher types makes sense at this location. (Bitbake rev: 1b1321f2b60c0a66159e3f20c6befcb0b3ccc4c7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Handle srcrevs for annotated tags in tag checkRichard Purdie2025-04-031-1/+6
| | | | | | | | | | | | | | If SRCREV points at an annotated tag, the comparision code can fail as the resolved tag might not be the same sha. Handle this by also resolving the SRCREV. We only need to do this if they don't match in the first place for a minor performance win. Also add a test for this. (Bitbake rev: 136c06e251de68ed64355ec6b47a522ff3a372e3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: use long arguments for clarityRoss Burton2025-03-271-5/+5
| | | | | | | | | | | It's best practise to use long-form arguments in scripts where the conciseness of short arguments is less useful than in an interactive terminal. (Bitbake rev: 54039bc9d169871af6d36578df4c21bff296f6de) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: don't conditionalise --continueRoss Burton2025-03-271-6/+1
| | | | | | | | | | | | If the target file exists, we pass --continue so that wget will try to resume the download instead of starting from the beginning. However if the file doesn't exist --continue has no effect, so there's no need to conditionalise the use of the argument. (Bitbake rev: a5ee50d4d2b2e6c00abe1551382afd9799345dbf) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: consider downloadfilename when checking for upstreamRoss Burton2025-03-271-1/+5
| | | | | | | | | | | | | | | | | | | | | latest_versionstring() currently looks at just the end of the URI when guessing what the filename to look for is, but this doesn't work if the URL filename is not simple. For example, miniupnpd has a SRC_URI of: http://miniupnp.tuxfamily.org/files/download.php?file=${BP}.tar.gz;downloadfilename=${BP}.tar.gz The filename component of this is "download.php", which causes the heuristics in latest_versionstring() to exit early. Instead, if the downloadfilename is set then use that, as it's often the actual filename that we're after. (Bitbake rev: 2d5f135e997d13fabda0ad266fd5c928ee33f487) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gomod: Fix mirroring problemChristian Lindeberg2025-03-271-13/+22
| | | | | | | | | | | | Build the 'downloadfilename' parameter by replacing path separators in the module path like the git fetcher builds the mirror tar ball name. Copy the downloaded file in the fetcher's unpack method like the crate fetcher instead of calling the base fetcher's unpack method. (Bitbake rev: 7762cea087597019460d66b04268757bd46befdf) Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: Drop multiple branch/revision support for single git urlsRichard Purdie2025-03-204-170/+134
| | | | | | | | | | | | | | | | | We used to use this for bare clones where a single git url could handle multiple revisions (which implied multiple branches). We don't use this any more and I doubt we'd want to go back to it. If we remove it, we can simplfy the looping in the code which seems desireable. This patch does change the warning for missing branch parameters to a error. The message has hinted about that for long enough. Some test cases are removed since they are no longer needed. (Bitbake rev: 2515fbd10824005fa7f34e87706000c079920366) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Ensure tags work with shallow clonesRichard Purdie2025-03-191-0/+2
| | | | | | | | | | If we want to validate a tag when using shallow clones, we need to ensure the tag is imported to the clone. Adding to extra_refs allows this to happen ensuring tag validation in shallow clones works. (Bitbake rev: 03945475886b8d31c7eccc80fe594ab5306bcf5d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Rework tag parameter handlingRichard Purdie2025-03-192-8/+17
| | | | | | | | | | | | | | | | | | | | | | Currently bitbake disallows tag parameters along with revision parameters. This isn't great since quite often, we'd like to verify that a given revision does match some tag. At the same time we don't want to or need to access the network to verify this, which normally a tag would require. Rework the code so that tag and revisions can both be specified together. Verify that any tag specified matches the revision in use at unpack time. This means we can start requiring people to put tags in git SRC_URIs when revisions are used, making review a little easier that it isn't some random revision. The test that is dropped looks like a different test but the comment is a copy and paste error. The SRCREV/rev mismatch test remains, this removes the rev and tag set test. (Bitbake rev: d591d7633fe8d739ec00395920e44910b0b77e27) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Restore escape quoting for the git url when usedPatrik Nordvall2025-03-181-1/+2
| | | | | | | | | | | | | | | | | | | | | | This fixes a bug where escapes in the url path would not be properly restored for the git commands in the git fetcher. For example, a space which is encoded as '%20' was not properly encoded before the clone command. e.g. SRC_URI="git://git.openembedded.org/bitbake%20example/bitbake;protocol=https" resulted in git clone 'https://git.openembedded.org/bitbake example/bitbake' instead of git clone 'https://git.openembedded.org/bitbake%20example/bitbake' (Bitbake rev: be48024253b93215bb110cd1d05925e789ec9680) Signed-off-by: Patrik Nordvall <patrik.nordvall95@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm: Adding npmrc support for private registry authenticationEric Meyers2025-03-091-0/+15
| | | | | | | | | (Bitbake rev: 5fa6137b6d98544766f3152b874e67d04fafb88f) Signed-off-by: Eric Meyers <eric.meyers@arthrex.com> Cc: Geoff Parker <geoffrey.parker@arthrex.com> Cc: Chuck Wolber <chuckwolber@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Partial revert decodeurl() to not use the URI classStefan Herbrechtsmeier2025-03-071-3/+51
| | | | | | | | | | | | | | | | | | | | | | | | This partial reverts commit a5d569c94700f04b8193c6bccae5af619931b00f which changes decodeurl() to use the URI class to parse the URL instead of doing it itself. While reusing code is generally a good idea, using urllib.parse.urlparse() (which the URI class does) to parse the regular expression "URLs" that are used in PREMIRRORS and MIRRORS does not work. A regular expression URL containing https?://... would be silently ignored, while a URL using a negative lookahead such as git://(?!internal\.git\.server).*/.* would result in a cryptic error: Exception: re.error: missing ), unterminated subpattern at position 0 The problem is that urllib.parse.urlparse() treats the ? as the start of URL parameters and thus stops parsing whatever part of the URL it was parsing. Restore the old function and use it in the PREMIRRORS and MIRRORS code. (Bitbake rev: f8a7712754e6d0199a0d227fca288307b935368d) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/gitsm: Unpack even when `ud.clonedir` is not availableStefan Koch2025-03-061-1/+4
| | | | | | | (Bitbake rev: 13d76361ec37faecd84e7b81da22ada7d4e0ba90) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Add support for fast initial shallow fetchStefan Koch2025-03-061-29/+85
| | | | | | | | | | | | | | | | | | | | | | | | When `ud.shallow == 1`: - Prefer an initial shallow clone over an initial full bare clone, while still utilizing any already existing full bare clones. - If the Git error "Server does not allow request for unadvertised object" occurs, the initial full bare clone is fetched automatically. This may happen if the Git server does not allow the request or if the Git client has issues with this functionality, especially with the Git client from Ubuntu 20.04. This improves: - Resolve timeout issues during initial clones on slow internet connections by reducing the amount of data transferred. - Eliminate the need to use an HTTPS tarball `SRC_URI` to reduce data transfer. - Allow SSH-based authentication (e.g. cert and agent-based) when using non-public repos, so additional HTTPS tokens may not be required. (Bitbake rev: 457288b2fda86fd00cdcaefac616129b0029e1f9) Signed-off-by: Stefan Koch <stefan-koch@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: Remove double importsMichael Estner2025-02-272-2/+1
| | | | | | | | | * Remove double imports mentioned by pylint (Bitbake rev: 741db6719efca5aa9ef2c15e60cdd624e4aa1a8d) Signed-off-by: Michael Estner <michaelestner@web.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Fix BB_FETCH_PREMIRRORONLY network disablingRichard Purdie2025-02-251-12/+15
| | | | | | | | | | | | When using BB_FETCH_PREMIRRORONLY we write to the datastore to disable the network. This change needs to be undo when handling later urls, so operate on a copy of the datastore to allow this. Reported by Julian Haller <julian.haller@philips.com> (Bitbake rev: 67a5ede8ae92ed7dcad29fd0dcfd62c6640b10b2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: clearcase: remove double DL_DIR from localfileStefan Herbrechtsmeier2025-02-111-2/+0
| | | | | | | (Bitbake rev: bcf090ed631bbd523a5341baebba0765f1a847f8) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: ssh: use common localpath handlingStefan Herbrechtsmeier2025-02-111-2/+1
| | | | | | | (Bitbake rev: 22ac6e84c70034a74f1729c7e0f31ca928fa1dea) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: remove unnecessary unquoteStefan Herbrechtsmeier2025-02-114-5/+5
| | | | | | | | | | The URI path is already unquoted. Remove the unnecessary unquote function calls for URI path values. (Bitbake rev: 3de12bbc28b5a4189b849720735cf3e268d3941d) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: local: use path variableStefan Herbrechtsmeier2025-02-112-6/+5
| | | | | | | | | | Use the path variable from the fetch data instead of decoding the path manually from the plain unexpanded url. (Bitbake rev: ad3a29fa6ea53741d4e1786de35f8e7fc4292e7a) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: remove unnecessary expand function callsStefan Herbrechtsmeier2025-02-117-10/+11
| | | | | | | | | | | | The fetch data class already expands the type, host, path, user, pswd and parm variables. The fetcher classes already expand the localfile variable. The getVar function expands the returned string per default. Remove unnecessary expand function calls to simplify the code. (Bitbake rev: 1b1eb037b861fbf20491ac17e519e9eaf232b858) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: remove duplicated code in url decode and encodeStefan Herbrechtsmeier2025-02-111-52/+14
| | | | | | | | | | Use the URI class to decode and encode an URL. Remove duplicate code and unify the behavior. (Bitbake rev: a5d569c94700f04b8193c6bccae5af619931b00f) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: do not decode user from file URIStefan Herbrechtsmeier2025-02-111-0/+3
| | | | | | | | | | A file URI can't contain a user. Do not treat the @ as reserved character for a file URI. (Bitbake rev: 11cf4062f48536547a352e24d6e963d91fdd1190) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb/fetch2/__init__.py: remove a DeprecationWarning in uri_replace()Bin Lan2025-01-141-1/+1
| | | | | | | | | | | | | There is the following warning when executing to bitbake linux-yocto: bitbake/lib/bb/fetch2/__init__.py:464: DeprecationWarning: 'count' is passed as positional argument This is because the 4th parameter of re.sub(pattern, repl, string, count=0, flags=0) is a keyword parameter. We use keyword arguments for parameters that are not positional. (Bitbake rev: 4bd62639f7023eec9f55ad7909fa9250538de936) Signed-off-by: Bin Lan <bin.lan.cn@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: add Go mod h1 checksum supportStefan Herbrechtsmeier2025-01-111-1/+1
| | | | | | | | | | | Add support for the Go mod h1 hash. The hash is based on the Go dirhash package. The package defines hashes over directory trees and is uses for Go mod files and zip archives. (Bitbake rev: deefb01592f717efba68e3997fefd04dc7611d88) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: read checksum from SRC_URI flag for npmStefan Herbrechtsmeier2025-01-081-1/+1
| | | | | | | (Bitbake rev: 8d3232152e545be958f8f3b069e7477c6f310583) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: do not prefix embedded checksumsStefan Herbrechtsmeier2025-01-082-7/+7
| | | | | | | | | | | | | The fetcher support entries with an embedded checksum like 'sha256sum' in the SRC_URI. It adds the parameter 'name' as prefix to the checksums if the parameter is set. This behavior is unexpected and leads to hacks in fetchers. Fallback to the checksum without the useless prefix and set the parameter 'name' in the gomod fetcher unconditional. (Bitbake rev: 7a86c5a20ea2586f1ae240613644e065e7b21683) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npmsw: remove old lockfile format supportStefan Herbrechtsmeier2025-01-081-61/+36
| | | | | | | | | | | | | Remove support for the old lockfile format. The old lockfile format is required by npm 6 / Node.js 14 which is out of maintenance [2]. [1] https://docs.npmjs.com/cli/v6/configuring-npm/package-lock-json [2] https://nodejs.org/en/about/previous-releases (Bitbake rev: 7824e19483d9b60a259d6e3a4c7068fade94f2bf) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: correctly construct the target URL in checkstatus()Ross Burton2024-12-201-2/+2
| | | | | | | | | | | | | | | | | | | | | ud.path has been unescaped (eg %20 is space) but as we're reconstructing a URL we should re-escape it. For example, unzip has a SRC_URI containing "UnZip%206.x%20%28latest%29/UnZip%206.0/unzip60.tar.gz" which then throws exceptions if the unescaped string " (latest)" is used. Also, this code uses the extracted ud.host and ud.path variables. These are unescaped but potentially stale as eg the cargo fetcher subclasses Wget() and reassigns ud.url on construction. Simplify the code by reconstructing a URL from ud.url directly instead of bouncing through intermediate variables that may be wrong or unescaped. (Bitbake rev: c9906bb289dcfd9ae41f10bd5399ccc17a4cc437) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: handle HTTP 308 Permanent RedirectRoss Burton2024-12-201-5/+37
| | | | | | | | | | | | | | | urllib2.HTTPRedirectHandler.redirect_request doesn't handle HTTP reponse code 308 (Permanent Redirect). This was fixed in c379bc5 but can't be worked around without copying the entire redirect_request() method. When we can depend on Python 3.13, FixedHTTPRedirectHandler can be removed. (Bitbake rev: 365829a2803b954ee6cb0364749551a91d806075) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: npmsw: ignore bundled dependencies in packagesStefan Herbrechtsmeier2024-12-201-0/+2
| | | | | | | | | | | | The npm package lock and shrinkwrap file list bundled dependencies which are supplied together with the parent dependency. The flag depends on the lock file version. Add the missing `inBundle` flag to packages list processing. (Bitbake rev: 042d7121ddc93c18306bf03d26ea774f66379a90) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix unpack of a single zstd filePascal Eberhard2024-11-291-1/+1
| | | | | | | | | | | | | | | | do_unpack generates a 'Bitbake Fetcher Error: UnpackError' when uncompressing a single 'file.zst', zstd command line returns the error 'unexpected end of file', and the input file 'file.zst' becomes empty. The command line is 'zstd --decompress --stdout file.zst > file.zst'. So the issue is that the input and output filenames are the same. Fix the output filename when using zstd for a single file. (Bitbake rev: 5881ca0e0d53937cd25a57ff682a3b7577b0b7f8) Signed-off-by: Pascal Eberhard <pascal.eberhard@se.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: Increase timeout to 100s from 30sRichard Purdie2024-11-211-2/+2
| | | | | | | | | | Testing shows the worst case CDN response time can be up to 100s. The wget fetcher is used for accessing sstate from the CDN so increase our timeouts there to match our worst case repsonse times. (Bitbake rev: 7aca591529e115bc277f93811d8c586630acc8c3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git: Clean broken symlinkRobert Yang2024-10-281-1/+1
| | | | | | | | | | | The file may be a symlink when mirror from local disk, and the symlink will be broken when the linked file is removed, use os.path.islink() to check and remove the broken symlink. (Bitbake rev: d0deb5fe6e7a84b07b49ef685b209729d2fdc780) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-07 10:59:47 +0000