summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2/git.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch/git: Fix local clone url to make it work with repoRobert Yang2023-03-301-2/+6
| | | | | | | | | | | | | | | | | | The "git clone /path/to/git/objects_symlink" couldn't work after the following change: https://github.com/git/git/commit/6f054f9fb3a501c35b55c65e547a244f14c38d56 But repo command manages the git repo as symlinks, so check whether the objects is an symlink to fix the problem: * Nothing is changed if git/objects is not a symlink * Use "git clone file://" when git/objects is a symlink (Bitbake rev: a0d8108eba8d542707740d00c66c1c5f5b963f18) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add autorev warning when it is set too lateRichard Purdie2023-03-221-1/+1
| | | | | | | | | | | | | | | | | | | | Bitbake expects a consistent metadata environment when it parses. There are plenty of ways you can set a recipe to autorev at a point where parts of the fetcher have already been triggered leading to obsure bugs which I struggled to debug, let alone anyone not familar with the code. If anyone is running into the message from the commit, the issue is likely one of timing. Keep in mind that the anonymous python code in base.bbclass will expand variables like FILESPATH, WORKDIR and others which contain PV. The recipe needs to be set to AUTOREV before that anonymous python runs. In particular, that means you can't set SRCREV = "${AUTOREV}" in other anonymous python, it needs to happen earlier. (Bitbake rev: 4d9ec332d5bfc8b60b54f8ec2a17d34e35aa903a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Rename __BBSEENSRCREV -> __BBSRCREV_SEENRichard Purdie2023-03-221-1/+1
| | | | | | | | | For readability of following patches, rename this internal variable to allow for others in a similar format. (Bitbake rev: 450b4a00895ed1f9396a8ff859dc1cc0eccc838f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Replace mkdtemp with TemporaryDirectory and avoid ↵Paulo Neves2023-02-221-4/+1
| | | | | | | | | | | | | | exception masking Due to using mkdtemp instead of TemporaryDirectory we needed to manually cleanup the directory in a try finally block. With tempfile.TemporaryDirectory we can handle the cleanup with a "with" statement and not need to manually clean up oursevels. (Bitbake rev: 36a231d2a45e7c87172196538d18ded81a234774) Signed-off-by: Paulo Neves <paulo@myneves.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Removed unused variables in _contains_lfsPaulo Neves2023-02-221-5/+0
| | | | | | | | | | branchname was set but never used in the context of _contains_lfs method. (Bitbake rev: a64ec64c2faf1ac6ce995cdd4e311d8be6046155) Signed-off-by: Paulo Neves <paulo@myneves.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Clarify the meaning of namespaceMarek Vasut2022-12-211-1/+2
| | | | | | | | | | | | | | Namespace in this context means a branch, a tag, etc., clarify it in the description. Also, fix a typo "a any", replace with plain "any". This patch is based of feedback on new applied patch d32e5b0e ("fetch2/git: Prevent git fetcher from fetching gitlab repository metadata") (Bitbake rev: b4999425c812b25cb359d5163d11e3c1b030dc28) Signed-off-by: Marek Vasut <marex@denx.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Prevent git fetcher from fetching gitlab repository ↵Marek Vasut2022-12-091-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | metadata The bitbake git fetcher currently fetches 'refs/*:refs/*', i.e. every single object in the remote repository. This works poorly with gitlab and github, which use the remote git repository to track its metadata like merge requests, CI pipelines and such. Specifically, gitlab generates refs/merge-requests/*, refs/pipelines/* and refs/keep-around/* and they all contain massive amount of data that are useless for the bitbake build purposes. The amount of useless data can in fact be so massive (e.g. with FDO mesa.git repository) that some proxies may outright terminate the 'git fetch' connection, and make it appear as if bitbake got stuck on 'git fetch' with no output. To avoid fetching all these useless metadata, tweak the git fetcher such that it only fetches refs/heads/* and refs/tags/* . Avoid using negative refspecs as those are only available in new git versions. Per feedback on the ML, Gerrit may push commits outsides of branches or tags during CI runs, which currently works with the 'nobranch=1' fetcher parameter. To retain this functionality, keep fetching everything in case the 'nobranch=1' is present. This still avoids fetching massive amount of data in the common case, since 'nobranch=1' is rare. Update 'nobranch' documentation. Reviewed-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> (Bitbake rev: d32e5b0ec2ab85ffad7e56ac5b3160860b732556) Signed-off-by: Marek Vasut <marex@denx.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: don't set core.fsyncobjectfiles=0Ross Burton2022-10-291-1/+1
| | | | | | | | | | | | | | | | | This git configuration variable is deprecated in 2.36.0 onwards, so git warns in the logs for every git call. Luckily the default value has always been false[1], so we can just remove this. [ YOCTO #14939 ] [1] https://github.com/git/git/commit/aafe9fbaf4f1d1f27a6f6e3eb3e246fff81240ef (Bitbake rev: 8ad310633e0c5d5593631c1196cbdde30147efce) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Remove unneeded conditionalPaulo Neves2022-09-211-7/+6
| | | | | | | | | | The condition will always evaluate to true and thus is redundant. (Bitbake rev: be1ee681e8a566564549068dcf90c95c36544815) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: show SRCREV and git repo in error message about fixed ↵Martin Jansa2022-07-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | SRCREV * in recipe with 17 git repos in SRC_URI I've accidentally pasted one SRCREV to be one character shorter and because fetcher uses: if not ud.revisions[name] or len(ud.revisions[name]) != 40 or (False in [c in "abcdef0123456789" for c in ud.revisions[name]]): to decide which SRCREV values are fixed SRCREVs this one was considered as tag or branch name, because it was only 39 chars long The original error message wasn't very helpful as it doesn't show which repo or which SRCREV was considered missing: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) with SRCPV included in PV as error recomments it's a bit better: bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve '0a92994d729ff76a58f692d3028ca1b64b145d9' in upstream git repository in git ls-remote output for github.com/Maratyszcza/FP16 The variable dependency chain for the failure is: SRCPV -> PV -> WORKDIR -> T with this change the first error will read: do_fetch: Bitbake Fetcher Error: FetchError("Recipe uses a floating tag/branch '0a92994d729ff76a58f692d3028ca1b64b145d9' for repo 'github.com/Maratyszcza/FP16' without a fixed SRCREV yet doesn't call bb.fetch2.get_srcrev() (use SRCPV in PV for OE).", None) (Bitbake rev: 04dc17bef9b762cef9eecdf91c9f37738d8ae44d) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Honour BB_FETCH_PREMIRRORONLY optionPavel Zhukov2022-06-081-4/+9
| | | | | | | | | | | | | | | | | | This should fix [Yocto 13353] and related to [Yocto 13233] as well. Previously if git repo mirror has been updated in between of two builds fetcher of the second build didn't try updated mirror but switched to git clone from upstream instead. This is problem for offline builds. Fix this to raise MirrorException if BB_FETCH_PREMIRRORONLY has been specified by the mirror doesn't contain SRC_REV. [YOCTO #13353] [YOCTO #13233] (Bitbake rev: b47ecab3e3aad5c5c376ec023aa82a51aa0f3b86) Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git : Use cat as pagerRichard Purdie2022-05-081-1/+1
| | | | | | | | | We don't have less in HOSTTOOLS in OE and this can confuse git. Force the pager to cat to be consistent and minimal everywhere. (Bitbake rev: d3d406e8552fdd865dc58b419a84411736475ad2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Simplify the validation of SHA-1 revisionsPeter Kjellerstedt2022-04-191-6/+8
| | | | | | | | | | Also correct two comments, and move slash_re from _revision_key() to the module top level (together with the new sha1_re). (Bitbake rev: 98cad8636e9c82bc40a033bb83633ec994758eb0) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: canonicalize ids in generated tarballsOlaf Mandel2022-04-141-1/+1
| | | | | | | | | | | | | | Change the owner information in the mirror tarballs generated using BB_GENERATE_MIRROR_TARBALLS="1". This is an extension of commit 0178ab83, which used the original pokybuild:user information, but failed to clean up the numerical user and group ids. Now set the more canonical values of oe:oe and 0:0. (Bitbake rev: 37437115d3fb1a9f5d8ed7356a0fc01a408e4f8c) Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> CC: Marek Vasut <marex@denx.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: stop generated tarballs from leaking infoOlaf Mandel2022-03-261-1/+4
| | | | | | | | | | | | | | | | | | | | When using BB_GENERATE_MIRROR_TARBALLS="1" to generate mirror tarballs of git repositories, they leaked local information: username, group and time of the last fetch. Remove all these by setting fixed information: * uname = pokybuild * gname = users * mtime = committer time of newest commit in repo The username and group value were taken from the archives available on the downloads.yoctoproject.org mirror. The modification time is chosen so it still retains some relationship to the contents of the archive. (Bitbake rev: 0178ab83e6312e97e528aa8c5e12105f5165d896) Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Remove duplicate "the"Zygmunt Krynicki2022-02-201-1/+1
| | | | | | | (Bitbake rev: 02e29e90eac56e46635e9ae44e5c5da684b70ed2) Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@huawei.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/cooker: Fix source revision handling with floating upstreamsRichard Purdie2022-02-171-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Where a git url uses a tag instead of a full source revision, breakage can currently occur in builds. Issues include: * the revision being looked up in multiple tasks (fetch and unpack) * the risk a different revision may be obtained in those tasks * that some tasks may not be allowed to access the network * that a revision may not be consistent throughout a given build * rerunning a specific task may given inconsistent results To fix this, stop the workers from cleaning out the source revision store. This should only be done in the cooker itself (based on current policy). Also, where the code "sees" an upstream access, mark the recipe as not to be cached. The reparse re-triggers the upstream lookup by the server. Add a test to ensure that if get_srcrev isn't called, the user is told they're using a configuration that is known to break. (Bitbake rev: 4b5eed1626709ef3dc06b32fd55d40a2a6edd179) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Show warning for invalid github urlsRichard Purdie2021-11-031-0/+1
| | | | | | | | On master, tell the users they need to update their urls for github. (Bitbake rev: 42526a402357e04794f4cb6f21ac18f562220a9b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Handle github dropping git:// supportRichard Purdie2021-11-031-0/+4
| | | | | | | | | github is dropping support for git protocol in Git urls. Add code to remap this to https in a way that could be used in older bitbake versions. (Bitbake rev: f05e19135b3ddee509c0cb427b3b9376bb4738d1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Add a warning asking users to set a branch in git urlsRichard Purdie2021-11-031-1/+4
| | | | | | | | | | | | There is much uncertainty around what tools and hosting providers will do about default git branch naming in the future. To help ensure we can handle the various scenarios, we will make branch names required in SRC_URI. To start that process, show users a warning if it isn't set. This may also allow us to change the default at some point in the future. (Bitbake rev: 86a9c26828479be55865bcce72bcc7e12b93caa7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Allow git fetcher to support subdir paramRichard Purdie2021-11-031-8/+19
| | | | | | | | | | | | | | | | | | The git fetcher is odd in that it supports destsuffix as a parameter but not the default documented subdir parameter. destsuffix is more limited as it can't take absolute paths. Rework the code to correctly support subdir. Also cleanup to use the None default .get() values and be a bit more pythonic and use subpath as the variable name for subpath for code clarity. We could consider dropping destsuffix as a parameter as some future point. Also fix the tests not to pass in a subdir parameter which was never used but now causes errors. (Bitbake rev: 66953f06fe822e4001efabd9fc1c985ea2b03f96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Use os.rename instead of mvRichard Purdie2021-09-211-4/+4
| | | | | | | | | os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Avoid races over mirror tarball creationRichard Purdie2021-09-171-2/+19
| | | | | | | | | | | | | | There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: run gc in foreground to avoid race with tarAdam Romanek2021-07-101-1/+1
| | | | | | | | | | | | | | | | | | | | It looks like git gc can interrupts the package creation when BB_GENERATE_MIRRORS_TARBALL is in use. Log excerpts: tar -czf TOPDIR/../downloads/git2_bitbucket.name-hidden.git.tar.gz . failed with exit code 1, output: tar: ./objects/pack/pack-89a1d76f6c08f53172ef1d02ff851d90564362c4.pack: file changed as we read it tar: ./objects/pack/pack-b4a48ada355d333630fdf6b4f67205b7c264dc2c.idx: File removed before we read it Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. (Bitbake rev: a8d8cb847063862d1a7998963dd8b767ff73d877) Signed-off-by: Adam Romanek <romanek.adam@gmail.com> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: Fix crash when using git LFS and submodulesNiels Avonds2021-04-201-1/+1
| | | | | | | | | | | | | | | | Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] (Bitbake rev: 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Niels Avonds <niels@codebits.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for disabling shared clones on unpackHenning Schild2021-04-181-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | By default the unpacker will create a "shared" clone when cloning from the DL_DIR to the WORKDIR. This patch introduces an option to control that behaviour. Imagine some recipe steps are executed in a namespace that is different from the one your downloader and unpacker ran in. (chroot) Because a "shared" clone has an absolute reference to its "alternate" you now have to make that "alternate" visible in that new namespace (chroot) at the exact place. With this patch you can unpack "noshared" and get a stand-alone copy. This copy will also work if the "alternate" is not visible or existant. The switch is a global bitbake switch and will affect all git urls. Build systems that need "noshared" most likely need it for everything they do with git. (Bitbake rev: 6ae6f1865d5e666ebc670f70b7401a7b41648102) Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: download LFS content too during do_fetchMatt Hoosier2021-01-231-3/+41
| | | | | | | | | | | | | | Insert an explicit pass to fetch all blobs needed by Git LFS, during the fetch() function. This avoids the default behavior of Git LFS to wait until 'git checkout' to begin downloading the blobs pointed to by LFS records. Network access is not allowed at that point in the recipe's lifecycle. [YOCTO #14191] (Bitbake rev: 0efac66043662e7a2027192f50e92e982db2ba1c) Signed-off-by: Matt Hoosier <matt.hoosier@garmin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Fix usehead for non-default namesJoey Degges2021-01-081-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The usehead url parameter for git repositories causes bitbake to use whatever commit the repository HEAD is pointing to if the repository happens to have the name 'default'. This is the default name so in many cases it works just fine, but if a different name is specified with the url parameter 'name=newName' then it will fail to parse the recipe with an error along the lines of: ERROR: ExpansionError during parsing /path/to/my/recipe.bb Traceback (most recent call last): File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 235, in Git.urldata_init: > ud.setup_revisions(d) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1302, in FetchData.setup_revisions: for name in self.names: > self.revisions[name] = srcrev_internal_helper(self, d, name) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1167, in srcrev_internal_helper(name='newName'): if srcrev == "AUTOINC": > srcrev = ud.method.latest_revision(ud, d, name) File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1562, in Git.latest_revision(name='newName'): except KeyError: > revs[key] = rev = self._latest_revision(ud, d, name) return rev File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 650, in Git._latest_revision(name='newName'): raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \ > (ud.unresolvedrev[name], ud.host+ud.path)) bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /path/to/local/git/repo Let's fix this by setting the unresolved rev of _all_ repository names to 'HEAD' when the usehead url parameter is specified. Update the currently failing test, test_local_gitfetch_usehead_withname, to now expect success. This change preserves existing behavior that allows usehead to be overridden by a valid looking revision if one happens to be specified instead of AUTOREV. (Bitbake rev: 01e901c44ab0f496606b1d45c8953dc54970204c) Signed-off-by: Joey Degges <jdegges@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Document that we won't support passwords in git urlsRichard Purdie2020-11-031-0/+5
| | | | | | | | This keeps coming up, document why we don't do this and the alternatives. (Bitbake rev: aded964eed4ce5a725ed1ab477efabc86b1aa481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch/git: use shlex.quote() to support spaces in SRC_URI urlCharlie Davies2020-10-171-6/+7
| | | | | | | | | | | This commit replaces the instances where escaped double quotes are used to support SRC_URI url containing spaces with the more pythonic shlex.quote(). (Bitbake rev: 4f9ba9c794de55bea0343267467bddea99844374) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch/git: add support for SRC_URI containing spaces in urlCharlie Davies2020-09-231-6/+6
| | | | | | | | | | | | | | | | | | | Microsoft's TFS VCS system allows for spaces in a git repository url. An example of a valid url is: ssh://tfs-my-company.org:22/tfs/My Projects/FooBar This commit adds support for such urls by implementing two changes. Firstly, when bitbake makes a git command line call the url is surrounded by quotes so that the url, regardless of spaces, is treated as one argument. Secondly, additional parsing of various filepath variables, which are based off of the url, are now completed with any spaces in the url replaced with underscores. (Bitbake rev: eb38b6f0935763f7ba19e5618f376fcae1dac41a) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Change git fetcher not to destroy old referencesRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 820ab886e79eea516560c0c008e4cf059c6e11a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git.py: Use the correct branch to check if the repository has LFS ↵Mauro Queirós2020-06-041-2/+9
| | | | | | | | | | | | | objects. Function "contains_lfs" was only looking at the master branch when searching for LFS content. LFS may be configured in specific branches only, so we need to use the correct branch. (Bitbake rev: 4fa67c2af830035a1ddedc14592ee25a15ebff22) Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git.py: LFS bitbake note should not be printed if need_lfs is not set.Mauro Queirós2020-06-041-1/+1
| | | | | | | | | | | The message "Repository %s has LFS content but it is not being fetched" was being printed, even when Git-LFS was available and "lfs=1" was set. In those situations, we want to fetch LFS content, so that message would not make sense. (Bitbake rev: 45028dfda5a29a34ab408cb3f11d72ae17963340) Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git.py: skip smudging if lfs=0 is setMauro Queirós2020-06-041-0/+3
| | | | | | | | | | | Git-LFS objects were being fetched even when lfs=0 was not set. This patch disables LFS smudging when lfs=0. That way, only the LFS pointers are downloaded during checkout. (Bitbake rev: 646d86df7de774255246a3d7051c308e43eb257d) Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: _revision_key: collapse adjacent slashesChris Laplante2020-01-191-1/+3
| | | | | | | | | | | | | >From a SRCREV caching point of view, there is no reason to treat the following upstreams as different: SRC_URI = "git://github.com/file/file.git" SRC_URI = "git://github.com//file/file.git" (Bitbake rev: 425e21c14955dd38868c6e97637df3bbe0f89fac) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: amend code to use proper singleton comparisons where possibleFrazer Clews2020-01-191-1/+1
| | | | | | | | | | | amend the code to handle singleton comparisons properly so it only checks if they only refer to the same object or not, and not bother comparing the values. (Bitbake rev: b809a6812aa15a8a9af97bc382cc4b19571e6bfc) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: fetch shallow revs when neededChristopher Larson2019-10-281-7/+16
| | | | | | | | | | | | | | When bitbake determines if a git clone needs updating, it only checks for the needed srcrevs, not the revs listed in BB_GIT_SHALLOW_REVS, which will fail if using shallow and the needed rev was added to the upstream git repo after a previous fetch. Ensure that we also check for shallow revs. [YOCTO #13586] (Bitbake rev: 93b65e3c6ca64f644946953980595c44fbbcc748) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: refactor check for git-lfs commandRoss Burton2019-09-271-8/+9
| | | | | | | | | | Refactor the git-lfs checking: this means both clearer code in the download() function and allows unit testing to monkeypatch the functionality. (Bitbake rev: 33cf9172ded50a869f7201ba463ab9ecc69b8252) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: add git-lfs toggle optionRoss Burton2019-09-191-7/+11
| | | | | | | | | | | | | | | Add a new 'lfs' option to the git fetcher so that we can optionally not fetch git-lfs content, for repositories that contain LFS data that we don't actually need for building. By default lfs is set to 1, so if the repository has LFS content then git-lfs is required. Setting lfs to 0 will mean that git-lfs won't be required to fetch, and some files will be missing. (Bitbake rev: be0b78ccfc5ede98041bc0545a15092494b12b26) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Strip old editor directives from file headersRichard Purdie2019-05-041-2/+0
| | | | | | | | | | There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. (Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie2019-05-041-12/+0
| | | | | | | | | | With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie2019-05-041-1/+3
| | | | | | | | | | | | | | | | | This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: fetch2/git: git-lfs checkNaveen Saini2019-04-291-0/+25
| | | | | | | | | | | | Build will fail if repository has lfs contents in absense of git-lfs tool on host. Build will pass if repository may or may not contains lfs content if host has git-lfs installed. [YOCTO #13198] (Bitbake rev: 3f2c2eb2f59707828bdcdd6414db837da8dc3b0e) Signed-off-by: Naveen Saini <naveen.kumar.saini@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Unify BB_FETCH_PREMIRRORONLYRobert Yang2019-03-261-1/+1
| | | | | | | | | | | | | | The fetch2/__init__.py checks whether "BB_FETCH_PREMIRRORONLY" == "1", but fetch2/git.py and hg.py checks whether it is None, this makes it discontinuous, and BB_FETCH_PREMIRRORONLY = "0" doens't work as expected in the later case, so unify it to the previous one. (As BB_NO_NETWORK does). And also use bb.utils.to_boolean() to make them consistent. (Bitbake rev: 85a0d22835588e9ad8ec29d88a8115227e88420c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: Fix clean to remove clonedirRobert Yang2019-03-241-3/+11
| | | | | | | | | | | | | | | | | | | | | | The localpath is a symlink to clonedir when it is cloned from a mirror, for example: $ bitbake systemtap-native -cfetch $ ls downloads/git2 sourceware.org.git.systemtap.git -> /path/to/downloads/git2/mirror.path.git.sourceware.org.git.systemtap.git mirror.path.git.sourceware.org.git.systemtap.git There are both sourceware.org.git.systemtap.git and mirror.path.git.sourceware.org.git.systemtap.git in DL_DIR/git2, the symlink sourceware.org.git.systemtap.git is created by try_mirror_url(), but do_cleanall" only removed the symlink, didn't remove the real dir mirror.path.git.sourceware.org.git.systemtap.git, this may cause confusions, for example, I assumed that do_cleanall removed everything, but it didn't, and it would the re-used next time when do_fetch. This patch fixes the problem. (Bitbake rev: 452e2200ad2c29dec3753f5f7a8cbc9183ec7dd8) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsAndré Draszik2019-02-121-1/+1
| | | | | | | | | | | | | | | | | getVar() has been defaulting to expanding by default for a long time (2016), thus remove the True option from getVar() calls with a regex search and replace. Search & replace made using the following command: sed -e 's|\(d\.getVar \?\)( \?\([^,()]*\), \?True)|\1(\2)|g' \ -i $(git grep -E 'getVar ?\( ?([^,()]*), ?True\)' \ | cut -d':' -f1 \ | sort -u) (Bitbake rev: 3bba0dbd524cf72176a765957adff544ae5c255a) Signed-off-by: André Draszik <andre.draszik@jci.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Fix Deprecated warnings from regexsRichard Purdie2019-01-161-2/+2
| | | | | | | | | Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. (Bitbake rev: c1fcc46e2498ddd41425d8756754f814d682aba3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: fix AttributeError in shallow extraction logicChristopher Larson2018-11-221-5/+8
| | | | | | | | | | | | | | | This code checks to see if shallow is either disabled or the tarball is missing, but the else block tries to print the tarball filename, and this attribute doesn't exist at all when shallow is disabled. Handle the two cases separately to give sane errors for both cases without the exception: Exception: AttributeError: 'FetchData' object has no attribute 'fullshallow' (Bitbake rev: bdbb558342ebb4e64384c9838d2485d9299d91a6) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: provide information about missing sourcesUrs Fässler2018-10-181-8/+21
| | | | | | | | | | | Provide more information in the case the sources are not found in the unpack step. (Bitbake rev: 27a2214bf6f2e7c61bfc422a20959a55f7e0d25d) Signed-off-by: Urs Fässler <urs.fassler@bbv.ch> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-06-15 05:16:00 +0000