| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds an API to bb.fetch2 to enable users to plug in an unpack
tracer that can trace each source file back to its corresponding
upstream source url, even when multiple upstream sources are combined
together in the same unpack directory. This may be required for software
composition analysis, license compliance, and detailed SBoM generation.
This patch provides only the needed hooks in bb.fetch2 code and a dummy
abstract class defining the API; users may load their own unpack tracer
class by setting the BB_UNPACK_TRACER_CLASS config parameter.
(Bitbake rev: 05051152cc42acc52bcf9af9a696f632fac4307f)
Signed-off-by: Alberto Pianon <alberto@pianon.eu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The bitbake logger overrode the definition of the debug() logging call
to include a debug level, but this causes problems with code that may
be using standard python logging, since the extra argument is
interpreted differently.
Instead, change the bitbake loggers debug() call to match the python
logger call and add a debug2() and debug3() API to replace calls that
were logging to a different debug level.
[RP: Small fix to ensure bb.debug calls bbdebug()]
(Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
removed unused imports which made the code harder to read, and slightly
but less efficient
(Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697)
Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Fix mercurial fetching after breakage from changes to the core fetcher.
Fix username and password usage and setting moddir needed by setup_revisions.
(Bitbake rev: 8962b27abc68427eae085624f26300108ae88aa1)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
There are much better ways to handle this and most editors shouldn't need this
in modern times, drop the noise from the files. Its not consitently applied
anyway.
(Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The fetch2/__init__.py checks whether "BB_FETCH_PREMIRRORONLY" == "1", but
fetch2/git.py and hg.py checks whether it is None, this makes it discontinuous,
and BB_FETCH_PREMIRRORONLY = "0" doens't work as expected in the later case,
so unify it to the previous one. (As BB_NO_NETWORK does).
And also use bb.utils.to_boolean() to make them consistent.
(Bitbake rev: 85a0d22835588e9ad8ec29d88a8115227e88420c)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently there is quite some variation between the fetchers in terms
of how they determine the subdirectory within DL_DIR and the base
fetch command to run. Some rely on variables being set externally
(e.g. from bitbake.conf in oe-core), some respect these external
variables but provide fallback defaults and some use only hardcoded
internal values. Try to unify the approach used across the various
fetchers.
(Bitbake rev: efd5e35af4b08501c67e8b30f30d9457f6fdf610)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove ud.mirrortarball in favor of ud.mirrortarballs. Each tarball will be
attempted, in order, and the first available will be used. This is needed for
git shallow mirror tarball support, as we want to be able to use either
a shallow or full mirror tarball.
(Bitbake rev: 02eebee6709e57b523862257f75929e64f16d6b0)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Cleanup some more usage of bb.data APIs in the fetchers.
(Bitbake rev: 9752fd1c10b8fcc819822fa6eabc2c1050fcc03b)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
For spelling's sake, rename Python routine "setup_revisons" to
"setup_revisions."
(Bitbake rev: 4df59b027c02ef39d72476251ccd3fd62fc20bf6)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
[YOCTO #10508]
(Bitbake rev: ddd3bc2d64d7240ecb6b6e4a1ae29b1faef6cc22)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
The old style bb.data.getVar/setVar API is obsolete. Most of bitbake
doesn't use it but there were some pieces that escaped conversion. This
patch fixes the remaining users mostly in the fetchers.
(Bitbake rev: ff7892fa808116acc1ac50effa023a4cb031a5fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Fix the methods in all fetchers so they don't change
the current working directory of the calling process, which
could lead to "changed cwd" warnings from bitbake.
(Bitbake rev: 6aa78bf3bd1f75728209e2d01faef31cb8887333)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
errno.ENOENT checked if deletion of the fullmirror fails.
Exception was thrown since module not imported
(Bitbake rev: d92ebfc34b69ad5df2d151e6b8299fbb5afa3e5f)
Signed-off-by: Logan Buchy <logan.buchy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We were maintaining state in the form of ud.repochanged to determine whether
we need to write the tarball in the case where the tarball already exists, but
this state is maintained only in memory. If we need to update the git repo,
set ud.repochanged to True, and then are interrupted or killed, the tarball
will then be out of date.
Rather than maintaining this state, simply remove the out of date tarball when
we update the git repo, and it will be re-created with updated content in
build_mirror_data.
[YOCTO #6366]
(Bitbake rev: eaaa81393f181432c8586b17ade623f42c9fed2e)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed:
* do_unpack error:
abort: repository DL_DIR/hg/vim.googlecode.com/hg/vim not found!
* The mirror tarball doesn't work
- Add the build_mirror_data to create the tarball
- Unpack the mirror tarball when needed
* The hg files will put in the dir like git: DL_DIR/hg, it was
DL_DIR/hg/path/to/src/uri/path in the past.
(Bitbake rev: 378647281ae883dd726f1e1b775dd35ef6a0e8d1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Like the Git fetcher, the Mercurial fetcher shouldn't expect recipes to
provide a checksum. As described [1], recipes using a mercurial
fetcher that don't provide a checksum will fail in a the repository has
previously been downloaded and archived.
Credit to Rafaël Carré for figuring out the bug.
[1] https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg41328.html
(Bitbake rev: 2df35a25b4968f64adfa673d5b73442c1a30829d)
Signed-off-by: Ash Charles <ashcharles@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed when bitbake vim -ccleanall:
File: '/path/to/bitbake/lib/bb/fetch2/__init__.py', lineno: 1462, function: clean
1458: def clean(self, urldata, d):
1459: """
1460: Clean any existing full or partial download
1461: """
*** 1462: bb.utils.remove(urldata.localpath)
1463:
1464: def try_premirror(self, urldata, d):
1465: """
1466: Should premirrors be used?
File: '/path/to/bitbake/lib/bb/utils.py', lineno: 633, function: remove
0629: subprocess.call(['rm', '-rf'] + glob.glob(path))
0630: return
0631: for name in glob.glob(path):
0632: try:
*** 0633: os.unlink(name)
0634: except OSError as exc:
0635: if exc.errno != errno.ENOENT:
0636: raise
Exception: OSError: [Errno 21] Is a directory: '/path/to/downloads/hg/vim.googlecode.com/hg/vim'
(Bitbake rev: 02763306662e15a4750395e5eab64ba98d1f9939)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use hg clone and hg pull to copy the source into the build
directory rather than taring up the cloned repository and
untarring in the destination.
This allows submodules to be cloned. While here, make the default
behaviour keep the hg scm data to match the behaviour of the git
fetcher.
(Bitbake rev: f002b1ca80cb542a4ed0c06c53c914cd5e076565)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
We should only add user/password options if they're specified as in the
fetch case. Patch from Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
(Bitbake rev: 303e6256947f4df4f283b75b7ccfdffa72864d67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
A fix for the former patch when checking out a repository with
username and password using HG
(Bitbake rev: 0e7b594ccbceb3149f38776cea204807031ef69f)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
specified in url
(Bitbake rev: 9a328262e047562a6edf7b2ec7b8a1949e287cbe)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Following up a previous patch for mercurial fetcher, I just fixed a
problem when calling update on a repository with subrepositories enabled.
Attached a patch that fixes this problem
>From caae519a2bd64bf7c729bb26aff344827def47fb Mon Sep 17 00:00:00 2001
From: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Date: Tue, 29 Apr 2014 15:29:47 +0200
Subject: [PATCH] bitbake: - fixed authentication issues in case of using sub
repositories
(Bitbake rev: dfa041c940caad12da50126a559afc8de089eeda)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
In 6a48474de9505a3700863f31839a7c53c5e18a8d the url parameter to a
number of functions was removed. However, not all calls to
latest_revision() were fixed...
(Bitbake rev: 7c94ca56b2fd85a989089f58b3dcce3172a778f2)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There is no good reason to keep passing around the url parameter when
its contained within urldata (ud). This is left around due to
legacy reasons, some functions take it, some don't and its time
to cleanup.
This is fetcher internal API, there are a tiny number of external users
of the internal API (buildhistory and distrodata) which can be fixed up
after this change.
(Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Trying to use a server with username and password authentication
within the URL of the SRC_URI variable doesn't appear to work.
This patch adds the missing parts to the hg fetcher to make this
work properly.
(Bitbake rev: dc3d6d73e44802c203b3f7247f6f212acc2f69bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Removed "-r REV" from hg clone invocation to fetch the entire repo,
rather than just the subset at the specified revision. This will ensure
that the specified tag exists for successive commands (e.g., the update
built on line 149.)
(Bitbake rev: f0a6261d3a8ede9ebdb6383e02cb2c2de1690640)
Signed-off-by: Jon Szymaniak <jon.szymaniak@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
| |
(Bitbake rev: f4a780a7f50fdd8f2bd75888dad790bcfe95b873)
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
The _build_revision method in Hg class gets called with the wrong number
of arguments. This tiny patch adds a 5th argument to the method
declaration to prevent python from throwing an exception.
(Bitbake rev: 623e9c7f7a9cf12b8c81c26cc608990682a601dd)
Signed-off-by: Julian Pidancet <julian.pidancet@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
The ordering constrains on the urldata_init functions are not straight
forward. To avoid further problems, create a helper function to setup
the source revisions which the init functions can all at the appropriate
point.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
| |
(Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* This patch fixes a cosmetic issue currently we get with master
WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733:
DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead. bb.mkdirhier("%s/%s" % (rootdir, destdir))
(Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
users usable error messages
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
and bug fix the code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
| |
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
functions for common operations where possible
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
| |
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
| |
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* SRC_URI format:
the SRC_URI are extended to allow multiple src rev:
name=<name1>,<name2>,...<name-n>
branch=<branch1>,<branch2>,...,<branch-n>
also SRCREV can be defined with
SRCREV_<name1> = xxxxx
SRCREV_<name2> = xxxxx
* FetchData extention
to support multiple src rev, several FetchData data are added:
- FetchData.names: list of name in SRC_URI, one name per srcrev. name is the index of revision and branch
- FetchData.revisions: dictionary of name->revision.
- FetchData.branches: dictionary of name->branch.
For example, linux-yocto recipes becomes:
SRC_URI = "git://git.pokylinux.org/linux-yocto-2.6.37;protocol=git;branch=${KBRANCH},meta;name=machine,meta"
FetchData.names = ['machine', 'meta']
FetchData.revisions = { 'machine':xxxxx, 'meta':xxxxxx }
FetchData.branches = { 'machine':${KBRANCH}, 'meta':'meta'}
* generic revision handling extension
the related revision handling code in fetch2.__init__.py are changed accordingly. the major change is add name parameter to indicate which src rev to handling. originally there is one src rev per FetchData, so FetchData parameter is enough. now since one FetchData has multiple src rev, it is necessary to use FetchData + name to specifiy src rev.
* git extension
git fetcher are also revised to take advantage of the multiple src rev in FetchData. especially the download() method are enhanced to fetch multiple src rev.
* other fetcher (svn, hg, ...) does not support multiple src rev. they just sync the API to add name, and then simply ignore the name. no actually functional change
Signed-off-by: Yu Ke <ke.yu@intel.com>
|
|
|
|
| |
Signed-off-by: Yu Ke <ke.yu@intel.com>
|
|
|
|
|
|
| |
no functional change
Signed-off-by: Yu Ke <ke.yu@intel.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Current fetcher has annoying "SRCREVINACTION" deadlock,
which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev():
get_srcrev()->setup_localpath()->srcrev_internal_helper()
->evaluate SRCREV->get_srcrev()
current fetcher resolve the deadlock by introducing a
"SRCREVINACTION" condition check. Althoguh it works, it is
indeed not clean.
This patch use antoehr idea to break the deadlock: break
the dependency among SRCREV and get_srcrev(), i.e. assign
a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet
this keyword, it will check and set the latest revision to
urldata.revision. get_srcrev later can use the urldata.revision
for value evaluation(SRCPV etc). In this case, SRCREV no longer
depends on get_srcrev, and there is not deadlock anymore.
Signed-off-by: Yu Ke <ke.yu@intel.com>
|
|
|
|
|
|
|
| |
move the hg specific urldata init from localpath to urldata_init
so that it can be called early
Signed-off-by: Yu Ke <ke.yu@intel.com>
|
|
|
|
|
|
|
|
| |
bb.fetch2 is copied from bb.fetch, and has many bb.fetch referrence.
Fix these referrence with bb.fetch2 referrence
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|