summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2: Change git fetcher not to destroy old referencessumoRichard Purdie2020-09-081-1/+1
| | | | | | | | | | | | | | | | It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 1df4fb1c9bf2fde0b274374bff1235c4b1f7ac79) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Resolve fetch error in bitbake-selftestArmin Kuster2019-11-071-2/+2
| | | | | | | | | | | | | | | | | | | | | FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 50d2166054338b7a63c4dc7116cd212a9ecfbfaf) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: COW: Fix StopIteration warningRichard Purdie2019-03-131-1/+1
| | | | | | | | | | | | | | | | Fix the warning: WARNING: lib/bb/data_smart.py:235: DeprecationWarning: generator 'COWDictMeta.iter' raised StopIteration for k, v in self.variables.iteritems(): by using return from the generator, not raising StopIteration. (Bitbake rev: c0af6c81f8d5487ea2cef54a78fd1cb1d0dc6520) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 407d6e07b09123c12c382b4a92107f002c314b05) Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: add optional callback to edit_bblayers_conf()Paul Eggleton2019-02-251-1/+14
| | | | | | | | | | | | Add a callback that lets you modify or remove items in addition to the current scheme where you can only add or remove. This enables you to for example replace a layer with a temporary copy (which is what we will use this for first in OE's oe-selftest). (Bitbake rev: 4f6ba26e8335f975038d90b9e1c1767160bd5272) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Rewrite multiple connection handlingRichard Purdie2019-02-251-22/+34
| | | | | | | | | | | | | | | | | | | | | | | | If the bitbake server recieved multiple connections, it currently closes ones it can't handle (while its dealing with another). This is rather antisocial behaviour which causes clients to quickly run through their retries and abort. Instead, queue any other connections until the current one is closed. This way the client can decide when it wants to stop waiting for the server. If the client is gone by the time we handle it, we handle that gracefully. This also fixes a number of bugs in the connection handling where connections which did drop early were badly handled causing tracebacks in the logs. Also, handle queue incomming connections in a loop to ensure that the main client handling doesn't starve that piece of the system. This code was stress tested by running 50 connection attempts in parallel at once, ensuring the code correctly handled them. (Bitbake rev: f675293d86504db66442532258d99b26333742bf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Handle EWOULDBLOCK in socket connectRichard Purdie2019-02-251-1/+8
| | | | | | | | | | | | | Now that we set a timeout for the socket, it can return EWOULDBLOCK if a signal or other event happens to wake up even if we don't timeout. If this happens, retry the connection, else we simply see it quickly loop through the retries and abort the connection in a very short interval. (Bitbake rev: f770d6a332812031682dc6bef1a2a84da52a4c32) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process.py: Set socket timeout to 10 secondsRichard Purdie2019-02-251-1/+1
| | | | | | | | | The current value of 2 seconds has shown to be short in wider testing. (Bitbake rev: c4a940991f261959eb08273d2250d3866b868938) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure socket has a timeout setRichard Purdie2019-02-251-0/+2
| | | | | | | | | | | We're seeing hangs in oe-selftest where server startup and shutdown are racing. The assumption was a connect would timeout however no timeout is set which can leave processes hanging. Set a short timeout for the connection to avoid this. (Bitbake rev: e53c1009356cc49c57d3b9af1e3dda6927acd78d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add some timing debug messages to the server startupRichard Purdie2019-02-251-0/+19
| | | | | | | | | We're seeing slow startup in bitbake, add some timeing debug messages so the logs are more useful for debugging when its slow. (Bitbake rev: 39548791c84982c44c872a579e5b42d2720af98f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Increase server startup time delayRichard Purdie2019-02-251-1/+4
| | | | | | | | | | | | | | | | | On loaded production systems we've seen bitbake server take over 40s to start up. Increase the timeout to 90s which tries to avoid failures in selftests. The delays come from setting up the inotify watches (31s) so can't really be avoided. After 5s delay we now warn the user we're waiting for 90s so the interactive exeperience shouldn't be much changed and its very unlikely the user would see that anyway. (Bitbake rev: 64055fdb0d8485b40ba710a762e9a0af5c4540b4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Show last 60 lines of the log if the server didn't ↵Richard Purdie2019-02-251-0/+6
| | | | | | | | | | | | start We're seeing issues where the server doesn't start with no logs as to why. Allow the server to print the last 60 log lines just in case this shows us something useful about what is failing. (Bitbake rev: da54a3fef9cee308dfa87eea9b9638796d734abd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Show the last 60 log lines, not the last 10Richard Purdie2019-02-251-2/+2
| | | | | | | | | 10 log lines may not capture any full traceback, increase the number of lines to 60 which covers most tracebacks. (Bitbake rev: 340a7d91b5030e4d62680f427286fc419509879b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: print a message when no logfileRobert Yang2019-02-251-0/+4
| | | | | | | | | | | [YOCTO #12898] There might be no bitbake-cookerdaemon.log, print a message for debugging. (Bitbake rev: cc32e703a8e68107353702edb230296503f5dad0) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Make lockfile handling clearerRichard Purdie2019-02-251-4/+6
| | | | | | | | | This simplifies the code and makes it easier to read but has the same functionality. (Bitbake rev: 21dcec291d58ab7ac5d2c07186c68e01d85e6f65) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: don't use shell=True when listing ar filesRoss Burton2019-02-251-1/+1
| | | | | | | (Bitbake rev: d150e50355444b0a67cad4d6a100d9bf6fadb2ae) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Don't use print() directly, use loggerRichard Purdie2018-12-061-1/+1
| | | | | | | | | | | | | | | | | | | Avoid failures like: 2018-12-04 03:30:56,033 - oe-selftest - INFO - ====================================================================== 2018-12-04 03:30:56,033 - oe-selftest - INFO - FAIL: pkgdata.OePkgdataUtilTests.test_find_path (subunit.RemotedTestCase) 2018-12-04 03:30:56,033 - oe-selftest - INFO - ---------------------------------------------------------------------- 2018-12-04 03:30:56,033 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): [...] AssertionError: 'Previous bitbake instance shutting down?,[77 chars]xist' != 'ERROR: Unable to find any package produci[14 chars]xist' - Previous bitbake instance shutting down?, waiting to retry... ERROR: Unable to find any package producing path /not/exist We need to use the logger so output is correctly handled in such cases. (Bitbake rev: 34a2ef91e6172c9d63819f4d8005a382956d741a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Flush key output to logsRichard Purdie2018-12-061-0/+4
| | | | | | | | | | Small tweak to ensure these items are printed into the log since there is other logging code which looks for the header and this makes it clearer the server did start but is slow somewhere in startup. (Bitbake rev: e698ff1817e3536211f40af161c563d15e2ef3b5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Ensure disk monitor is started when no setscene tasks are runRichard Purdie2018-12-061-5/+6
| | | | | | | | | | | Currently if there are no setscene tasks, the disk monitor isn't started. Move the startup code to somewhere to ensure it always is started. This issue would partially explain occasional selftest failures. (Bitbake rev: 9f72f5af7eb04c87be0e11a9c519a3941789bf7e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix ConnectionRefusedError tracebacksRichard Purdie2018-12-061-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Improve connetion refused error handling: NOTE: Retrying server connection... NOTE: Reconnecting to bitbake server... NOTE: Retrying server connection... (Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 471, in connectProcessServer sock.connect(os.path.basename(sockname)) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 502, in connectProcessServer os.close(i) TypeError: an integer is required (got type NoneType) ) WARNING: /home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> logger.info("Retrying server connection... (%s)" % traceback.format_exc()) (Bitbake rev: 0d0d8558d753ab3723325246c7d6c11e93cd18b8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/server: Avoid UnboundLocalError tracebackRichard Purdie2018-12-061-3/+3
| | | | | | | | | | | | | Traceback (most recent call last): File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 490, in connectProcessServer if command_chan_recv: UnboundLocalError: local variable 'command_chan_recv' referenced before assignment (Bitbake rev: bdffac83bcbd12668d262867f3a6e329327ff1e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix unclosed socket warnings upon server connection ↵Richard Purdie2018-12-061-8/+7
| | | | | | | | | | | | | | | | | | | | | | | refused Extend the server error handling to avoid: Reconnecting to bitbake server... NOTE: Retrying server connection... (Traceback (most recent call last): File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake server_connection = bb.server.process.connectProcessServer(sockname, featureset) File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 457, in connectProcessServer sock.connect(os.path.basename(sockname)) ConnectionRefusedError: [Errno 111] Connection refused ) WARNING: /home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0> logger.info("Retrying server connection... (%s)" % traceback.format_exc()) NOTE: Starting bitbake server... (Bitbake rev: 115919f0c11e22df783348422e06ab95f5c943e1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetcher: Fixed remote removal not throwing exception.Paulo Neves2018-10-101-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this fix it is assumed that the removal of the remote can only fail because there is not remote to remove. This is a false assumption. Example error which would be ignored: git -c core.fsyncobjectfiles=0 remote rm origin failed with exit code 1, output: Note: A branch outside the refs/remotes/ hierarchy was not removed; to delete it, use: git branch -d master error: could not lock config file config error: Could not remove config section 'remote.origin' Due to the masking of this error a stranger error will be presented to the user, because this time we do not mask the exception: git -c core.fsyncobjectfiles=0 remote add --mirror=fetch origin https://github.com/ptsneves/tl-wn722.git failed with exit code 128, output: fatal: remote origin already exists. The most likely reason that the remote cannot be removed nor modified is that the DL_DIR/git2 does not have permissions compatible with the user running bitbake. This commit fixes: https://bugzilla.yoctoproject.org/show_bug.cgi?id=12728 (Bitbake rev: 67189588a68b9bcb39421ef12103507b4c8820c3) Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Fix environment handling for UI module importsRichard Purdie2018-10-101-3/+4
| | | | | | | | | | | | | | | The environment was being cleared before the UI imports occurred which caused problems for graphical UIs like taskexp. The full environment was intended to be available to UI clients and it was only meant to be cleared for the server/cooker, so tweak the code order so this is the case. This fixes problems reported for taskexp. [YOCTO #12670] (Bitbake rev: 66ac33b6888edb7077b6e94a4d3d1e1d1d53c0c5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: checksum: sanity check path when recursively checksummingRoss Burton2018-08-291-0/+2
| | | | | | | | | | In case something goes tragically wrong, catch a request to checksum / and refuse. (Bitbake rev: 7444419b7fda34e14d653ba8470f5dfabb5da4f3) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils/md5_file: don't iterate line-by-lineRoss Burton2018-08-291-4/+9
| | | | | | | | | | | | | | | Opening a file in binary mode and iterating it seems like the simple solution but will still break on newlines, which for binary files isn't really useful as the size of the chunks could be huge or tiny. Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't want to fill up memory: use mmap() to open the file and read the file in 8k blocks. (Bitbake rev: 41e6161c8ce8cc90ebc93d72852673ae60fac923) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Update gnome.org urls after upstream changesRichard Purdie2018-08-291-2/+2
| | | | | | (Bitbake rev: 822dd77333896538393c657dd220d8f66d8a46a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix import error for Python 3.6.5Tzu Hsiang Lin2018-07-022-3/+1
| | | | | | | | | | | | When running bitbake command with Python 3.6.5 always result in import error causing by the change of distutils module. This patch replaces the method to search executable in PATH by "/usr/bin/env <command>". (Bitbake rev: eb3bdcc9e7cb273de4114367887d1d7df9f036b7) Signed-off-by: Tzu Hsiang Lin <t9360341@ntut.org.tw> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Update version to stable release branch version 1.38Richard Purdie2018-06-061-1/+1
| | | | | | (Bitbake rev: 6a9d86998fdd42b13690f024af33b6bf36b9d028) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils: prevent movefile from changing dest dir permissionsMattias Hansson2018-04-231-2/+2
| | | | | | | | | | | | | | | Prevent movefile from falsely setting the source file's owner and permissions on the destination directory instead of the destination file when moving between devices. This bug caused the last file moved into a directory to dictate the directory's owner and permissions. (Bitbake rev: 82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40) Signed-off-by: Mattias Hansson <mattias.hansson@axis.com> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Issue warning if layer doesn't set LAYERSERIES_COMPAT_xRichard Purdie2018-04-061-0/+2
| | | | | | | | | | | We'd like layers to set this variable so that we know which layers are compatible with which others, even if the branch is a generic un-updated "master" branch. Start printing a warning to highlight this issue. (Bitbake rev: cca81e33b58c390dcf5cc3a31555a43b79177166) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/shell.py: dropPaul Eggleton2018-03-281-820/+0
| | | | | | | | | | | | | | | | This was removed in the bitbake repository back in 2010 (bitbake revision 24857e2ceb405916b0b0b3e75c6c2375a909b9ba) - it's not clear to me what happened, but for some reason the file remained in the poky repository. However it does not appear that it has been used since then and the reasons for removing even more valid now than they were then - the code has moved on even further, we have memory resident bitbake server, and if we were to re-implement this we would do it in a different way. Drop the file and bring us back in sync with the bitbake repo. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/git: log exception if ls-remote failsRoss Burton2018-03-281-1/+2
| | | | | | | (Bitbake rev: 8212446de11c0e370c55f88cde86334b760cd939) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: state which upstream failedRoss Burton2018-03-281-2/+2
| | | | | | | (Bitbake rev: 21098de09ee2f7a9f0b3f895bf2ffbdeb8c9ded5) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils.py: Add option for explode_dep_versions2 to return unsortedAmanda Brindle2018-03-151-2/+3
| | | | | | | | | | | | Before, explode_dep_versions2 would sort the OrderedDict before returning. This function will still sort the OrderedDict by default, but will now have the option to return the OrderedDict unsorted. This option will allow us to check if the order of the package list has changed. (Bitbake rev: 39d6a30a28f66c599e18beddbd847f40dcff623c) Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Fix environment double key expansion issueRichard Purdie2018-03-041-3/+2
| | | | | | | | | | | The base configuration needs key expansion and anon python execution, the parsed configurations do not. Fix this consistently, its been broken and causing double key expansion for a while, only relised when we started double anonymous python exeution too. (Bitbake rev: 6138897de5ac6becf3bff56ce7a78f3ec208fcdf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Run registered anonymous python before displaying environmentRichard Purdie2018-03-041-1/+3
| | | | | | | | | | | The output of bitbake -e can differ from what actually is used due to anonymous python making changes to the data store. Execute any anonymous python added in the base configuration to make things more consistent. (Bitbake rev: bcdc2f73e3b4a10b1e479c2891f251d9507a9e30) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: Abstract anonymous function execution into a functionRichard Purdie2018-03-041-4/+7
| | | | | | | | | This allows us to call this code from other contexts without duplicating it. (Bitbake rev: c6be487f9bd5d95915f2495d555b9f539adb1d44) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Ensure only recursive task dependencies are prunedRichard Purdie2018-03-011-1/+1
| | | | | | | | | | | | | If a standalone tasks adds a dependency on X:do_build, the code in runqueue would currently remove it if that do_build was part of an image recipe which uses recrdeptask on do_build. Such individual tasks shouldn't do this, therefore tweak the recursive reference code to only process recurseive tasks, not all tasks. (Bitbake rev: 4cfca360891e1ed876a9c19487b4f6210686af26) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Handle missing donestamp file when content is validNathan Rossi2018-02-142-11/+114
| | | | | | | | | | | | | | | | | | | | | | | | In order to allow users to manually populate the download directory with valid content change the assumption that missing the donestamp file means unfetched content. This allows users to populate the download dir without needing to create dummy .done files such that a user does not need a PREMIRROR when using BB_NO_NETWORK to provide valid content files in the download directory. To ensure the correct result this change also fails first if the localpath does not exist. This prevents further parts of the function attempting to calculating the checksum on non-existent files. This also fixes some edge conditions around where if the donestamp exists but the localpath does not it returns, and did not remove the donestamp. Also added test cases to cover this use case and additional use cases where for example the fetcher does not support checksums. (Bitbake rev: a335dbbb65d5b56e71d98cf3e4fa9bfbec1dcde6) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/npm.py: Fix inverted condition to prevent infinite loopBöszörményi Zoltán2018-02-141-3/+5
| | | | | | | | | | | | | | | At least the cli-color node module has dependencies that have cyclic dependency among themselves. npm.py is prepared to deal with such a case but the condition is handled only for downloading or not a dependency again, but then it goes checking the its dependency which causes an infinite loop in _getdependencies(). Make this function simply return when a dependency is already downloaded and only download and check its dependencies when not. (Bitbake rev: 545540420112992e53f4a83104af10452df168d0) Signed-off-by: Zoltán Böszörményi <zboszor@pr.hu> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: parse: fixes for resolve_file()Robert Yang2018-02-143-7/+2
| | | | | | | | | | The resolve_file() calls mark_dependency(), so the one which calls resolve_file() doesn't need call mark_dependency() again. (Bitbake rev: 4682571107323a39b42cd9ec8ee67419e7f15acc) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cache: improve debug messageRobert Yang2018-02-141-1/+4
| | | | | | | | | | | | | * Print message when cachefile is found/not can help debug. * Update "Using cache in" to "Cache dir:" since it was the same as the debug message of "codeparser & file checksum caches", which caused confusion. And whether the cache file will be used or not is still unknown at that time, so just print the cache dir. (Bitbake rev: c8d3a2016f432e8ed9e99d9c28850149ab6fd6d8) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: don't stop file notifier when cooker is shutdownRobert Yang2018-02-142-2/+2
| | | | | | | | | | | | | | | | | | | | | It should be live/exited with server rather than cooker, fixed: $ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quilt [snip] ERROR: When reparsing /workspace1/lyang1/poky/meta/recipes-connectivity/openssl/openssl_1.0.2m.bb.do_package, the basehash value changed from c216f7f4fdd3cf4a0b10b975a636426c to d5a8e9431ab261381752d7a64c7b2fa9. The metadata is not deterministic and this needs to be fixed. [snip] This is because the server doesn't know local.conf is changed since the notifiers are stopped, so it doesn't reparse, and then we would get the errors, let the notifiers live/exited with server can fix the problem. (Bitbake rev: a6a641cb9c5f3abe901b150da915372e295383d7) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: fix for BBFILE_PATTERN matches bbappendRobert Yang2018-02-141-3/+7
| | | | | | | | | | | | | | | | | | | | | The old code couldn't handle nestled layers correctly, e.g.: parent_layer/sub_layer/foo.bb Note there are two layers, parent_layer and sub_layer. And in parent_layer/conf/layer.conf: BBFILE_PATTERN_parent_layer = ""^${LAYERDIR}/" This setting is incorrect since it also matches parent_layer/sub_layer/foo.bb, so it warns that no files matched sub_layer, this is the expected behavior, but it doesn't warn when there is a parent_layer/sub_layer/bar.bbappend, this was incorrect since the bbappend is also matched by BBFILE_PATTERN_parent_layer, it should warn and let the user fix the problem. Check the bbappend in already "matched set" before return it as matched by "unmatched set" can fix the problem. (Bitbake rev: ec90245d28e52ea718d2ce084eb304cdc4355c9c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: skip when BBFILE_PATTERN is emptyRobert Yang2018-02-141-0/+1
| | | | | | | | | There is nothing to do when BBFILE_PATTERN is empty. (Bitbake rev: b7da31641c24c53d47ea45a7119d1bd353011b39) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: Use 'git-make-shallow' from bin directoryNathan Rossi2018-02-142-5/+5
| | | | | | | | | | | | | | Move the code that existed in tests/fetch.py for determining the path to 'git-make-shallow' into the git module and reference it. This ensures that 'git-make-shallow' is always available and the desired version regardless of the path variable or whether git exposes the command. (Bitbake rev: 6b508ab8fd5aa796c1c00c970e81e5e93f84d35d) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main/runqueue: Rework runall task and add runonly optionRichard Purdie2018-02-143-14/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | The runall commandline option was confusing people. There are in fact two different behaviours people may want. a) For a given target (or set of targets) look through the task graph and run task X only if its present and would have been built. b) For a given target (or set of targets) look through the task graph and run task X if any recipe in the taskgraph has such a target even if it wasn't in the original task graph. I've decided to interpret the existing "runall" option as b), even if right now if behaves like a). For a), which is a valid use case, this patch adds a "runonly" option. With both behaviours present, I'm hoping we can then kill off the "fetchall", "checkuriall" and other tasks from OE metadata and replace them with this option. This would significantly speed up task graph processing. (Deleting the checkuriall and fetchall tasks takes "bitbake core-image-sato -g" from 22s to 8s). (Bitbake rev: 546a662c877b2d3af35e3996950582ed2df41fe4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Rewrite and optimize recrdepends handlingRichard Purdie2018-02-061-51/+83
| | | | | | | | | | | | | | | | | | This is a performance sensitive piece of code and the shear number of recursive loops is causing a significant and unscalable performance pain point. This change moves to a two step approach, firstly generating a list of recursive dependencies for any task, then applying this to the recursive tasks, iterating over things until no further dependencies are added. It was noticed an optimisation is possible and the list of recursive tasks need not contain the taskname, only the base task id. This allows a significant performance improvement and limits the size of the resursive task lists, improving speed. (Bitbake rev: eba738ac5672556eaab4f3374c8025c322761c4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Optimize recrdepends handlingRichard Purdie2018-02-061-10/+13
| | | | | | | | | | | | | | | | We can optimise the loops slightly so we only process given substrings once rather than many times. This means expanding out add_resolved_dependencies. Also add a function which allows replacement of the task element of a task id, reducing the amount of string handling we're doing in a performance critical loop. Its also clear that later code adds to the tasks depends so we don't need to add .depends() to extradeps at the start. (Bitbake rev: 4ad281224e92b5f94e3a9c17e8898ec8f1086cdc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Remove unused variablesRichard Purdie2018-02-061-2/+0
| | | | | | (Bitbake rev: 7e56c285f5ebae9b4b367514e60e3a6ba8cd5693) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-21 07:33:16 +0000