| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
| |
Function "git" is defined in super class FetcherTest already.
(Bitbake rev: bfed4ad2d50bcd635bb2ef99977e03e879fc79a2)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As reported by Martin Jansa <Martin.Jansa@gmail.com>:
bitbake/lib/bb/cooker.py:16: DeprecationWarning: module 'sre_constants' is deprecated
import sre_constants
it's deprecated since 3.11 with:
https://github.com/python/cpython/issues/91308
The correct replacement for our usage is re.error so use that instead.
(Bitbake rev: 3c0cd401472ffee06d5a93bdba566cb033851fcf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In theory this should have been worked around but is still occurring. Add
it to the list of things to ignore when bitbake is shutting down.
Traceback (most recent call last):
File "/usr/lib64/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 698, in startCallbackHandler
event = self.reader.get()
File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/bitbake/lib/bb/server/process.py", line 722, in get
res = self.reader.recv_bytes()
File "/usr/lib64/python3.9/multiprocessing/connection.py", line 221, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/usr/lib64/python3.9/multiprocessing/connection.py", line 426, in _recv_bytes
return self._recv(size)
File "/usr/lib64/python3.9/multiprocessing/connection.py", line 384, in _recv
chunk = read(handle, remaining)
TypeError: an integer is required (got type NoneType)'
(Bitbake rev: 7a28ac4fe478bee1e52e84412da9626495f9c6c7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We're seeing UI deadlocks occasionally and this is possibly due to the
use of a daemonic thread in the UI event queue processing. This thread
could terminate holding a threading Lock() which would cause issues
for the process when exitting.
Change the shutdown process to handle this more cleanly.
(Bitbake rev: f5ad8349a5dbff9824a89f5708cfd011d61888c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The open coded lock acquire/release in the UI event handler doesn't
cover the case an exception occurs and if one did, it could deadlock
the code. Switch to use 'with' statements which would handle this
possibility.
We have seen deadlocks in the UI at exit this so this removes a
possible cause.
(Bitbake rev: bd12792f28efd2f03510653ec947ebf961315272)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: cd7cce4cf4be5c742d29671169354fe84220b47a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Basic test to cover functionality of BB_FETCH_PREMIRRORONLY using
local git repository. Local repository has been chosen to allow easy
manipulation with the repo to simulate behaviour reported in
[Yocto 13233]
(Bitbake rev: 773e0815ba0c2183afcb169cda525b7625e60e42)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This should fix [Yocto 13353] and related to [Yocto 13233] as well.
Previously if git repo mirror has been updated in between of two builds
fetcher of the second build didn't try updated mirror but switched to
git clone from upstream instead. This is problem for offline builds.
Fix this to raise MirrorException if BB_FETCH_PREMIRRORONLY has been
specified by the mirror doesn't contain SRC_REV.
[YOCTO #13353]
[YOCTO #13233]
(Bitbake rev: b47ecab3e3aad5c5c376ec023aa82a51aa0f3b86)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
This allows OE to depend on the unihash copy functionality.
(Bitbake rev: 928aa3dc715d1d3be4b7a8d1247ea2ab5ee2d38e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
We see rare failures in eSDK generation with zero sized unihash cache files. This is
almost certainly due to races in the cache file being updated. Add a copy function
where the cache file can be copied with the lock held to avoid this.
(Bitbake rev: 9e72a3915e36cb843037040cb68a82077436dbef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Very occasionally we see errors in eSDK testing on the autobuilder where the task
hashes in the eSDK don't match what was just built. I was able to inspect one of
these build directories and noticed that the bb_unihashes.dat file in the eSDK
was zero sized. Whilst inspecting the code to understand the cause, I noticed that
updated hashes are not saved out in subsequent updates of the values in the rehash
process.
Add a missing sync call to ensure this happens.
(Bitbake rev: 7912dabbcf444a3c3d971cca4a944a8b931e301b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
* fixes:
bitbake/lib/bb/fetch2/osc.py:93: DeprecationWarning: invalid escape sequence '\d'
match = re.match('<directory ?.* rev="(\d+)".*>', output)
(Bitbake rev: 6a346df51b96a6c0e1ee516df36eb0b6c292b063)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Add support to query latest revision. This makes it possble to use
osc fetcher without specifying the rev parameter.
(Bitbake rev: aa4cee1bb7415c498e4dc6af4dbb3d0c841faf2e)
Signed-off-by: Gunjan Gupta <viraniac@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We currently have no API to be able to remove all the potential stamps of a
task. It is unusual to need to do this, particularly as you could race against
other things happening in the system but we do have a use case for this in
cleaning up sysroots in OE-Core. The alternative is to mess with CLEANMASK in
OE-Core but that is just going to add potential for errors.
We need the first part of the make_stamp() function so separate that out so
it can be called seperately.
(Bitbake rev: 4d671504a25863018ac51c21c005cef0a4d8f05c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If SRC_URI contains python function that extends vardepvalueexclude its
value is being tracked by sstate-cache, which can lead to rebuilds if
value is set dynamically (for example gerrit replicas).
Return empty string if vardepvalueexclude is checked to fix this
behaviour.
(Bitbake rev: f5f9a7b89a7d8321f03184e61ad6d5ed8d0f840e)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current fetcher seemed to have some issues that made it difficult when
trying to use the same. This patch fixes the following
* Make consistent use of the path that needs to be used as oscdir
* The path mentioned in os.access in download function was not same as
ud.moddir which would result into invoking of fetch command instead of
update command even if directory already existed
* Before creating oscrc, make sure oscdir exists and create it if it does
not exist
* Updated the configuration to use apiurl and added a new parameter to
control whether http or https needs to be used to connect to apiurl
(Bitbake rev: 3ec78686f3c0ea2304097b86a965f9be4b0cb879)
Signed-off-by: Gunjan Gupta <viraniac@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
compare_sigfiles() recursively calculates differences on all dependent
tasks with changed hashes. This is done in arbitrary/alphabetical order, and
only the last of those results is returned, while everything else is discarded.
This changes the behavior to instead return the first difference and not calculate
any more, which significantly speeds up diffs of tasks with many dependencies.
(Bitbake rev: ea6a676c9aa2864c2eff40eea41ba09ce903a651)
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
We don't have less in HOSTTOOLS in OE and this can confuse git. Force the
pager to cat to be consistent and minimal everywhere.
(Bitbake rev: d3d406e8552fdd865dc58b419a84411736475ad2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
File names containing colons cause split() to return a list with more
than two elements, which will lead to a stack trace ending in:
ValueError: too many values to unpack (expected 2)
Split only once at the last colon, thereby making sure that only two
elements are returned.
(Bitbake rev: a70a7376a8708bde07959deb5d5842d7f84ee5f8)
Signed-off-by: Roland Hieber <rhi@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Users are misreading this message. Reorder the wording to make
the empahsis clearer on the shell environment.
(Bitbake rev: dd2475e36af470ea47b0841d181444e36aa6e947)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
We're using the wrong data store when trying to locate siginfo files,
fix this. Thanks to Gregory Lumen <gregorylumen@microsoft.com> for
spotting.
[YOCTO #14774]
(Bitbake rev: 0ed800e19a3197f8e622c8d3b630aae384e60aba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The path has been encoded by urllib.parse.quote(), so decode it back for ssh.
Fixed when fetch from PREMIRRORS via ssh:
$ bitbake bonnie++ libsigc++-2.0 -cfetch
scp: /path/to/downloads/libsigc%2B%2B-2.10.7.tar.xz: No such file or directory
(Bitbake rev: c1c8fc678eb4783cea3974328a5fa8d1b79f1266)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This probably means the osc fetcher isn't being used but fix the missing
parameter.
(Bitbake rev: a23c201cb6efc5c0abf763c26f905442f0eebb68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: d720dfa40620e64a557edef527148d58fcb1d858)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This is set at the start of the loop anyway so it does nothing. Drop
the pointless code.
(Bitbake rev: e6a3173c9cdf349ccbd4cf612868f92cce8717c8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 7254eb6b3e8ef504ef2274541dcc55f1d42238c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 625565087d8c9e7a6a79b0b4f3e5be2d77d5f100)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 42809f6acb79e39042e81d54c28efb92b7481e44)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 879f17ecd5ba09e217cef74f6a51339b145e8ef5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
The function has a loop where the variable is never used which I was going
to fix but the entire function never seems to be called so remove it entirely.
(Bitbake rev: 3bcb20f025907f4e88bbe3d14f5638d5f01010cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 140929b404ee1e2f5e0e1a3a1d3aa49fb3759ade)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 1c811ad6f10560e7a7fb6830cf83707551ba04bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: aca0ff85109f4b0f3c201c02c3f59cad7ee2e787)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
| |
(Bitbake rev: 543315e6463f15ca7ab2b4ef3e8ed41bb4207ccf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Fix an issue where two tests have the same name with one overwriting the
other.
(Bitbake rev: da812d938fd79e2cc7bdf355ccf5b0f9ead684c4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
This ignores flake8 rules:
* E402 module level import not at top of file
* E501 line too long
(Bitbake rev: e8b176de448dc387c7a578c92b52aef28591038f)
Signed-off-by: Marius Kriegerowski <marius.kriegerowski@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Also correct two comments, and move slash_re from _revision_key() to
the module top level (together with the new sha1_re).
(Bitbake rev: 98cad8636e9c82bc40a033bb83633ec994758eb0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During parsing, Python raises
RuntimeError: dictionary changed size during iteration
in getRuntimeProviders, if you happen to have a recipe
with an explicit RDEPENDS on a dynamic package containing a '+'
character, such as 'gtk+3-locale-en'.
This is because we're using the modified pattern as the
key into the packages_dynamic dict to append to rproviders,
and since that key doesn't exist, the dict is getting modified
to add a new, empty, entry for it. So even without the runtime
error, we'd be generating an incorrect result.
Fix this by using a local variable for modifying the pattern
and using the original key to retrieve the value on a match.
(Bitbake rev: 07de375c3e57f17ab7b47569186f24ecd9896825)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
logger.debug was giving an integer value (2) as event message, causing
knotty to crash when running with debug enabled.
bitbake/lib/bb/ui/knotty.py", line 685, in main
event.msg = taskinfo['title'] + ': ' + event.msg
TypeError: can only concatenate str (not "int") to str
Same issue also happens in the original code that was taken from
oe-core (openembedded-core/meta/lib/crate.py honister) / meta-rust.
(Bitbake rev: c212b0f3b542efa19f15782421196b7f4b64b0b9)
Signed-off-by: Ricardo Salveti <ricardo@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use bitwise operators to manipulate the received event mask in
_ProcessEvent.
Also minor clarification & clean up of the related comments.
(Bitbake rev: 2ab60c7be124d928d304ab1fb73f0dbff29964ae)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In `runtaskhashes`, the keys contain the absolute paths to the recipe. When
working with shared sstate caches (where these absolute paths can be different)
we see that compare_sigfiles does not identifiy a changed hash of a dependent
task as "changed", but instead as "removed"&"added", preventing the function
from recursing and continuing the comparison.
By calling `clean_basepaths` before comparing the `runtaskhashes` dicts, we
avoid this.
(Bitbake rev: 7358378b90b68111779e6ae72948e5e7a3de00a9)
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, if you use one of the functions from EXPORT_FUNCTIONS,
the meaning of cleandirs and fakeroot are lost. This leads to the function
changing in behaviour depending upon it's caller context. This isn't intended
so add mapping for the cleandirs and fakeroot flags too.
This does break devtool in OE-Core and there is a separate fix for that.
[YOCTO #8621]
(Bitbake rev: b074f4aff00923acc5bf6649d204d541a79fd2b6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
GIT_SSH_COMMAND is more convinient to use if arguments have to be passed
and the user doesn't want to create a wrapper script around ssh.
(Bitbake rev: 5e746cb9d26ce87d6c9d52d9022122081a9811c5)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change the owner information in the mirror tarballs generated using
BB_GENERATE_MIRROR_TARBALLS="1". This is an extension of commit
0178ab83, which used the original pokybuild:user information, but failed
to clean up the numerical user and group ids. Now set the more canonical
values of oe:oe and 0:0.
(Bitbake rev: 37437115d3fb1a9f5d8ed7356a0fc01a408e4f8c)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
CC: Marek Vasut <marex@denx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If there are spaces in the URI filenames it can break the code.
We already solved this issue once somewhere else in the code so
use the same regex trick here as well.
We should ultimately refactor this code but at least fix the issue
for now.
(Bitbake rev: 57e2fc4d7f60afea4d4b2c84761324dd99e74a87)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
While the progress bar is good at conveying information about how much
of a task has executed, the elapsed time of the task is still very
much relevant to show.
(Bitbake rev: 41eeb4f34fb2306303f7688ec5e0ae965a573aa4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This was only meant to be added for the handle_contains function in
a previous commit, fix it.
(Bitbake rev: 7399be398df39bc29e1b5eaac23b29cfae017abd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The kernel inotify code can set more than one of the bits in the mask,
fsnotify_change() in linux/fsnotify.h is quite clear that IN_ATTRIB,
IN_MODIFY and IN_ACCESS can arrive together. We don't care about two
of these from a bitbake perspective but it probably explains why in
real world builds, we've seen:
pyinotify.ProcessEventError: Unknown mask 0x00000006
This module code assumes only one mask bit can be present. Since we
don't care about two of these events, just mask them out for now.
The "upstream" code is unmainained since 2015.
(Bitbake rev: 7fb93c2ce6dacd9b53fc3a227133a3493e6a6a1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The recent inotify changes can cause entire build trees to be monitored
which is suboptimal for performance.
Rather than trying increasingly convoluted tricks to try and handle add/removed
directories, rebuild the inotify watch when we reparse the configuration or
metadata.
(Bitbake rev: 3df322a200c28b45af1f2c92478c85eb7d20c38b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The python gc can trigger whilst we're holding the event stream lock
and when cleaning up objects, they can trigger warnings. This translates
into a new event which would then need the lock and we can deadlock.
Disable gc whilst we hold that lock to avoid this unfortunate and
problematic situation.
(Bitbake rev: 96a6303949cefd469bcf5ed250ff512271354357)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|