| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
While the progress bar is good at conveying information about how much
of a task has executed, the elapsed time of the task is still very
much relevant to show.
(Bitbake rev: 41eeb4f34fb2306303f7688ec5e0ae965a573aa4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This was only meant to be added for the handle_contains function in
a previous commit, fix it.
(Bitbake rev: 7399be398df39bc29e1b5eaac23b29cfae017abd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The kernel inotify code can set more than one of the bits in the mask,
fsnotify_change() in linux/fsnotify.h is quite clear that IN_ATTRIB,
IN_MODIFY and IN_ACCESS can arrive together. We don't care about two
of these from a bitbake perspective but it probably explains why in
real world builds, we've seen:
pyinotify.ProcessEventError: Unknown mask 0x00000006
This module code assumes only one mask bit can be present. Since we
don't care about two of these events, just mask them out for now.
The "upstream" code is unmainained since 2015.
(Bitbake rev: 7fb93c2ce6dacd9b53fc3a227133a3493e6a6a1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The recent inotify changes can cause entire build trees to be monitored
which is suboptimal for performance.
Rather than trying increasingly convoluted tricks to try and handle add/removed
directories, rebuild the inotify watch when we reparse the configuration or
metadata.
(Bitbake rev: 3df322a200c28b45af1f2c92478c85eb7d20c38b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The python gc can trigger whilst we're holding the event stream lock
and when cleaning up objects, they can trigger warnings. This translates
into a new event which would then need the lock and we can deadlock.
Disable gc whilst we hold that lock to avoid this unfortunate and
problematic situation.
(Bitbake rev: 96a6303949cefd469bcf5ed250ff512271354357)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
contains items
Adding bb.utils.filter('WARN_QA', 'patch-fuzz', d) when WARN_QA is in
BB_BASEHASH_IGNORE_VARS or in vardepsexclude should not add a dependency
on WARN_QA.
Fix it and add some tests.
(Bitbake rev: 6aecc2fe51a52020f6f13be08449e18d42e7a6b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
We're still seeing issues with unclosed asyncio event loops. At the
init site, make sure any existing one is closed first to try and avoid
this.
(Bitbake rev: 78dee3c03c75a27531fcff26f9298fce2519bdde)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
In some cases we'd never have setup a siggen so don't error in that case.
(Bitbake rev: bbaaf2cf7b5a9339d3790610e622020c19d52f5a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Ensure any exiting hash server connection is terminated before we start
a new bitbake session. This avoids errors seen with memory resident bitbake
when the asyncio event loop isn't closed correctly.
(Bitbake rev: 42ff9de77f24e2a0bec48a14b64c4b538e00b4af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
When memory resident bitbake is active and we re-parse, the old module
configuration is present which can lead to strange errors. Reset this
when reparsing so the state is consistent. This fixes memory resident
bitbake errors.
(Bitbake rev: 951942c3c284ec2c62e730e145688033190af9b2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
The previous fix for inotify wasn't quite correct as we need to modify
bbseen before calling add_filewatch(). We also need to ensure the parse
mtime cache is cleared when directories are added/removed. There was also
a typo in the original fix and the wrong watcher was being changed. Fix
the various issues which improves memory resident bitbake testing results.
(Bitbake rev: 66cadd6be58bce5f7a56556cf92efd8159fb0b0e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Replace hyphens by em dashes when necessary
See https://www.grammarly.com/blog/hyphens-and-dashes/
- No uppercase after em dashes
- No uppercase after colons if what follows is not a
complete sentence.
- Fix spacing before colons ":"
- Replace em-dashes with colons for consistency in a section
(Bitbake rev: 72230d6a9976b3bfca1f1e6fb09736fec195e2fe)
(Bitbake rev: f1c4ac816e927f490fb9852c12aa408e8c9403b1)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes:
File "bitbake/lib/bb/build.py", line 585, in _exec_task
event.fire(TaskInvalid(task, d), d)
TypeError: __init__() missing 1 required positional argument: 'metadata'
(Bitbake rev: 16ea4429344e36ff022616abd7fa74e5964b6d6a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The "force" option to parser shutdown was often the cause of lockups and
there is no good reason we should have two different behaviours.
Change and unify the codepaths to always:
* Wait for longer for a controlled shutdown of a process (0.5s). Usually
it will be much faster if it has finished so the delay doesn't really matter.
* Send processes a SIGINT
* Failing that, send a SIGTERM
* Call .close() if available to release zombies
This means we no longer need the "force" parameter to the function so it is removed.
(Bitbake rev: de88c62ef873e9fce78ba162f5311d846de96f2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a parser process is terminated while holding a write lock, then it
will lead to a deadlock (see
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.terminate).
With SIGTERM, we don't want to terminate holding the lock. We also don't
want a SIGINT to cause a partial write to the event stream.
I tried using signal masks to avoid this but it doesn't work, see
https://bugs.python.org/issue47139
Instead, add a signal handler and catch the calls around the critical section.
We also need a thread lock to ensure other threads in the same process don't
handle the signal until all the threads are not in the lock.
(Bitbake rev: a40efaa5556a188dfe46c8d060adde37dc400dcd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
Not sure why this is so convoluted but we should simplify it!
(Bitbake rev: 6195343c46ba9d2685fc2d42366922f88ff3f369)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
If an exception occurs in the parsing process, ensure the cleanup
is called via all codepaths using a try/finally.
(Bitbake rev: e1ba2a69f1fc02f01a851bce20b1badf0b991f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
When parsing, the parser isn't servicing the main loop so a Ctrl+C in the
UI won't be seen on the cooker/server side. Fix this by returning when queue
timeouts occur. This helps where there is a hung or slow parsing thread.
(Bitbake rev: a2cde38311a51112dca5e7bb4e7feaf4e6a281b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
We shouldn't be generating exception inside a generator. Rearrange the
code to improve the handling of this. Also fix the misconverted code
from when multiconfig was added and pass the exception as "result".
(Bitbake rev: ae89e23696de2f27c00ae00922933395171de5d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
| |
(Bitbake rev: b4a157b2fe2fb481ffa40e0f32659d05dd6320c2)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
SRC_URIs
(Bitbake rev: 981d1929937519cd98200a6ef291d97a16d358ad)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
This parameter is now required by the git fetcher module
(Bitbake rev: d61b349581c219e7f9d50f683177184fa473cb83)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using BB_GENERATE_MIRROR_TARBALLS="1" to generate mirror tarballs
of git repositories, they leaked local information: username, group and
time of the last fetch. Remove all these by setting fixed information:
* uname = pokybuild
* gname = users
* mtime = committer time of newest commit in repo
The username and group value were taken from the archives available on
the downloads.yoctoproject.org mirror. The modification time is chosen
so it still retains some relationship to the contents of the archive.
(Bitbake rev: 0178ab83e6312e97e528aa8c5e12105f5165d896)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Thanks to great debugging from pavel@zhukoff.net we had a simpler reproducer
for the corruption see in oe-selftest when using BB_SERVER_TIMEOUT=60, i.e.
with bitbake in memory resident mode. This was effectively:
oe-selftest -r devtool.DevtoolUpgradeTests.test_devtool_upgrade devtool.DevtoolUpgradeTests.test_devtool_upgrade_git -j 1 -K
The issue is that if directories are removed (such as workspace), if
they are added again, we don't have the watches in place any more. This
patch adds some slightly paranoid checks to ensure we do the correct things
for directory additions and removals (we track directories, not files
specifically to avoid running out of watches).
[YOCTO #14023]
(Bitbake rev: 2c414f659d793d732041614caedd773959eb4f27)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
This debug is useful but the cooker shutdown or post_serve() may have cleanup
left so run after those.
(Bitbake rev: 1463fc0448d1a6a7265806a4a8b165b610dfb43f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
When we have a client connection, we should close that connection when reset()
is called on the siggen. Add the missing function.
(Bitbake rev: 770b4ea81b6126b0830e51649c40f7a46c64132a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We have shutdown functions within the async client code but the siggen
doesn't currently call them. We notice on python 3.10 (such as on fedora35)
that at exit, there is a stray asyncio process left behind. Usually this
doesn't cause problems but it could potentially be a cause of a hang.
For general cleanliness and completness, add in hooks to call the exit handler.
(Bitbake rev: 9ee3fb95330003878fbd64b3ce8897aad96fcd0f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
There are some commands where we want to see the events returned so allow
the caller to request this. This also allows us to fix an infamous bug in
the tinfoil testsuite in OE-Core.
(Bitbake rev: 0e8421c41d97d5d50a553d70c8f775d521f1a199)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the path to bitbake.lock is in a deep directory, bitbake will hang. The
reason was that the max file length limiting code (to 255 chars) was including
the directory name and it should only act on the filename within the directory.
Fix it to just use the base filename.
[YOCTO #14766]
(Bitbake rev: 89d70e7b71eecfe06592202f326e566c579ba01d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
I'd never spotted this until it was pointed out but the task isn't dependent,
it is a dependency. Fix this confusing reference.
(Bitbake rev: 93395559c9dda734a3dec9ffd9bb2156a71a2c17)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to 10 minutes
The keep alive timeout is excessively long at 83 minutes (5000 seconds),
reduce this to 10 minutes: this should be long enough that it rarely
triggers in normal builds, but when it does it has useful information.
(Bitbake rev: 2e47346b95b09d7ab8f0603e2d62cfb549dc1f5c)
(Bitbake rev: dcf52157d3635925491783be656c6b76d1efe1a4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In interactive bitbake sessions it is obvious what tasks are running
when one of them hangs or otherwise takes a long time. However, in
non-interactive sessions (such as automated builds) bitbake just prints
a message saying that it is "still alive" with no clues as to what tasks
are active still.
By simply listing the active tasks when printing the keep alive message,
we don't need to parse the bitbake log to identify which of the tasks
is still active and has presumably hung.
(Bitbake rev: f9f57fb7d2c8a13df1eb9d5b9766f15e229dcf97)
(Bitbake rev: 30f6c3f175617beea8e8bb75dcf255611e3fc2fd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a Dockerfile for building the PR service in a container.
Based on the hash equivalence server container [hashserv].
Use the following environment variables to configure the PR service
container:
- DBFILE: database filename. Default:
/var/lib/bbprserv/prserv.sqlite3.
- LOGFILE: log filename. Default: /var/lib/bbprserv/prserv.log.
- LOGLEVEL: logging level. Default: DEBUG.
- HOST: ip address to bind. Default: 0.0.0.0.
- PORT: port number. Default: 8585.
- DBMODE: database mode. Default: Empty (RW).
Note: DBMODE in RO: "--read-only".
[hashserv]: contrib/hashserv/Dockerfile
(Bitbake rev: a5497428b539e8598263924f63a9df8fe1ea70f9)
(Bitbake rev: 1d05abd92da56e284fcd904cf32bd12485903f10)
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Force a sync point for end-of build event handler force
the build's outcome status commit, to resolve a race
condition with the build completion takedown.
[YOCTO #14765]
(Bitbake rev: f76c13536e19d2401365c0fc240ef8dbcb3ee7e8)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a polling check on tracebacks in a build's log. This
can for example indicate that bitbake crashed, which would
stop the event stream that Toaster normally uses to detect
build errors.
[YOCTO #14085]
(Bitbake rev: 32b1c0b3477e359d2e2a61a23a294e317e417f95)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Creating the new log instances triggers a lot of python logging overhead
in a commonly called function (about 600,000 for parsing OE-Core).
We only need the log functionality if we're parsing, not if we just hit
from the cache. Therefore defer the log setup overhead until we know it
is a cache miss.
Whilst this complicates the code slightly, the performance gain is worth
it as for parsing OE-Core we drop 60 million funciton calls (from 225
overall).
(Bitbake rev: ac868167ad854f9bb32dcb2e63528870547805a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
We can call getVarFlags() instead of the multiple getVarFlag calls
which is a little more efficient. This reduces the number of overall
function calls so is good but probably isn't much faster (or slower).
(Bitbake rev: 505a4dd34e822bdf902d9b348dbcdf5b2c94e784)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
context lookup
The code tries to expand missing entities when they're encountered in
python expressions. Looking at traces, these are often things which
would not be in the data store such as "bb".
Optimise to skip the data store queries for things we know will never
be there. The expansion cache usually covers these but skipping
entirely cuts a few million function calls parsing OE-Core.
(Bitbake rev: 1ae712635a2eef3ecbf541e1f7cc2eeb2f3799c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
With the upcoming LTS, it is time we changed the bitbake version so
move to 2.0.
(Bitbake rev: 9a13bf8e20b1841ec99281d45be4c4fc1118438c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
By alphabetical order, to get the same order as in the HTML
output, sorted thanks to the ":sorted:" directive.
(Bitbake rev: 8b97af89316a4407dd07649e8aadd495edd86ee4)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
This template is not referenced anywhere and therefore not used.
It also refers to the obsolete "build mode" in the old docs.
(Bitbake rev: f89a35155e3b8d12ac609a165c63ed206751b8da)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
connection.features.autocommits_when_autocommit_is_off
was deprecated in 3.0 and is no longer present in 3.2
(Bitbake rev: cc0f526fb4298349d0eea44c8f35d3dd226cc8d6)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Follow the default guidelines from Django 3.2 setting DEFAULT_AUTO_FIELD
to django.db.models.BigAutoField which leads to these migrations.
(Bitbake rev: ccfdcf5ed6b5d2024c268bace30e53753d1f4da4)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New in Django 3.2.
Silence warnings by adding:
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
in toastermain/settings.py
(Bitbake rev: cbc5a8e339a8b5eb4cdd89ad898af432f05cadfb)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Django 3.2 no longer has django.utils.six module.
(Bitbake rev: ae03c1f05e37f70e72a2ac08f7c718b8fc4fc36e)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
Update docs links from www.yoctoproject.org to the docs.yoctoproject.org
equivalents.
(Bitbake rev: 74194962eb294c9bd1192ddd7935d57258712ee2)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Django 2.2.x goes EOL April 2022.
Switch the to current LTS Django 3.2.x supported until April 2024.
(Bitbake rev: 23a15c5703d5ef4190921f9bb0273e43b1de489c)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
Fix a bad patch merge and move the file to the correct path.
(Bitbake rev: d173f45a88cc9460734e7cc0b87952c0ea0ff270)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
Add script to safely generate the fixture files from a table
[YOCTO #14759]
(Bitbake rev: ec5e9fff8f71ba81bf5e9207abcd3011d63d86d9)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
'Dunfell' and 'Gatesgarth'.
[YOCTO #14757]
(Bitbake rev: 49308d1db4f7af81e31db21dc8954947de5976c7)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|