summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: server/process: Fix typo in code causing tracebacksRichard Purdie2020-08-261-1/+1
| | | | | | (Bitbake rev: 14caa3d4e5615252b9453162183980044d896d2f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Drop broken timeout functionRichard Purdie2020-08-261-14/+0
| | | | | | | | | I strongly suspect this function doesn't work with modern python so and its unused now, drop it. (Bitbake rev: a3033cea089c66c8b4614e7ee57c166f4262c590) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Avoid bb.utils.timeoutRichard Purdie2020-08-261-24/+27
| | | | | | | | | | | | | | I have a suspicion based on process traces that the flock() call is no longer interrupted by SIGALRM and hence the timeout doesn't work. We were relying on EINTR triggering around syscalls but python is likely protecting us from that in modern versions. Re-implement this code with a different mechanism which doesn't have that potential issue. (Bitbake rev: 8eb52afdfd4c3e6478d4f8cc56e99def3f1c924c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: compat.py: remove file since it no longer actually implements anythingChris Laplante2020-08-264-30/+21
| | | | | | | | | | Now that compat.py just imports Python standard library stuff, get rid of the layer of indirection. (Bitbake rev: e2be6defbb9fcf25f9df04c3b452d0dba48dfd03) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Use sys.executable for bitbake-serverRichard Purdie2020-08-261-1/+1
| | | | | | | | | | Using sys.executable ensures we're using the same python binary for the server as the client, which is probably advisable. "bitbake-server" is left as the process name as its more distinctive in process listings. (Bitbake rev: 387a339b330e8122a62a148249beb3f064dd4e3d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Drop globbing supprt in file:// SRC_URIsRichard Purdie2020-08-264-27/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Globbing in file:// urls is terminally broken. Currently when its used, the file checksum code is basically bypassed. This means changes to the source files don't change the task checksum, the task doesn't rebuild when the inputs change and things generally break. To make globbing work generically, we'd have to scan the file system for all possible matches to the glob and log whether they exist or not. We can't simply log the files which exist, we have to also know which files could later exist and influence the choice of file so we know when to reparse. For a simple file://xxx/*, this could be done but for bigger patterns, it becomes much more problemtic. We already support file://xxx/ in urls. So, lets decide we'll not support globs in file://urls. Worse case users can put files in a directory and reference that, moving files into place if needed. Remove all the glob special cases (see the comments if anyone doesn't believe this is terminally broken) and error to the user if they have such urls. (Bitbake rev: 0c9302d950c6f37bfcc4256b41001d63f668bdf7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: fix most undefined code picked up by pylintFrazer Clews2020-08-2522-42/+51
| | | | | | | | | | Correctly import, and inherit functions, and variables. Also fix some typos and remove some Python 2 code that isn't recognised. (Bitbake rev: b0c807be5c2170c9481c1a04d4c11972135d7dc5) Signed-off-by: Frazer Clews <frazerleslieclews@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure cooker's enviroment is updated on updateConfigRichard Purdie2020-08-251-0/+4
| | | | | | | | | | | | | Only the env variables which were added to the datastore were being updated. We need to update the whole copy, we just only trigger a reparse if any of the variables making it into the datastore change. This avoids a bug where variables such as DISPLAY in a new UI context would break under memory resident bitbake. (Bitbake rev: 4bb71b627767297269e762b414443e15e28bfac4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Don't use sys.argvRichard Purdie2020-08-251-5/+4
| | | | | | | | | | | | We should not be using sys.argv inside the server to decide which targets the user added on the commandline. There might not even be a commandline. Thankfully the targets variable is easily accessible in this context and contains this exact data we want. (Bitbake rev: 5b12bf30bccdd00262e74964223220c649040be4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add bitbake-server and exec() a new server processRichard Purdie2020-08-251-39/+55
| | | | | | | | | | | | Trying to have a new python process forked off an original doesn't work out well and ends up having race issues. To avoid this, exec() a new bitbake server process. This starts with a fresh python interpreter and resolves various atexit and other multiprocessing issues once and for all. (Bitbake rev: 9501dd6fdd7a7c25cbfa4464cf881fcf8c049ce2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Log extra threads at exitRichard Purdie2020-08-251-0/+3
| | | | | | | | | Dump info into the logs if there are extra threads left at process exit time. (Bitbake rev: 1c9496797b753e67351bd5cb98ef2b8e9435d51e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure BB_ORIGENV is updated by changes to configuration.envRichard Purdie2020-08-251-0/+6
| | | | | | | | Changes to configuration.env were not updating BB_ORIGENV, fix this. (Bitbake rev: c5fbd8452f87e0a2d234eaf27d0450aacdeb8891) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main/server/process: Drop configuration object passingRichard Purdie2020-08-255-28/+12
| | | | | | | | | | | | | | | | | | | The first thing the UIs do is update the server config from the UI. We can just rely upon that and start the server with a standard config, removing the need to pass the confusing configuration object around as well as configParams, which contains a similar copy of some of the data. This makes memory resident bitbake work the same way as the normal mode, removing the opportunity for some class of bugs. The xmlrpcinterface and server_timeout values are passed in at server startup time now and there no longer a second option in the configuration which is effective ignored once the server starts. (Bitbake rev: 783a03330802e83c525c55522e3ee2a933bded3a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Move the socket code to server process onlyRichard Purdie2020-08-251-17/+15
| | | | | | | | | The sock object isn't used client side so we can just created it in the server process and save passing around the fd/object. (Bitbake rev: ee5d2c92dcce89ccb701e028ffc6419eb315f5ce) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Defer configuration init to after UI connectionRichard Purdie2020-08-253-27/+37
| | | | | | | | | | | | | Currently we end up parsing the base configuration multiple times as initially, the right settings haven't come from the UI. We can defer this until later in startup using runCommand as a trigger. The advantage to doing this is improved startup times and ultimately we should be able to avoid the double parse of the base configuration. (Bitbake rev: 3caa43b665604475d2c87ba505efb0b9fca9c2e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/cookerdata: Ensure UI event log is updated from commandlineRichard Purdie2020-08-252-8/+12
| | | | | | | | | Currently the eventlog is not handled correctly for memory resident bitbake. Fix this by allowing adpations to configuration changes. (Bitbake rev: f7d2c9116116659ea42260a3bb96dca100aadae7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Ensure UI options are updated to the serverRichard Purdie2020-08-251-1/+3
| | | | | | | | | | | | There were some options which were not being passed to the server. This was breaking, particularly in memory resident bitbake mode. Add the missing options to the correct place to ensure the server is updated correctly. (Bitbake rev: 5dc178e7fae3ca8558146e385a05f5d96a021777) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/cookerdata/main: Improve loglevel handlingRichard Purdie2020-08-253-9/+16
| | | | | | | | | | | Rather than passing debug/verbose/debug_domains around, pass the computed output of these. Ensure that the cooker sets the levels to match the levels currently set in the UI and generally try and make it easier to understand what the code is doing. (Bitbake rev: f0535beecc692a6213be2a22f9eef5956450ecf8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build/msg: Cleanup verbose option handlingRichard Purdie2020-08-255-16/+13
| | | | | | | | | | | | The levels of indirection to set these verbose logging options is rather crazy. This attempts to turn things into two specific options with much more specific meanings. For now its all still controlled by the commandline verbose option and should funciton as previously, with the addition that the BB_VERBOSE_LOGS option can now be task specific. (Bitbake rev: 423c046f2173aaff3072dc3d0882d01b8a0b0212) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Drop cups.org from wget status checksRichard Purdie2020-08-241-3/+1
| | | | | | | | | Its becomming clear the upstream server doesn't like this, drop these two urls from the test, not sure we need them here anyway. (Bitbake rev: ab2ef942dc21f9639793c972f2e546edf9444783) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Explictly shut down the sync threadRichard Purdie2020-08-241-1/+7
| | | | | | | | | | | | | | | | Hongxu Jia reported a problem where the bb_cache files were not always being written out correctly. This was due to the sync thread being terminated prematurely. Whilst the preceeding changes mean the exit handler for this thread is now correctly called since we switch to using sys.exit() instead of os._exit(), this write can happen after we drop the bitbake lock, leading to potential races. Avoid that headache by adding in explicit thread join() calls before we drop the lock (which atexit or Finalize can't do). (Bitbake rev: afd1900939f7b042297558f4cb01f50f3a299267) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure parse_quit thread is closed downRichard Purdie2020-08-241-6/+8
| | | | | | | | | | | Each run through the parser would leak a thread from the queue created to shut the parser down. Close this down correctly and clean up the code flow slightly whilst in the area, making sure this thread does shut down correctly (we don't care if it loses data). (Bitbake rev: 51ba35e9bbd4da8b5f3b3b5b8213cb634a6b0f06) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: Remove buffering parameterRichard Purdie2020-08-241-4/+1
| | | | | | | | | | The buffering parameter was removed in python 3.1 and made default so we can clean up the code. This removes weird looking double exceptions when connections fail. (Bitbake rev: 06b7bafbd18a47c8db2f7b943dc535c65df176bf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz ↵Jean-Francois Dagenais2020-08-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | present Before this fix, this example basepath (a): virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb:do_compile would get incorrectly "cleaned" into: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb When searching backwards in `a` trying to isolate the 'virtual:xyz' to add it to the end of the string, we need to consider `a` still has the recipe path and taskname. So stoping the rsplit after only 1 split is not enough. We want to reach the second ':' from the end. This way, we obtain: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native reviewed-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com> (Bitbake rev: d193d93422a0ad62aa35b5d4ca5da8d422f72180) Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: make shell traps less chatty when 'bitbake -v' is useduninative-2.9Chris Laplante2020-08-211-2/+3
| | | | | | | | | | | | This shuts up both the DEBUG and EXIT handlers. Also, remove an unneeded "set -o errtrace" (i.e. set -E), since we no longer use an ERR trap. (Bitbake rev: 89e851fa0403d1e98aeed69990101e3f84f0b283) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace with the original metadata locations of ↵Chris Laplante2020-08-152-1/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash shell funcs Leverage the comments that emit_var writes and the backtrace that the shell func writes to generate an additional metadata-relative backtrace. This will help the user troubleshoot shell funcs much more easily. Example: | WARNING: /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955:171 exit 1 from 'exit 1' | WARNING: Backtrace (BB generated script): | #1: myclass_do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 171 | #2: do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 166 | #3: actually_fail, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 153 | #4: my_compile_extra, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 155 | #5: do_compile, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 141 | #6: main, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 184 | | Backtrace (metadata-relative locations): | #1: myclass_do_something, /home/laplante/repos/oe-core/meta/classes/myclass.bbclass, line 2 | #2: do_something, autogenerated, line 2 | #3: actually_fail, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 36 | #4: my_compile_extra, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 38 | #5: do_compile, autogenerated, line 3 ERROR: Task (/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 542 tasks of which 541 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. (Bitbake rev: ae1aa4ea79826c32b20e1e7abdf77a15b601c6f2) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace when a Bash shell function failsChris Laplante2020-08-151-11/+50
| | | | | | | | | | | | | The trick here is to use a DEBUG trap to record the line number of each command as they execute. This is so we can report the real line number of the failing command, which is otherwise not possible in a Bash EXIT trap. See http://gnu-bash.2382.n7.nabble.com/trap-echo-quot-trap-exit-on-LINENO-quot-EXIT-gt-wrong-linenumber-td3666.html (Bitbake rev: 8366e2ad5fdad3972766b40b23188908330067ee) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add extra logfile flushingRichard Purdie2020-08-131-13/+22
| | | | | | | | | | | | There is the possibility data is being lost from the logfile due to data buffering. Add in a couple of extra flush calls to ensure data is being written out before the lock file is dropped. Possible fix for [YOCTO #14000]? (Bitbake rev: 596ea229a87d26a8e970c7ee77179519ad081fef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Pass timeout/xmlrpc parameters directly to the serverRichard Purdie2020-08-131-4/+5
| | | | | | | | Further cleanup, just pass these settings directly. (Bitbake rev: ac2284357f1fc7044dac9c146fad218fc9906412) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Simplfy idle callback handler functionRichard Purdie2020-08-134-14/+9
| | | | | | | | | | Having the idle callbacks abstracted via the configuration object makes no sense. Its like this for historical reasons from the multiple server backends but we don't need this now so simplfy. (Bitbake rev: e56c49717355c9493b07d5fc80981a95ad8a0ec8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Remove pointless process forkingRichard Purdie2020-08-131-3/+2
| | | | | | | | | | We already call bb.daemonize.createDaemon() in BitBakeServer so the extra multiprocessing.Process() appears to be totally unneeded and just an extra layer of forking which confuses things. Remove it. (Bitbake rev: d214e55c45f9733b3289138feec0ae3361a4a48b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: emit filename/lineno information for shell functionsChris Laplante2020-08-081-0/+6
| | | | | | | | | | | | | | | | | | Make it easier for users to debug shell task failure by including some breadcrumbs in the emitted .run file that (hopefully) points to the .bb/.bbclass file where the shell function was defined. Unfortunately this won't work with functions with _append or _prepends, since BitBake wipes the filename/lineno information. This shouldn't be too hard to fix; for now, you'll just see comments like this for such functions: [YOCTO #7877] (Bitbake rev: 9747211cbb45401cbf4dd0409e9c80c648a178c6) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/color: add test suite for ANSI color code filteringChris Laplante2020-08-081-0/+95
| | | | | | | | | Includes tests for bb.progress integration. (Bitbake rev: c472a8da521cc7f1d61ac2f28596167d47ab8a5a) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: filter ANSI escape codes before looking for progress textChris Laplante2020-08-081-3/+22
| | | | | | | | | This is in prepartion for introducing the log-colorizer bbclass into poky. (Bitbake rev: 889a873d71a6543efb71a0eb4ea6632c9f17175d) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: fix hypothetical NameError if 'progress' isn't setChris Laplante2020-08-081-3/+5
| | | | | | | (Bitbake rev: ff821022ef1fdf05482590d8e4fe003abf227135) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: modernize syntax, formatChris Laplante2020-08-081-10/+17
| | | | | | | | | Also fixes DummyMultiStageProcessProgressReporter calling the wrong super __init__ (Bitbake rev: 7a1b4a7e4fffe54afe8d1d7e169ff558ad8c92d9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: create_progress_handler: fix calling 'get' on NoneTypeChris Laplante2020-07-291-1/+1
| | | | | | | | | Just use the |resolve| function which already takes care of it. (Bitbake rev: 91b809a0902ffd42be4edf7f0a7d25e6d354d822) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print traceback if progress handler can't be createdChris Laplante2020-07-291-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ERROR: expat-native-2.2.9-r0 do_compile: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_compile.90289 Log data follows: | ERROR: 'NoneType' object has no attribute 'get' ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_compile) failed with exit code '1' After: ERROR: expat-native-2.2.9-r0 do_configure: Failed to create progress handler ERROR: expat-native-2.2.9-r0 do_configure: Traceback (most recent call last): File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 400, in exec_func_shell logfile = create_progress_handler(func, progress, logfile, d) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 344, in create_progress_handler cls_obj = functools.reduce(resolve, cls.split("."), bb.utils._context) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 343, in resolve return x.get(y) AttributeError: 'NoneType' object has no attribute 'get' ERROR: expat-native-2.2.9-r0 do_configure: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_configure.22982 ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_configure) failed with exit code '1' (Bitbake rev: ba0472a672eacbbbb2295252a0d2056d3399c6a9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Account for xmlrpc connectionsRichard Purdie2020-07-281-1/+1
| | | | | | | | | | UI control can happen via the xmlrpc connection. Account for this when timing out UI connections. This was causing issues for toaster on systems where it couldn't parse the metadata within the timeout. (Bitbake rev: fa85a8263971c25e67fa3b421c686a90e46acd87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix UI first connection trackingRichard Purdie2020-07-281-1/+3
| | | | | | | | | | We're only meant to be doing UI connection timeouts on the first connection but haveui changes for each connection. We need to add a specific variable to track this correctly and get the intended behaviour. (Bitbake rev: e7c387c2e2fb2cc3ca1dc9d2029362909c326d72) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: command: Handle multiconfig in findSigInfoJoshua Watt2020-07-251-3/+3
| | | | | | | | | | Changes the findSigInfo command to accept a recipe specified with the multiconfig prefix (Bitbake rev: 379951b6417eacbafc92ac1413ae1358bafdddfb) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle multiconfig name mappings correctlyRichard Purdie2020-07-231-1/+1
| | | | | | | | | | | | | | "bitbake mc:arm:bash mc:arm:busybox" works but "bitbake multiconfig:arm:bash multiconfig:arm:busybox" does not. The reason is the list is modified whilst iterating. Don't do that. [YOCTO #13607] (Bitbake rev: cd041a78d96e656438d93fb1e288080b8a6fe8bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Improve multiconfig configuration error reportingRichard Purdie2020-07-231-1/+6
| | | | | | | | | This avoids a traceback if an invalid multiconfig is referenced in the bitbake commandline and tweaks the message to make it more understanable. (Bitbake rev: f31d7d0ad57b0ecc2ae06ed4b547c98df2aaa1a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Allow deltask to take multiple tasknamesRichard Purdie2020-07-213-10/+11
| | | | | | | | | | | | | | | | | | | deltask currently supports only one task to delete but it would be useful if it could support a variable which gets expanded to allow flexibility in the metadata. This is simple to support in bitbake and is how other directives such as inherit operate so adjust the parser/code to handle that. It means that syntax like: EXTRA_NOPACKAGE_DELTASKS = "" deltask ${EXTRA_NOPACKAGE_DELTASKS} is now allowed. (Bitbake rev: 883d926120833c85a16dcf60425dd7af7699046a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Fix unmatched files handling leading to misleading warningsRichard Purdie2020-07-213-32/+52
| | | | | | | | | | | | | | | | | | | | | Currently if all recipes in a layer are skipped, there are warnings that the BBFILE_PATTERN_ entry didn't match anything. We probably shouldn't do this for skipped recipes. The current code is hard to understand, not least as it passes variables which functions modify by reference rather than giving a return value. Update calc_bbfile_priority() to return values rather than modifying them. Refactor the code to try and make it clearer what its doing and fix the skipped recipe issue by passing in the list of parsed files. The function is complicated by the need to not rerun regex matching more than we ever have to which complicates the flow, it would be easier if we just reran operations multiple times. (Bitbake rev: 969cb27b4d978551817612ff4558bec81cfb655c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix note reference -> infoRichard Purdie2020-07-211-1/+1
| | | | | | | | Its bb.note or logger.info, this avoids a backtrace. (Bitbake rev: 82c534ca1a1313de067b0d79c79857e89fa2764a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure UI-less servers don't sit in infinite loopsRichard Purdie2020-07-131-4/+15
| | | | | | | | | | | | | | | | | | | | If server startup is broken for some reason (e.g. lockfile issues) and no UI connection is made, the server will just sit inifinitely waiting. Add a timeout upon startup in the non-memory resident case so that such infinite waits are avoided. In the memory resident case, the server wouldn't have shut down in the first place or will timeout according to configuration. Since any race may mean the socket file is no longer present, ensure the unlink doesn't fault upon exit, thus ensuring any hashequiv or PRServ is removed from memory, allowing all processes to exit cleanly in such scenarios. (Bitbake rev: 39888b750df12478e8bdea6727cca112dce1df85) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix a rare lockfile raceRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're seeing rare occasional races on the autobuilder as if two server processes have the lockfile at the same time. We need to be extremely careful this does not happen. I think there is a potential race in this shutdown code since we delete the lockfile, then call unlockfile() which also tries to delete it. This means we may remove a lock file now held by another process if we're unlucky. Since unlockfile removes the lockfile when it can, just rely on that and remove any possible race window. An example cooker-deamonlog: --- Starting bitbake server pid 2266 at 2020-07-11 06:17:18.210777 --- Started bitbake server pid 2266 Entering server connection loop Accepting [<socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([]) Processing Client Connecting Client Running command ['setFeatures', [2]] Running command ['updateConfig', XXX] Running command ['getVariable', 'BBINCLUDELOGS'] Running command ['getVariable', 'BBINCLUDELOGS_LINES'] Running command ['getSetVariable', 'BB_CONSOLELOG'] Running command ['getSetVariable', 'BB_LOGCONFIG'] Running command ['getUIHandlerNum'] Running command ['setEventMask', XXXX] Running command ['getVariable', 'BB_DEFAULT_TASK'] Running command ['setConfig', 'cmd', 'build'] Running command ['getVariable', 'BBTARGETS'] Running command ['parseFiles'] --- Starting bitbake server pid 8252 at 2020-07-11 06:17:28.584514 --- Started bitbake server pid 8252 --- Starting bitbake server pid 13278 at 2020-07-11 06:17:31.330635 --- Started bitbake server pid 13278 Running command ['dataStoreConnectorCmd', 0, 'getVar', ('BBMULTICONFIG',), {}] Running command ['getRecipes', ''] Running command ['clientComplete'] Processing Client Disconnecting Client No timeout, exiting. Exiting where it looks like there are two server processes running which should not be. In that build there was a process left sitting in memory with its bitbake.sock file missing but holding the lock (not sure why it wouldn't timeout/exit). (Bitbake rev: e1a7c1821483031b224a1570bfe834da755219cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: pyshyacc: allow double COMMA statementsKonrad Weihmann2020-07-121-0/+1
| | | | | | | | | | this allows shell statements like '; ;' to pass the parser. As it may be bad code but still valid enough to execute (Bitbake rev: b7732b1b5085bea73e17d112e1bd9ac3d4dc34fb) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Change git fetcher not to destroy old referencesRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 820ab886e79eea516560c0c008e4cf059c6e11a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 01:33:45 +0000