summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: fetch2: Drop cups.org from wget status checksRichard Purdie2020-08-241-3/+1
| | | | | | | | | Its becomming clear the upstream server doesn't like this, drop these two urls from the test, not sure we need them here anyway. (Bitbake rev: ab2ef942dc21f9639793c972f2e546edf9444783) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Explictly shut down the sync threadRichard Purdie2020-08-241-1/+7
| | | | | | | | | | | | | | | | Hongxu Jia reported a problem where the bb_cache files were not always being written out correctly. This was due to the sync thread being terminated prematurely. Whilst the preceeding changes mean the exit handler for this thread is now correctly called since we switch to using sys.exit() instead of os._exit(), this write can happen after we drop the bitbake lock, leading to potential races. Avoid that headache by adding in explicit thread join() calls before we drop the lock (which atexit or Finalize can't do). (Bitbake rev: afd1900939f7b042297558f4cb01f50f3a299267) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure parse_quit thread is closed downRichard Purdie2020-08-241-6/+8
| | | | | | | | | | | Each run through the parser would leak a thread from the queue created to shut the parser down. Close this down correctly and clean up the code flow slightly whilst in the area, making sure this thread does shut down correctly (we don't care if it loses data). (Bitbake rev: 51ba35e9bbd4da8b5f3b3b5b8213cb634a6b0f06) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: Remove buffering parameterRichard Purdie2020-08-241-4/+1
| | | | | | | | | | The buffering parameter was removed in python 3.1 and made default so we can clean up the code. This removes weird looking double exceptions when connections fail. (Bitbake rev: 06b7bafbd18a47c8db2f7b943dc535c65df176bf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: clean_basepath: remove recipe full path when virtual:xyz ↵Jean-Francois Dagenais2020-08-221-1/+1
| | | | | | | | | | | | | | | | | | | | | | | present Before this fix, this example basepath (a): virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb:do_compile would get incorrectly "cleaned" into: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native:/full/path/to/recipes-example/helloworld/helloworld_1.2.3.bb When searching backwards in `a` trying to isolate the 'virtual:xyz' to add it to the end of the string, we need to consider `a` still has the recipe path and taskname. So stoping the rsplit after only 1 split is not enough. We want to reach the second ':' from the end. This way, we obtain: helloworld/helloworld_1.2.3.bb:do_compile:virtual:native reviewed-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com> (Bitbake rev: d193d93422a0ad62aa35b5d4ca5da8d422f72180) Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: make shell traps less chatty when 'bitbake -v' is useduninative-2.9Chris Laplante2020-08-211-2/+3
| | | | | | | | | | | | This shuts up both the DEBUG and EXIT handlers. Also, remove an unneeded "set -o errtrace" (i.e. set -E), since we no longer use an ERR trap. (Bitbake rev: 89e851fa0403d1e98aeed69990101e3f84f0b283) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace with the original metadata locations of ↵Chris Laplante2020-08-152-1/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bash shell funcs Leverage the comments that emit_var writes and the backtrace that the shell func writes to generate an additional metadata-relative backtrace. This will help the user troubleshoot shell funcs much more easily. Example: | WARNING: /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955:171 exit 1 from 'exit 1' | WARNING: Backtrace (BB generated script): | #1: myclass_do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 171 | #2: do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 166 | #3: actually_fail, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 153 | #4: my_compile_extra, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 155 | #5: do_compile, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 141 | #6: main, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 184 | | Backtrace (metadata-relative locations): | #1: myclass_do_something, /home/laplante/repos/oe-core/meta/classes/myclass.bbclass, line 2 | #2: do_something, autogenerated, line 2 | #3: actually_fail, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 36 | #4: my_compile_extra, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 38 | #5: do_compile, autogenerated, line 3 ERROR: Task (/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile) failed with exit code '1' NOTE: Tasks Summary: Attempted 542 tasks of which 541 didn't need to be rerun and 1 failed. Summary: 1 task failed: /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile Summary: There was 1 ERROR message shown, returning a non-zero exit code. (Bitbake rev: ae1aa4ea79826c32b20e1e7abdf77a15b601c6f2) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print a backtrace when a Bash shell function failsChris Laplante2020-08-151-11/+50
| | | | | | | | | | | | | The trick here is to use a DEBUG trap to record the line number of each command as they execute. This is so we can report the real line number of the failing command, which is otherwise not possible in a Bash EXIT trap. See http://gnu-bash.2382.n7.nabble.com/trap-echo-quot-trap-exit-on-LINENO-quot-EXIT-gt-wrong-linenumber-td3666.html (Bitbake rev: 8366e2ad5fdad3972766b40b23188908330067ee) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Add extra logfile flushingRichard Purdie2020-08-131-13/+22
| | | | | | | | | | | | There is the possibility data is being lost from the logfile due to data buffering. Add in a couple of extra flush calls to ensure data is being written out before the lock file is dropped. Possible fix for [YOCTO #14000]? (Bitbake rev: 596ea229a87d26a8e970c7ee77179519ad081fef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Pass timeout/xmlrpc parameters directly to the serverRichard Purdie2020-08-131-4/+5
| | | | | | | | Further cleanup, just pass these settings directly. (Bitbake rev: ac2284357f1fc7044dac9c146fad218fc9906412) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Simplfy idle callback handler functionRichard Purdie2020-08-134-14/+9
| | | | | | | | | | Having the idle callbacks abstracted via the configuration object makes no sense. Its like this for historical reasons from the multiple server backends but we don't need this now so simplfy. (Bitbake rev: e56c49717355c9493b07d5fc80981a95ad8a0ec8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Remove pointless process forkingRichard Purdie2020-08-131-3/+2
| | | | | | | | | | We already call bb.daemonize.createDaemon() in BitBakeServer so the extra multiprocessing.Process() appears to be totally unneeded and just an extra layer of forking which confuses things. Remove it. (Bitbake rev: d214e55c45f9733b3289138feec0ae3361a4a48b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: emit filename/lineno information for shell functionsChris Laplante2020-08-081-0/+6
| | | | | | | | | | | | | | | | | | Make it easier for users to debug shell task failure by including some breadcrumbs in the emitted .run file that (hopefully) points to the .bb/.bbclass file where the shell function was defined. Unfortunately this won't work with functions with _append or _prepends, since BitBake wipes the filename/lineno information. This shouldn't be too hard to fix; for now, you'll just see comments like this for such functions: [YOCTO #7877] (Bitbake rev: 9747211cbb45401cbf4dd0409e9c80c648a178c6) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/color: add test suite for ANSI color code filteringChris Laplante2020-08-082-0/+96
| | | | | | | | | Includes tests for bb.progress integration. (Bitbake rev: c472a8da521cc7f1d61ac2f28596167d47ab8a5a) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: filter ANSI escape codes before looking for progress textChris Laplante2020-08-081-3/+22
| | | | | | | | | This is in prepartion for introducing the log-colorizer bbclass into poky. (Bitbake rev: 889a873d71a6543efb71a0eb4ea6632c9f17175d) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: fix hypothetical NameError if 'progress' isn't setChris Laplante2020-08-081-3/+5
| | | | | | | (Bitbake rev: ff821022ef1fdf05482590d8e4fe003abf227135) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: progress: modernize syntax, formatChris Laplante2020-08-081-10/+17
| | | | | | | | | Also fixes DummyMultiStageProcessProgressReporter calling the wrong super __init__ (Bitbake rev: 7a1b4a7e4fffe54afe8d1d7e169ff558ad8c92d9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: create_progress_handler: fix calling 'get' on NoneTypeChris Laplante2020-07-291-1/+1
| | | | | | | | | Just use the |resolve| function which already takes care of it. (Bitbake rev: 91b809a0902ffd42be4edf7f0a7d25e6d354d822) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: print traceback if progress handler can't be createdChris Laplante2020-07-291-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ERROR: expat-native-2.2.9-r0 do_compile: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_compile.90289 Log data follows: | ERROR: 'NoneType' object has no attribute 'get' ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_compile) failed with exit code '1' After: ERROR: expat-native-2.2.9-r0 do_configure: Failed to create progress handler ERROR: expat-native-2.2.9-r0 do_configure: Traceback (most recent call last): File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 400, in exec_func_shell logfile = create_progress_handler(func, progress, logfile, d) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 344, in create_progress_handler cls_obj = functools.reduce(resolve, cls.split("."), bb.utils._context) File "/home/laplante/repos/oe-core/bitbake/lib/bb/build.py", line 343, in resolve return x.get(y) AttributeError: 'NoneType' object has no attribute 'get' ERROR: expat-native-2.2.9-r0 do_configure: 'NoneType' object has no attribute 'get' ERROR: Logfile of failure stored in: /home/laplante/repos/oe-core/build/tmp-glibc/work/x86_64-linux/expat-native/2.2.9-r0/temp/log.do_configure.22982 ERROR: Task (virtual:native:/home/laplante/repos/oe-core/meta/recipes-core/expat/expat_2.2.9.bb:do_configure) failed with exit code '1' (Bitbake rev: ba0472a672eacbbbb2295252a0d2056d3399c6a9) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Account for xmlrpc connectionsRichard Purdie2020-07-281-1/+1
| | | | | | | | | | UI control can happen via the xmlrpc connection. Account for this when timing out UI connections. This was causing issues for toaster on systems where it couldn't parse the metadata within the timeout. (Bitbake rev: fa85a8263971c25e67fa3b421c686a90e46acd87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix UI first connection trackingRichard Purdie2020-07-281-1/+3
| | | | | | | | | | We're only meant to be doing UI connection timeouts on the first connection but haveui changes for each connection. We need to add a specific variable to track this correctly and get the intended behaviour. (Bitbake rev: e7c387c2e2fb2cc3ca1dc9d2029362909c326d72) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: command: Handle multiconfig in findSigInfoJoshua Watt2020-07-251-3/+3
| | | | | | | | | | Changes the findSigInfo command to accept a recipe specified with the multiconfig prefix (Bitbake rev: 379951b6417eacbafc92ac1413ae1358bafdddfb) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Handle multiconfig name mappings correctlyRichard Purdie2020-07-231-1/+1
| | | | | | | | | | | | | | "bitbake mc:arm:bash mc:arm:busybox" works but "bitbake multiconfig:arm:bash multiconfig:arm:busybox" does not. The reason is the list is modified whilst iterating. Don't do that. [YOCTO #13607] (Bitbake rev: cd041a78d96e656438d93fb1e288080b8a6fe8bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Improve multiconfig configuration error reportingRichard Purdie2020-07-231-1/+6
| | | | | | | | | This avoids a traceback if an invalid multiconfig is referenced in the bitbake commandline and tweaks the message to make it more understanable. (Bitbake rev: f31d7d0ad57b0ecc2ae06ed4b547c98df2aaa1a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Allow deltask to take multiple tasknamesRichard Purdie2020-07-213-10/+11
| | | | | | | | | | | | | | | | | | | deltask currently supports only one task to delete but it would be useful if it could support a variable which gets expanded to allow flexibility in the metadata. This is simple to support in bitbake and is how other directives such as inherit operate so adjust the parser/code to handle that. It means that syntax like: EXTRA_NOPACKAGE_DELTASKS = "" deltask ${EXTRA_NOPACKAGE_DELTASKS} is now allowed. (Bitbake rev: 883d926120833c85a16dcf60425dd7af7699046a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Fix unmatched files handling leading to misleading warningsRichard Purdie2020-07-213-32/+52
| | | | | | | | | | | | | | | | | | | | | Currently if all recipes in a layer are skipped, there are warnings that the BBFILE_PATTERN_ entry didn't match anything. We probably shouldn't do this for skipped recipes. The current code is hard to understand, not least as it passes variables which functions modify by reference rather than giving a return value. Update calc_bbfile_priority() to return values rather than modifying them. Refactor the code to try and make it clearer what its doing and fix the skipped recipe issue by passing in the list of parsed files. The function is complicated by the need to not rerun regex matching more than we ever have to which complicates the flow, it would be easier if we just reran operations multiple times. (Bitbake rev: 969cb27b4d978551817612ff4558bec81cfb655c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix note reference -> infoRichard Purdie2020-07-211-1/+1
| | | | | | | | Its bb.note or logger.info, this avoids a backtrace. (Bitbake rev: 82c534ca1a1313de067b0d79c79857e89fa2764a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure UI-less servers don't sit in infinite loopsRichard Purdie2020-07-131-4/+15
| | | | | | | | | | | | | | | | | | | | If server startup is broken for some reason (e.g. lockfile issues) and no UI connection is made, the server will just sit inifinitely waiting. Add a timeout upon startup in the non-memory resident case so that such infinite waits are avoided. In the memory resident case, the server wouldn't have shut down in the first place or will timeout according to configuration. Since any race may mean the socket file is no longer present, ensure the unlink doesn't fault upon exit, thus ensuring any hashequiv or PRServ is removed from memory, allowing all processes to exit cleanly in such scenarios. (Bitbake rev: 39888b750df12478e8bdea6727cca112dce1df85) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Fix a rare lockfile raceRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're seeing rare occasional races on the autobuilder as if two server processes have the lockfile at the same time. We need to be extremely careful this does not happen. I think there is a potential race in this shutdown code since we delete the lockfile, then call unlockfile() which also tries to delete it. This means we may remove a lock file now held by another process if we're unlucky. Since unlockfile removes the lockfile when it can, just rely on that and remove any possible race window. An example cooker-deamonlog: --- Starting bitbake server pid 2266 at 2020-07-11 06:17:18.210777 --- Started bitbake server pid 2266 Entering server connection loop Accepting [<socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([]) Processing Client Connecting Client Running command ['setFeatures', [2]] Running command ['updateConfig', XXX] Running command ['getVariable', 'BBINCLUDELOGS'] Running command ['getVariable', 'BBINCLUDELOGS_LINES'] Running command ['getSetVariable', 'BB_CONSOLELOG'] Running command ['getSetVariable', 'BB_LOGCONFIG'] Running command ['getUIHandlerNum'] Running command ['setEventMask', XXXX] Running command ['getVariable', 'BB_DEFAULT_TASK'] Running command ['setConfig', 'cmd', 'build'] Running command ['getVariable', 'BBTARGETS'] Running command ['parseFiles'] --- Starting bitbake server pid 8252 at 2020-07-11 06:17:28.584514 --- Started bitbake server pid 8252 --- Starting bitbake server pid 13278 at 2020-07-11 06:17:31.330635 --- Started bitbake server pid 13278 Running command ['dataStoreConnectorCmd', 0, 'getVar', ('BBMULTICONFIG',), {}] Running command ['getRecipes', ''] Running command ['clientComplete'] Processing Client Disconnecting Client No timeout, exiting. Exiting where it looks like there are two server processes running which should not be. In that build there was a process left sitting in memory with its bitbake.sock file missing but holding the lock (not sure why it wouldn't timeout/exit). (Bitbake rev: e1a7c1821483031b224a1570bfe834da755219cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: pyshyacc: allow double COMMA statementsKonrad Weihmann2020-07-121-0/+1
| | | | | | | | | | this allows shell statements like '; ;' to pass the parser. As it may be bad code but still valid enough to execute (Bitbake rev: b7732b1b5085bea73e17d112e1bd9ac3d4dc34fb) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Change git fetcher not to destroy old referencesRichard Purdie2020-07-121-1/+1
| | | | | | | | | | | | | | | | It looks like we're about to see a lot of changes in branch names in repos. If we have the prune option here, those old names are lost, the changes propagate to our source mirrors and our old releases break. We have the force option so any replaced references should be replaced, its only orphaned branches which will now be preserved. I believe this behaviour will cause us fewer problems given the changes that look likely to happen. (Bitbake rev: 820ab886e79eea516560c0c008e4cf059c6e11a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Increase timeout for commandsRichard Purdie2020-07-081-1/+3
| | | | | | | | | | We're running into this timeout on loaded autobuilders in situations where things should otherwise succeed. Log a note in these cases and continue to try for longer. (Bitbake rev: e567743e70f426786ae54dcb5ab550748d9266e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ui/teamcity: don't use removed logging classesChris Laplante2020-07-081-2/+0
| | | | | | | | | Allows the TeamCity frontend to be used again. (Bitbake rev: c5477ba79fcad4a887808dd0df9cfe3554e2c17a) Signed-off-by: Chris Laplante <mostthingsweb@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: contrib/vim: synchronize from kergoth/vim-bitbake rev ↵Chris Laplante2020-07-086-17/+96
| | | | | | | | | 4225ee8b4818d7e4696520567216a3a031c26f7d (Bitbake rev: 24fb1f2fee449589b0c5468cbdebe9a6ffaac932) Signed-off-by: Chris Laplante <mostthingsweb@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/ui/taskexp: Fix missing Gtk importDavid Khouya2020-07-071-0/+1
| | | | | | | | | | | Adding back gtk objects import. Fix bug introduce when adding validation on gtk import. (Bitbake rev: 765be5ef60668f8a1cfbcba248f4995725807196) Signed-off-by: David Khouya <dakhouya@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/ui/taskexp: Validate gi importDavid Khouya2020-07-061-3/+9
| | | | | | | | | | | | | When running bitbake -g -u taskexp without having gi python module or and invalid gtk version, bitbake fails with a stack trace. In case of import or version error, bitbake should exit with an error message instead of a stack trace. (Bitbake rev: 2a2c507f239b047f34765312df4168030e38b90d) Signed-off-by: David Khouya <dakhouya@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: msg: Avoid issues where paths have relative componentsRichard Purdie2020-06-281-1/+1
| | | | | | | | | | The autobuilder can end up using build/../ syntax which is an issue if the build directory is cleaned. Avoid this by using normpath() on the file path passed in. (Bitbake rev: 41988fec47eb196ab7195a75330a6d98de19101b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Avoid unpickle errors in rare casesRichard Purdie2020-06-281-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | | In rare cases the pickled data from a task contains "</event>" which causes backtrace. This can be reproduced with something like: do_unpack_prepend () { bb.warn("</event>") } There are several solutions but the easiest is to catch this exception and look for the next marker instead as this should be the only way such an unpickle error could occur. This fixes rare exceptions seen on the autobuilder. Also add in other potential exceptions listed in the pickle manual page so that better debug is obtained should there be an error in this code path in future. exitcode doesn't need the same handling since we control what is in that data field and it could never contain </exitcode> (Bitbake rev: 5ada512d6f9cbbdf1172ff7818117c38b12225ca) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Fix error when hash equivalence has an exceptionJoshua Watt2020-06-281-0/+1
| | | | | | | | | | The code that handled exceptions from the hash equivalence client was raising an exception itself because hashserv.client wasn't imported (Bitbake rev: a76290dfc6f34ff9f6efdb13a6db74b6b4759daf) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Chunkify large messagesJoshua Watt2020-06-284-41/+152
| | | | | | | | | | | | | | | | The hash equivalence client and server can occasionally send messages that are too large for the server to fit in the receive buffer (64 KB). To prevent this, support is added to the protocol to "chunkify" the stream and break it up into manageable pieces that the server can each side can back together. Ideally, this would be negotiated by the client and server, but it's currently hard coded to 32 KB to prevent the round-trip delay. (Bitbake rev: e27a28c1e40e886ee68ba4b99b537ffc9c3577d4) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: test/fetch: change to better svn sourceakuster2020-06-281-1/+1
| | | | | | | | | | | | | | | fixes: svn: warning: W175002: Unexpected HTTP status 504 'Gateway Timeout' on '/openembedded/bitbake/!svn/vcc/default' svn: E205011: Failure occurred processing one or more externals definitions picked pcre2 [Yocto #13948] (Bitbake rev: 1483d17108da02f5d615e83403d5fd6288ca957c) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: taskdata: Improve handling of regex in ASSUME_PROVIDEDRichard Purdie2020-06-281-2/+7
| | | | | | | | | | | | | | | | | | ASSUME_PROVIDED can take regexs however the current way of handling this in code is suboptimal. It means that you can add something like: DEPENDS += "texinfo-nativejunk-that-does-not-exist" and if texinfo-native is in ASSUME_PROVIDED, no error will occur. Update the code to only treat something as a regex if a start or end anchor character is present (which wouldn't be valid in a recipe name). [YOCTO #13893] (Bitbake rev: 3d72e23109990970fbb1086923277af752168b4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: update perforce fetcher docsAlexandru N. Onea2020-06-251-0/+60
| | | | | | | | | | | | | | This change updates the perforce documentation by describing two new parameters: module and remotepath. Additionally, a general statement regarding the fetcher implementation has been added, to make it clear that the fetcher does not use a perforce client for the job. (Bitbake rev: 44b57216548fa96a5ecab02cfed517e0d631dc44) Signed-off-by: Alexandru N. Onea <onea.alex@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: perforce: add local path handling SRC_URI optionsAlexandru N. Onea2020-06-251-2/+45
| | | | | | | | | | | | | | | | | | | | This patch implements three new SRC_URI options for the perforce fetcher, namely: * module * remotepath The options are intended to provide the user more control over the downloaded file paths by allowing the user to specify how much of the remote path should be preserved locally. The changes in this patch are backwards compatible, i.e. if none of the introduced options is specified, the default (old) behavior is enforced. (Bitbake rev: aab228822d2f221c01337dd57d7582c51ce9a505) Signed-off-by: Alexandru N. Onea <onea.alex@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: perforce: add basic progress handler for perforceAlexandru N. Onea2020-06-251-1/+33
| | | | | | | | | | | | This patch adds a basic implementation of a progress handler for the perforce fetcher, based on the number of files to be downloaded and the output behavior of the p4 print command used in the fetcher implementation. (Bitbake rev: f0582292bf79b0988048683dfd086aa3b9787344) Signed-off-by: Alexandru N. Onea <onea.alex@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: Add BBFILES_DYNAMICKonrad Weihmann2020-06-231-0/+61
| | | | | | | | | | - add missing entry for BBFILES_DYNAMIC, ported from yocto-docs - add description for the new inverse mode (Bitbake rev: 9186ca47ce73b4d1c87eb69163698a04679fb55c) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Add BBFILES_DYNAMIC inverse modeKonrad Weihmann2020-06-231-2/+5
| | | | | | | | | | | | Add an inverse mode for BBFILES_DYNAMIC. Entries where the layername is prefix with '!' will act in inverted mode including given glob mask in absence of the referenced layer. This is useful to backfill recipes from potentially missing layer. (Bitbake rev: bfd729cfe06351ccd0071004e23af1d04546c723) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: Switch from git.infradead.org to a YP mirrorRichard Purdie2020-06-221-1/+2
| | | | | | | | | | Upstream is unavailable, breaking tests. Switch to a YP mirror since if we can't reach that there are bigger problems. This should remove a source of intermittent failures on the autobuilder. (Bitbake rev: 232471083d6c574c7ada6320f9379ad7d7862a9c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache: Bump cache versionJoshua Watt2020-06-191-1/+1
| | | | | | | | | | | | | | | | | | The layout of the cache data hasn't changed, but the cache has now been split into different files for multiconfig. If a user pulls in these changes, it's possible that their base cache will still contain the combined multiconfig cache entries, which are now unexpected and generate errors like: Unexpected multiconfig: foo Bumping the version fixes this since the old cache data won't be considered valid anymore. (Bitbake rev: 1082188ce633ec6891c961d726f584b3f1259941) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: Remove TERM from BB_HASHBASE_WHITELIST exampleJacob Kroon2020-06-171-1/+1
| | | | | | | | | | TERM is no longer included in OE-Core's default BB_HASHBASE_WHITELIST, so remove it. (Bitbake rev: 7689fa781646039524a655295a884e525ad8732c) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-01-15 10:24:26 +0000