summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: runqueue: Further fixes for confused setscene tasksRichard Purdie2021-04-061-13/+26
| | | | | | | | | | | | | | | There is further evidence of tasks ending up being "covered" and "notcovered" which shouldn't happen and is bad. The code that caused this problem last time appears to have issues where stamps for tasks already exist. Split out the setscene stamp checking code to a separate function and use this when checking "hard dependencies" (like pseudo-native) so that if the stamps exist and it will be "covered", it is not put on the notcovered list. (Bitbake rev: a1848a481e36b729c8e4130c394b1d462d4b488a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: Update links to documentationMichael Opdenacker2021-03-232-2/+2
| | | | | | | (Bitbake rev: ed8e1fd4cf9d5ac8a8203638add99d686b4b3521) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/event: Add an event for notifying of stale setscene tasksRichard Purdie2021-03-232-0/+25
| | | | | | | | | | | Use the new functionality in build.py to identify stale setscene tasks and send an event to the metadata listing them. The metadata then has the option of performing cleanup operations if it thinks that appropriate. (Bitbake rev: ef8c980a3ae92c168b7ca16a4d19cd38a9574761) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add find_stale_stamps functionRichard Purdie2021-03-231-0/+17
| | | | | | | | | | Add a new function which compares the stamp filename we want (including taskhash) with what is in the stamp directory (using the clean mask). This tells us which stamp files are stale and are due to be rerun. (Bitbake rev: b126a53882d202e4df0f9661303355c9fe9ec80e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: fix syntax error in layer.conf exampleRobert P. J. Day2021-03-221-1/+2
| | | | | | | | | | | While this example really needs to be rewritten to not define multiple patterns in the same layer.conf, as long as it's there, it might as well be syntactically correct. (Bitbake rev: 1c2d2875099a3ff9149710d42c679ab31b00f68b) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: fix glossary link for BB_INVALIDCONF variableRobert P. J. Day2021-03-221-1/+1
| | | | | | | (Bitbake rev: 4a4d55c41806815c32add32863a05351a6df30f7) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: mention that addtask handles multiple dependenciesRobert P. J. Day2021-03-221-0/+11
| | | | | | | | | | Add a note explaining that "addtask" can accept multiple dependencies, just in case someone runs across such an example and is confused. (Bitbake rev: 5e4e9acd323b5ebc3a14c07384f6cf8f8a272066) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: doc: move BBFILES_DYNAMIC for alphabetical orderRobert P. J. Day2021-03-151-39/+39
| | | | | | | | | | Since BBFILES_DYNAMIC does not have a "BB_" prefix, it belongs further down in the variable glossary. (Bitbake rev: 493f291601c7680f8b8146f358c545fcb5f53cc9) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: Fix TypeError in bitbake debug modeTomasz Dziendzielski2021-03-151-3/+3
| | | | | | | | | | | | | | | | | Commit 75f87db413 fixed the confusion between bitbake and python logger but in codeparser still old method of setting debug level was used causing TypeError, because debug level value was incorrectly returned and assigned to event.msg. | File "./bitbake/lib/bb/ui/knotty.py", line 660, in main | event.msg = event.fn + ': ' + event.msg | TypeError: can only concatenate str (not "int") to str [YOCTO #14298] (Bitbake rev: bec9ea7ab28a8dfad1a6010ca5c6ec691754748d) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bblayers/query: Update to handle REQUIRED_VERSIONRichard Purdie2021-03-151-1/+1
| | | | | | | | | | | Ii is unclear whether any changes are needed to bblayers are needed to handle the extra data from REQUIRED_VERSION. Update to at match the new API, at present it doesn't look necessary to handle the required version data. (Bitbake rev: 53c30efec4099035d19e6717059dfceff8ff88fd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker: Add REQUIRED_VERSION checksCharlie Davies2021-03-151-2/+8
| | | | | | | | | | | | | Add logic to check that if REQUIRED_VERSION has been set that the cooker class method findBestProviders properly handles the case where the REQUIRED_VERSION has not been found. Fixes [YOCTO #10096] (Bitbake rev: 5df201d746f26154213e6d15d2721cd35b38b59e) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: lib: add Required Version to Bitbake --show-versions commandCharlie Davies2021-03-152-14/+24
| | | | | | | | | | | | | This commit adds a new column to the Bitbake --show-versions command called Required Version. This column will display any packages which have a REQUIRED_VERSION successfully set. Fixes [YOCTO #10096] (Bitbake rev: 90c7d1815e41243323d32b9dbb865757a922578a) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: check for REQUIRED_VERSION in _filterProvidersCharlie Davies2021-03-151-2/+5
| | | | | | | | | | | | | | | | | | Before the REQUIRED_VERSION variable was introduced the PREFERRED_VERSION variable allowed for a fallback to the next most suitable version. Since REQUIRED_VERSION does not allow a fallback to a different version implement a check in the _filterProviders function to make sure that if a requested REQUIRED_VERSION is not found then the function returns no eligible providers have been found. Fixes [YOCTO #10096] (Bitbake rev: c41386b78aa53e0bf081cd973c950b88126670a7) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: introduce logic for REQUIRED_VERSION variableCharlie Davies2021-03-151-15/+41
| | | | | | | | | | | | | | | | | | This commit adds checks during the findPreferredProvider function for a new variable REQUIRED_VERSION. This can be set, in exactly the same manner as PREFERRED_VERSION, on a per package basis to enforce the use of a particular version of a package. REQUIRED_VERSION is similar in behaviour to PREFERRED_VERSION except if the version specified by REQUIRED_VERSION is not found an error occurs and the execution of Bitbake stops. Fixes [YOCTO #10096] (Bitbake rev: 5cbf6d95fc1009e78e7d0745a49e0bf418b37abb) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: consistent single line formatting between functionsCharlie Davies2021-03-151-6/+0
| | | | | | | (Bitbake rev: e8c74ac8e589ee44d862b71fd63dd75ab303d3b5) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: remove unneeded logging callCharlie Davies2021-03-151-1/+0
| | | | | | | | | | | | The logging line for when no eligible providers are found only adds unnecessary noise and is not required. As the stack unwinds better logging occurs which satisfactorily describes the no eligible providers error condition. (Bitbake rev: 933c9df919f7947f515d843199cce2621cf8195f) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/color: Fix event register to pass the datastoreRichard Purdie2021-03-121-1/+1
| | | | | | | | | | | | | | Adding the "if d is None" to the event handling code meant some of these tests stopped working. The reason is that len(d) was zero but not equal to None. Passing the data object to the register() function in the test correctly registers the event handler and avoids the problem, it just happened to work previously, incorrectly. (Bitbake rev: 426eb83c6668d82a2ebaca6c672db131e37c11da) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Fix another 'if d' test to test for None explictlyRichard Purdie2021-03-111-1/+1
| | | | | | | | This should have been fixed in the previous patch, well spotted Chris! (Bitbake rev: c3f6fee42bfa23f23f167cb29f0cfa05ac2fa197) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Fix multiconfig event handler change performance regressionsRichard Purdie2021-03-111-11/+11
| | | | | | | | | | | | | | There were two issues in this code, firstly the code could stack duplicates in the variable, secondly, calling "if data" caused the datastore to compute len(data) which is comparitively expensive. Checking "if data is not None" is much much faster/cheaper. The issue was clear from "bitbake -p -P" output where the time in register() showed large amounts of time in the __len__ function of the datastore. (Bitbake rev: 9298d08b7dcf9d79f54536f87686d65c4ad7deb9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: Honor quiet when parsing recipesManuel Leonhardt2021-03-111-1/+1
| | | | | | | | | | | When using parse_recipes, honor quiet so that scripts and custom plugins for recipetool are able to mute progress bars from bitbake that would otherwise print to STDOUT. (Bitbake rev: ab9d26c4847a062cadaae5fb8caac0ead5f958db) Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Print pseudo.log if fakeroot task failedTomasz Dziendzielski2021-03-112-9/+40
| | | | | | | | | | | | | | | | | | | | | | | | Currently if pseudo fails we can only see the path to pseudo.log. If we have no access to server and can only rely on bitbake log then debugging becomes impossible. This printing needs to be added in runqueue level, not inside task execution, because in some cases task fails with pseudo abort really early and we don't even see any log. In this change I'm adding pseudo log printing in every fakeroot task failure that logged `mismatch`, `error` or `fatal` to logfile, because we have no other way to communicate with pseudo if it failed or not. Only lines from last pseudo server execution will be printed. (Bitbake rev: e7c664a947903ed7b868abef62af2ff5f8ef0dc6) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com> Signed-off-by: Adrian Walag Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Mikolaj Lasota <mikolaj.lasota@protonmail.com> Signed-off-by: Wiktor Baura <wbaura@gmail.com> Signed-off-by: Kamil Kwiek <kamil.kwiek@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event.py: fix regression about INVALIDCONFChen Qi2021-03-111-0/+4
| | | | | | | | | | | | | A previous commit uses __BBHANDLERS_MC to record all handlers, but it does not take into consideration of INVALIDCONF, thus causing regression. We need to record the name before returned AlreadyRegistered, otherwise, when reparsing due to INVALIDCONF, bascially all handlers are not called. (Bitbake rev: f6fb6f22fa6ba1ffe121948d7f53ac724358491e) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: docs: Add AZ_SAS definition to glossaryAlejandro Hernandez Samaniego2021-03-111-0/+13
| | | | | | | (Bitbake rev: 134df841b273d94e767426876459da348d96dd23) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: docs: Add Az fetcher documentationAlejandro Hernandez Samaniego2021-03-112-0/+30
| | | | | | | (Bitbake rev: e3b480636a3c2716effd619b59cf55e11f9a6db0) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add Azure Storage fetcher implementationAlejandro Hernandez Samaniego2021-03-112-1/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allows bitbake to fetch from an Azure Storage account. The fetcher submodule is compatible with the az:// URI protocol, its functionality is based on bitbakes wget fetcher, superior in performance to using a propietary tool like azcopy which can handle cloud storage account operations with more functionality (that we dont need in a fetcher) but less compatibility. A sample URI uses can be defined in the following way: SRC_URI = "az://<azure-storage-account>.blob.core.windows.net/<container>/foo.tar.xz" This fetcher can easily be used with PREMIRRORS and SSTATE_MIRRORS, e.g.: SSTATE_MIRRORS = "file://.* az://<azure-storage-account>.blob.core.windows.net/sstate-cache/PATH;downloadfilename=PATH \n" PREMIRRORS_prepend = "\ git://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ ftp://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ http://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ https://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ " Can also be used with non-public access Azure Storage accounts/containers via a Shared Access Signature by declaring the AZ_SAS variable which will be automatically used by the fetcher: AZ_SAS="?sv=2000-01-01&ss=...&sig=somesignature" (Bitbake rev: b103b02f2ce2f8f5079f17ec1a854f904c2110a4) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: __init__.py: Fix bitbake debug log handlingRichard Purdie2021-03-091-1/+1
| | | | | | | | | | | | For a while I've been puzzled as to why debug logging from runqueue wouldn't appear on the console with -DD. The logic in the bbdebug handling is inverted so fix it and now we see the expected messages from runqueue with -D and -DD. This should then let us debug other issues using those log messages. (Bitbake rev: 34145b950be03aff8f9b88207cf843abf002ab13) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Add setscene task overlap sanity checkRichard Purdie2021-03-091-0/+4
| | | | | | | | | | We've seen hard to debug issues where a task ends up in both the covered and notcovered list. Add a sanity check to ensure if this happens in future, we see it in the logs. (Bitbake rev: 6e001410854792f9bb66a0409a2ac176171b0507) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix task execution corruption issueRichard Purdie2021-03-091-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | We've seen occasional issues where linux-yocto:do_compile_kernelmodules would run without do_shared_workdir running before it. do_shared_workdir is an setscene task but never has an sstate object generated so it will always rerun. This should not happen since compile_kernemodules should only execute if a setscene that depends on it didn't run and that should trigger do_shared_workdir not to be marked as covered. The issue is that build-appliance-image:do_package is one of the tasks which covers linux-yocto:do_compile_kernelmodules but it is also a noexec task and has a dependecy on pseudo-native:do_populate_sysroot. In the problem case, pseudo-native:do_populate_sysroot is unavailable but marked as covered since it is noexec. The "harddeps" code then also marks it as notcovered. No task should ever be both covered and notcovered and this is where the problems come from. The solution is for the harddeps code only to to fail tasks if they've not already been handled in some way. The code is assuming code couldn't have handled revdeps at this point but we now have clear evidence they can. (Bitbake rev: f66556bbb38449789ceea2fd105e9f68df7fb660) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget: Avoid crashing when connection drops mid checkstatusAlejandro Hernandez Samaniego2021-02-261-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | If an exception is raised when running host python code, the fetcher immediately crashes, this might be temporary depending on the servers reliability. Catch the exception when the connection was reset and try once again to fetch the data. File: '/usr/lib/python3.8/socket.py', lineno: 669, function: readinto 0665: if self._timeout_occurred: 0666: raise OSError("cannot read from timed out object") 0667: while True: 0668: try: *** 0669: return self._sock.recv_into(b) 0670: except timeout: 0671: self._timeout_occurred = True 0672: raise 0673: except error as e: Exception: ConnectionResetError: [Errno 104] Connection reset by peer (Bitbake rev: d0f5c5905bc664e415a05e3130dfe0ae541d8b3e) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Fix broken builds when multiconfig has a hyphen in the nameJoshua Watt2021-02-217-26/+26
| | | | | | | | | | | | | | | 5f7fdf7b2d ("bitbake: event: Prevent bitbake from executing event handler for wrong multiconfig target") broke multiconfig builds contain a hyphen, since it's attempt to use the multiconfig as part of a function name and python functions are not allowed to contain a hyphen. Rework the bitbake multiconfig test to test a multiconfig with a hyphen and one with an underscore to validate this doesn't break in the future. (Bitbake rev: c3168df330a4563cbd03ba74de55a22217d823ed) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Force parser shutdown after catching an exceptionJan Brzezanski2021-02-211-5/+5
| | | | | | | | | | | | | | | | Commit bebef58b21bdff7a3ee1fa2449b7df19144f26fd introduced forcing parser shutdown as default in case of build abort. In this case bitbake sometimes hangs after facing error during parsing, waiting for child processes to finish. Killing it then will spawn zombie processes. Thus we force the shutdown after catching an exception. (Bitbake rev: 915330e1dbae1ee8fd9a0358decf2c294f771961) Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: use pythonic empty list checkCharlie Davies2021-02-211-1/+1
| | | | | | | (Bitbake rev: 591542734988856afa53b99e5eef6e5ca15630b8) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: providers: fix incorrect return type bugCharlie Davies2021-02-211-1/+1
| | | | | | | | | | | | | | | | | | If no eligible providers are found then an integer zero is returned. This causes the following error, in two possible places in taskdata.py, when the return value is used in a list comprehension: [snip] eligible = [p for p in eligible if not p in self.failed_fns] TypeError: 'int' object is not iterable [\snip] Fix by returning the variable eligible itself, of type list. (Bitbake rev: 217c4b436b588a6a47aeaddf61531711ad3fca67) Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: contrib: Add Dockerfile for building hash serverJoshua Watt2021-02-161-0/+19
| | | | | | | | | | Adds a Dockerfile for build the reference hash equivalence server in a container (Bitbake rev: d9a0a88db5888039bfbb6ce5129b43350f79b1c1) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump version to 1.49.2Richard Purdie2021-02-162-2/+2
| | | | | | | | This allows metadata to rely upon BB_DEFAULT_UMASK. (Bitbake rev: 969ac64adab236ce2d5196bcc294005a497913ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker/runqueue: Add support for BB_DEFAULT_UMASKRichard Purdie2021-02-163-2/+11
| | | | | | | | | | | | | Currently each task has to have a umask specified individually. This is leading to determinism issues since it is easy to miss specifying this for an extra task. Add support for specifing the default task umask globally which simplifies the problem. (Bitbake rev: 3e664599fd54a8a37ce587022fcbce5ca26f2ed3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Prevent bitbake from executing event handler for wrong ↵Tomasz Dziendzielski2021-02-164-6/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | multiconfig target When multiconfig is used bitbake might try to run events that don't exist for specific mc target. In cooker.py we pass `self.databuilder.mcdata[mc]` data that contains names of events' handlers per mc target, but fire_class_handlers uses global _handlers variable that is created during parsing of all the targets. This leads to a problem where bitbake runs event handler that don't exist for a target or even overrides them - if multiple targets use event handler with the same name but different code then only one version will be executed for all targets. See [YOCTO #13071] for detailed bug information. Add mc target name as a prefix to event handler name so there won't be two different handlers with the same name. Add internal __BBHANDLERS_MC variable to have the handlers lists per machine. (Bitbake rev: 5f7fdf7b2d8c59805c8ef4dae84f536baa5e172b) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Bump release to 1.49.1Richard Purdie2021-02-102-2/+2
| | | | | | (Bitbake rev: 9f23fa605c542a705d00c6c263491899d55bb0d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: logging: Make bitbake logger compatible with python loggerJoshua Watt2021-02-1034-228/+237
| | | | | | | | | | | | | | | | | The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bblayers/action: When adding layers, catch BBHandledExceptionRichard Purdie2021-02-101-2/+2
| | | | | | | | | | | When adding a layer, parse error can occur, raising BBHandledException. Catch this and error, aborting the layer add to meet user expectations. [YOCTO #14054] (Bitbake rev: ceddb5b3d229b83c172656053cd29aeb521fcce0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure reparsing is handled correctlyRichard Purdie2021-02-101-0/+2
| | | | | | | | | | | | | | | From tinfoil, if you edit bblayers.conf and break it, then call parseConfiguration (e.g. by adding a bad layer with bitbake-layers), the system doens't show any parse error yet it should. Add in a call to the updateCache function so that things really are reparsed when requested. Partially fixes [YOCTO #14054] (Bitbake rev: e655f9361b9c3b77906b8e06b5cc76bc5180640e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Don't classify shell functions that names start with ↵Tomasz Dziendzielski2021-02-101-1/+1
| | | | | | | | | | | | | | | | | | "python*" as python function If shell function name starts with 'python' or 'fakeroot' parser wrongly assumes it's python/fakeroot function. [YOCTO #14204] Use regex lookahead assertions to check if 'python' expression is followed by whitespace or '(' and if 'fakeroot' is followed by whitespace. (Bitbake rev: b07b226d5d1b3acd3f76d8365bc8002293365999) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add get-outhash messagePaul Barker2021-02-102-14/+38
| | | | | | | | | | | | | | | The get-outhash message can be sent via the get_outhash client method. This works in a similar way to the get message but looks up a db entry by outhash rather than by taskhash. It is intended to be used as a read-only form of the report message. As both handle_get_outhash and handle_report use the same query string we can factor this out. (Bitbake rev: dc19606ada29a4d8afde4fcecd8ec986b47b867e) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: server: Support searching upstream for outhashPaul Barker2021-02-102-0/+32
| | | | | | | | | | | | | Use the new get-outhash message to perform a read-only query against an upstream server (if present) when a reported taskhash/outhash combination is not found in the current database. If a matching entry is found upstream it is copied into the current database so it can be found by future queries. (Bitbake rev: 2be4f7f0d2ccb09917398289e8140e1467e84bb2) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Add short forms of remaining command line argumentsPaul Barker2021-02-101-3/+3
| | | | | | | | | Short form arguments are added for convenience. (Bitbake rev: 921199a4923ce383b27e23c9b7e34eb785c8bae3) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Support upstream command line argumentPaul Barker2021-02-101-1/+2
| | | | | | | | | | | The hashserv server already implements support for pulling hash data from another "upstream" server. Add the -u/--upstream argument to the bitbake-hashserv app to expose this functionality to users. (Bitbake rev: 8de510f1de35e581bcd5858edf23619c6a4cf923) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: Support read-only serverPaul Barker2021-02-104-12/+53
| | | | | | | | | | | | | | | | | | The -r/--readonly argument is added to the bitbake-hashserv app. If this argument is given then clients may only perform read operations against the server. The read-only mode is implemented by simply not installing handlers for write operations, this keeps the permission model simple and reduces the risk of accidentally allowing write operations. As a sqlite database can be safely opened by multiple processes in parallel, it's possible to start two hashserv instances against a single database if you wish to export both a read-only port and a read-write port. (Bitbake rev: 492bb02eb0e071c792407ac3113f92492da1a9cc) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: Try and avoid potential short write events issuesRichard Purdie2021-02-061-2/+6
| | | | | | | | | | | | | | | | | We're seeing occasional issues where builds fail as events were written from the worker children in the form <event>partial data<event>full event</event>. This causes failures as bitbake server can't parse that and exits. This could be due to short writes to the worker event pipe which we weren't checking. Check this and loop accordingly. Also add some asserts to detect other potential causes. Thanks to Joshua Watt for help in spotting the issue. [YOCTO #14181] (Bitbake rev: a9451746a4bd7ccedf4c72cd03ad4ff0ab0143aa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hashserv: client: Fix handling of null responsesPaul Barker2021-02-061-1/+1
| | | | | | | | | | If the server returns an empty response ("null" in json), this cannot be iterated to check for the presence of the "chunk-stream" key. (Bitbake rev: bf75370bcd6d02ed08cd959eec6190196b792515) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-hashclient: Remove obsolete call to client.connectPaul Barker2021-02-061-3/+0
| | | | | | | | | | | | The connect function was previously removed from the hashserv Client class but the bitbake-hashclient app was not updated. The client is connected during hashserv.create_client() anyway so not separate connect call is needed. (Bitbake rev: 99bdb236bceeffa0083a0fa529280b217c1d310d) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-20 12:11:36 +0000