summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/parse
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: parse/ast: add support for 'built-in' fragmentsAlexander Kanavin2025-06-202-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When reviewing proposed fragments to add settings for DISTRO and MACHINE, RP noted that such fragments only add clutter and overhead, and there's no need to maintain them as separate files. Rather when bitbake sees 'fragmentvar/fragmentvalue' it can expand that into FRAGMENTVAR = "fragmentvalue". To achieve that, 'addfragments' directive is extended with a parameter that sets the name of the variable that holds definitions of such built-in fragments, for example like this: "machine:MACHINE distro:DISTRO" Then each enabled fragment name is matched against these definitions and the respective variable is set, e.g. 'machine/qemuarm' would match 'machine:MACHINE' and result in MACHINE set to 'qemuarm'. This happens before any fragment files are looked up on disk, and no such lookup happens if there was a match, which should prevent possible misuse of the feature. So the builtin fragment definition is also an allowlist for them. Please also see the patches for oe-core that show an application of the feature. (Bitbake rev: 3b9d7bea915dc7f10e845854f1dae325743f9456) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/BBHandler: Add support for BB_DEFER_BBCLASSESRichard Purdie2025-06-162-6/+12
| | | | | | | | | Add support for automatically promoting class inherits to deferred inherits by listing them in the BB_DEFER_BBCLASSES variable. (Bitbake rev: 8e741b2e885a12d119788d04aa4efcd724dd6bfa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Add event for deferred inheritsRichard Purdie2025-06-161-0/+3
| | | | | | | | | | | | | | | Now that deferred inherits are extension specific, we can pass this list to an event, which our metadata can use to set class overrides earlier (as an example). There are limitations to this, the list of classes is unexpanded and recursive classes are not visible. There isn't much that can be done about this, the ones we are interested in would usually be visible at the top level (such as class extensions). (Bitbake rev: 205d461c05fc7b4a7c81039af3bc3fd71cbb982c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast: Change deferred inherits to happen per recipeRichard Purdie2025-06-161-9/+11
| | | | | | | | | | | | | | | | | | | | | | | | Currently deferred inherits get processed once for all class extensions as a minor speed optimisation. Unfortunately this limits our options for being able to report deferred classes to our code. There are two challenges with using our deferred classes in OE at present. One is that PACKAGECONFIG values don't work well with class overrides like class-native if there are deferred classes based on PACKAGECONFIG, such as python support. The second is that toolchain selection is proving problematic to implement due to interactions between the toolchain deferred inherit, the class extensions and class overrides being very late. By changing deferred inherits to be recipe extension specific, we open the way to generate events and "peek" at where things will end up, allowing the class overrides to be set earlier. The class extension code is updated to use a deferred inherit for the class extension inheriting so that it is still inherited last. (Bitbake rev: 29277cf4d88eb4dfa9572851177d009eab5afd0c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ConfHandler: Add warning for deprecated whitespace usageRichard Purdie2025-04-161-2/+4
| | | | | | | | | | | | | | | A lack of whitespace around variable assignment operators makes the files harder to read. There is a deeper issue in that a "+" character can sometimes be confused between the variable name and the assignment operator. Start showing warnings for such usage so we encourage people to use consistent whitespace which helps with file readability in general. (Bitbake rev: 24772dd2ae6c0cd11540a260f15065f906fb0997) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: codeparser: Add function decorators for vardepsJoshua Watt2025-04-101-0/+37
| | | | | | | | | | | | Adds bb.parse.vardeps bb.parse.excludevardeps function decorators that can be used to explicitly add or exclude variables from a python function parsed by bitbake (Bitbake rev: 030fb3dee067640a3a50f24a53d200bdb5048376) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Forbid ambiguous assignments to ${.}, ${+}, and ${:} variablesNikolai Merinov2025-02-051-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Old code that parse variable names in assignment commands behave differently for variables that ends with special symbol for single-character variable names and multi-character variable names. For example: A+="1" # Change variable ${A}, '+' glued to '=' A+ = "1" # Change variable ${A+} +="1" # Change variable ${+}, the '+' symbol not part of assignment operator + = "1" # Change variable ${+} New code would always assume that '.=', '+=', and ':=' is assignment operator. As result code like the following would raise parsing error +="value" While code with extra spaces would work as before + = "value" # Change variable ${+} This change allow to catch issues in code that generate bitbake configuration files in a manner like "echo ${VARNAME}+=${VALUE} >> conf/local.conf" (Bitbake rev: 93059aad13a12cd69d86368795c88e5349197d5d) Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: Add support for BB_RECIPE_VIRTUAL_PROVIDERSRichard Purdie2025-01-211-0/+25
| | | | | | | | | | | | | Currently, providers are set on a global config basis. This change allows for a select set of providers configured in BB_RECIPE_VIRTUAL_PROVIDERS to be selected on a per recipe basis. This would allow for the selection of virtual/cross-cc as gcc or clang for example in OE-Core. DEPENDS and task flag [depends] values are processed. (Bitbake rev: fb119c7888ae8a749aa824f8c51780176af077f9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: fix ??= operator for variable flagsYoann Congal2025-01-141-1/+4
| | | | | | | | | | | | | | | | | ??= operator works for variable value by defining a "_defaultval" flag. Use something similar for flags: For the default value of the flag "flag_name", define a flag "_defaultval_flag_flagname" that is used when reading flag_name but no other value has been set. Fixes [YOCTO #15685] (Bitbake rev: 0329a7e3ac694737f2d2c1861f65492551360663) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Reviewed-by: Ghislain Mangé <ghislain.mange@smile.fr> Suggested-by: Ola Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Add include_all conf file directiveRichard Purdie2024-12-172-0/+24
| | | | | | | | | | | | | | | | | | | | In some cases it would be helpful to be able to have an include file in a standard location which is included in all layers that are added to the system. The intent is for these to provide configuration tweaks of specific types so that a given file pattern can be adopted more widely for such configuration. The code will search for any named configuration file within BBPATH, so a configuration directive of: include_all conf/distro/include/myinc.conf would include the myinc.conf file in that subpath if present in any directory in BBPATH. Multiple files will be included if present. (Bitbake rev: d01d5593e7829ac60f37bc23cb87dc6917026471) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: parse: add support for flagged fragment variablesAlexander Kanavin2024-12-132-3/+10
| | | | | | | | | | | | | | | | | It was suggested that using flags in fragment content to mark fragment-specific metadata (such as descriptions and dependencies) is prone to quiet regressions when fragments are renamed or moved, and it clutters the fragment content. With this change allowed fragment metadata variables must be explicitly listed in a variable that is given to addfragment directive, and parser will add the flag containing the fragment name to them. (Bitbake rev: ed9a3ca9426500511feb77f41a146953dbfe9af7) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: add support for 'addfragments' directiveAlexander Kanavin2024-12-082-0/+42
| | | | | | | | | | | | | | | | It takes two parameters: - location prefix for fragments - name of variable that holds the list of enabled fragments, each of them prefixed by layer id Implementation of this directive essentially expands the fragment list obtained from the variable into absolute fragment paths and hands them to the implementation of 'require'. (Bitbake rev: f687746703e7b096c5480668fd4f49bd4951182b) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ConfHandler: allow / in variable flagsAlexander Kanavin2024-12-081-1/+1
| | | | | | | | | | | | This is beneficial for config fragments, as their names are specified via flags, and those names can include slashes: BB_CONF_FRAGMENT_SUMMARY[init/systemd] = "This fragment enables systemd as an init manager" (Bitbake rev: 80805988492b35593067230f68782a5687c8f557) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler/ast: Improve addtask handlingRichard Purdie2024-08-132-33/+34
| | | | | | | | | | | | | | | | | | The recent addtask improvement to handle comments complicated the regex significantly and there are already a number of corner cases in that code which aren't handled well. Instead of trying to complicate the regex further, switch to code logic instead. This means the following cases are now handled: * addtask with multiple task names * addtask with multiple before constraints * addtask with multiple after constraints The testcase is updated to match the improvements. (Bitbake rev: 417016b83c21fca7616b2ee768d5d08e1edd1e06) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Handle comments in addtask/deltaskRichard Purdie2024-08-131-2/+2
| | | | | | | | | | | | | | | Technically our syntax would allow for comments after an addtask/deltask. Currently these get silently processed in various ways by the code which is bad. Tweak the regex to drop any comments and add test cases to ensure this continues to work in the future. [YOCTO #15250] (Bitbake rev: 64f8796e55a8826ffec0d76b993c8256713f67a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Improve/fix cache invalidation via mtimeRichard Purdie2024-05-161-4/+8
| | | | | | | | | | | | | | | | | | We have been seeing obscure failures in devtool, particularly on newer autobuilder workers where it appears the cache is assumed to be valid when it shouldn't be. We're using the 'seconds' granulation mtime field which is not really a good way of telling if a file has changed. We can switch to the "ns" version which is better however also add in inode number and size as precautions. We already have all this data and tuples are fast so there isn't really any cost to do so. This hopefully fixes [YOCTO #15318]. (Bitbake rev: d9e5d313c79500e3c70ab9c3239b6b2180194f67) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Handle unclosed functions correctlyRichard Purdie2024-04-141-1/+7
| | | | | | | | | | | | | | | | | A function accidentally defined as: somefunction() { : } which is unclosed due to the space at the end, would currently silently cause breakage. Have the parser throw and error for this. [YOCTO #15470] (Bitbake rev: a7dce72da6be626734486808f1b731247697e638) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/BBHandler: Add inherit_defer supportRichard Purdie2024-01-182-1/+31
| | | | | | | | | | | | | | | | | | | | | | Add support for an inherit_defer statement which works as inherit does but is only evaulated at the end of parsing. This allows conditional expressions to be evaulated 'late' meaning changes to PACKAGECONFIG from bbappends can be applied for example. This addresses a usability/confusion issue users have with the current conditional inherit mechanism since they don't realise the condition has to be expanded immediately with the current model. There is a commented out warning we could enable in future which warns about the use of a conditional inherit that isn't deferred. There is a behaviour difference in the placement of the inherit, particularly around variables set using ?= which means wen swapping from one to the other, caution must be used as values can change. (Bitbake rev: 5c2e840eafeba1f0f754c226b87bfb674f7bea29) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast: Fix EXPORT_FUNCTIONS bugRichard Purdie2024-01-101-4/+5
| | | | | | | | | | | | | | | | If you have two classes, both of which set EXPORT_FUNCTIONS for the same funciton and a standard funciton definition for the function that is exported, the export function can sometimes overwrite the standard one. The issue is that the internal flag the code uses isn't ovweritten if the variable is giving a new value. Fix the issue by using a comment in the code that is injected so that we know if it is ours or not. Also add some testing for EXPORT_FUNCTIONS, not perfect but a start. (Bitbake rev: 66306d5151acb0a26a171c338d8f60eb9eb16c6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: Drop inotify support and replace with mtime checksRichard Purdie2023-09-181-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the flush in serverlog() removed and a memory resident bitbake with a 60s timeout, the following could fail in strange ways: rm bitbake-cookerdaemon.log bitbake-layers add-layer ../meta-virtualization/ bitbake-layers add-layer ../meta-openembedded/meta-oe/ bitbake -m specifically that it might error adding meta-oe with an error related to meta-virt. This clearly shows that whilst bblayers.conf was modified, bitbake was not recognising that. This would fit with the random autobuilder issues seen when the serverlog flush() call was removed. The issue appears to be that you have no way to "sync()" the inotify events with the command stream coming over the socket. There is no way to know if there are changes in the IO queue which bitbake needs to wait for before proceeding with the next command. I did experiment with os.sync() and fsync on the inotify fd, however nothing addressed the issue. Since it is extremely important we have accurate cache data, the only realistic thing to do is to switch to stat() calls and check mtime. For bitbake commands, this is straightforward since we can revalidate the cache upon new connections/commands. For tinfoil this is problematic and we need to introduce and explict command "revalidateCaches" that the code can use to force bitbake to re-check it's cache validity. I've exposed this through tinfoil with a new "modified_files" function. So, this patch: a) drops inotify support within bitbake's cooker/server and switch to using mtime b) requires a new function call in tinfoil when metadata has been modified (Bitbake rev: da3ec3801bdb80180b3f1ac24edb27a698415ff7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: ConfHandler: Allow variable flag name with a single characterKai Kang2023-04-131-2/+2
| | | | | | | | | | | | | | Update regex pattern to allow variable flag name with a single character. Regression tests have also been updated in `bb.parse` and `bin/bitbake-selftest -k ParseTest` has been successfully executed. Eliminate a trailing space as well. (Bitbake rev: bb9e523291a3cad6e1596ee6a1e715b5e5feba8f) Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler: Allow the '@' character in variable flag namesJames R T2023-03-231-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch enables the usage of the '@' character in variable flag names. One use case of variable flags is to assign the network namespaces of some systemd services/targets to configure other parts of the build process of some system. The filenames of systemd services/targets might contain the '@' character if they are template unit files that can take in a single parameter/argument and be instanced multiple times, as indicated by systemd's official manual page. The '@' character is disallowed as the first character in a variable flag name. Imposing more restrictions on the first character is a compromise to make parsing easier and to allow for more options in the future to extend the syntax. This patch is successfully verified by creating a custom BitBake recipe that sets and unsets the value of a variable flag with the '@' character in its name and ensuring that no ParseError is being thrown. Regression tests have also been added to `bb.parse`. `bin/bitbake-selftest` has also been successfully executed and all tests passed. (Bitbake rev: 00f9ab2cacfbd2a63b6b4959cf5401babae7e32a) Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: Add autorev warning when it is set too lateRichard Purdie2023-03-221-0/+3
| | | | | | | | | | | | | | | | | | | | Bitbake expects a consistent metadata environment when it parses. There are plenty of ways you can set a recipe to autorev at a point where parts of the fetcher have already been triggered leading to obsure bugs which I struggled to debug, let alone anyone not familar with the code. If anyone is running into the message from the commit, the issue is likely one of timing. Keep in mind that the anonymous python code in base.bbclass will expand variables like FILESPATH, WORKDIR and others which contain PV. The recipe needs to be set to AUTOREV before that anonymous python runs. In particular, that means you can't set SRCREV = "${AUTOREV}" in other anonymous python, it needs to happen earlier. (Bitbake rev: 4d9ec332d5bfc8b60b54f8ec2a17d34e35aa903a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/data/codeparser: Add dependencies from python module functionsRichard Purdie2022-12-171-0/+12
| | | | | | | | | | | | | | | | | | | | Moving code into python modules is a very effective way to reduce parsing time and overhead in recipes. The downside has always been that any dependency information on which variables those functions access is lost and the hashes can therefore become less reliable. This patch adds parsing of the imported module functions and that dependency information is them injected back into the hash dependency information. Intermodule function references are resolved to the full function call names in our module namespace to ensure interfunction dependencies are correctly handled too. (Bitbake rev: 605c478ce14cdc3c02d6ef6d57146a76d436a83c) (Bitbake rev: 91441e157e495b02db44e19e836afad366ee8924) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Add support for addpylib conf file directive and ↵Richard Purdie2022-12-084-7/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BB_GLOBAL_PYMODULES For many years OE-Core has injected it's own python modules into the python namespace using an immediate expansion of a variable in base.bbclass. It also added all entries from BBPATH to sys.path. We really need this to become a first class citizen of the langauge, this new addpylib directive allows that. Usage is of the form: addpylib <directory> <namespace> The namespace is imported and if there is an attribute BBIMPORT, that list of names is iterated and imported too. This mirrors what OE-Core has done for a long time with one difference in implmentation, sys.path is only appended to. This means later imported namespaces can't overwrite an earlier one and can't overwrite the main python module space. In practice we've never done that and it isn't something we should encourage or support. The new directive is only applied for .conf files and not other filetypes as it only makes sense in that context. It is also only allowed in the "base configuration" context of cookerdata since adding it at the recipe level wouldn't work properly due to the way it changes the global namespace. At the same time, move the list of modules to place in the global namespace into a BB_GLOBAL_PYMODULES variable. It is intended that only the core layer should touch this and it is meant to be a very small list, usually os and sys. BB_GLOBAL_PYMODULES is expected to be set before the first addpylib directive. Layers adding a lib directory will now need to use this directive as BBPATH is not going to be added automatically by OE-Core in future. The directives are immediate operations so it does make modules available sooner than the current OE-Core approach. The new code appends to sys.path rather than prepends as core did, as overwriting python standard library modules would be a bad idea and naturally encouraging people to collaborate around our own core modules is desireable. (Bitbake rev: afb8478d3853f6edf3669b93588314627d617d6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Remove pointless global variable declarationsRichard Purdie2022-11-291-2/+2
| | | | | | | | | | These are static regex compilations which don't change or a standard module import (bb). There is noneed to declare them as global so drop the pointless code which doesn't do anything. (Bitbake rev: 09a4c159e3fd184f730821e7bd99916b0d28dc70) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler/cooker: Implement recipe and global classesRichard Purdie2022-08-121-11/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | We have some confusion for users since some classes are meant to work in the configuration space (or "globally") and some are meant to be selected by recipes individually. The cleanest way I could find to clarify this is to create "classes-global" and "classes-recipe" directories which contain the approproate classes and have bitbake switch scope between them at the appropriate point during parsing. The existing "classes" directory is always searched as a fallback. Once a class is moved to a specific directory, it will no longer be found in the incorrect context. A good example from OE is that INHERIT += "testimage" will no longer work but IMAGE_CLASSES += "testimage" will, which makes the global scope cleaner by only including it where it is useful and intended to be used (images). (Bitbake rev: f33ce7e742f46635658c400b82558cf822690b5e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Make inherit calls more directlyRichard Purdie2022-08-121-1/+4
| | | | | | | | | | | Rather than recursing into the conf handler code, simply call into the parse code directly when inheriting files as we've already resolved the paths and don't need anything the other codepath brings. This makes the codepath clearer at the expense of some slight duplication. (Bitbake rev: 0f4f3af6d93a0018df58b8a1d8d423c78ba6526d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Allow earlier exit for classes not foundRichard Purdie2022-08-121-0/+3
| | | | | | | | | Rather than relying on later code to error if the class isn't found, exit earlier and more clearly from a code perspective. (Bitbake rev: f7c55c8147329670fd5bc55b1ae3f47f25b89bab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler/BBHandler: Improve comment error messages and add testsRichard Purdie2022-07-042-4/+9
| | | | | | | | | | | | | | | | | | | Currently if you trigger one of the comment errors, the newline characters are stripped and the line numbers are incorrect. In one case it prints the empty line which is also unhelpful. Rework the code around these errors so the line numbers are correct and the lines in question are more clearly displayed complete with newlines so the user can more clearly see the error. I also added a couple of simplistic test cases to ensure that errors are raised by the two known comment format errors. [YOCTO #11904] (Bitbake rev: 712da71b24445c814d79a206ce26188def8fce0a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler: Remove lingering closeOla x Nilsson2022-06-211-2/+0
| | | | | | | | | | | | The f.close() statement should have been removed in 459ad524756a3f9b50feeedf31e33502dceae8d5. (Bitbake rev: 9fc1bab6b7e3c0fca3ddec4bc8c7763d2aff8bab) Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast: Improve function flags handling for EXPORT_FUNCTIONSRichard Purdie2022-04-141-1/+1
| | | | | | | | | | | | | | | Currently, if you use one of the functions from EXPORT_FUNCTIONS, the meaning of cleandirs and fakeroot are lost. This leads to the function changing in behaviour depending upon it's caller context. This isn't intended so add mapping for the cleandirs and fakeroot flags too. This does break devtool in OE-Core and there is a separate fix for that. [YOCTO #8621] (Bitbake rev: b074f4aff00923acc5bf6649d204d541a79fd2b6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse: Ensure any existing siggen is closed down firstRichard Purdie2022-04-031-0/+2
| | | | | | | | | | We're still seeing issues with unclosed asyncio event loops. At the init site, make sure any existing one is closed first to try and avoid this. (Bitbake rev: 78dee3c03c75a27531fcff26f9298fce2519bdde) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb: fix exit when found renamed variablesMarta Rybczynska2022-02-211-0/+4
| | | | | | | | | | | | | | | | Until now, if a renamed variable was found, bitbake exited immediately if it was in a class, but continued after an error message if the variable was in a recipe. This was caused by cookerdata.py CookerDataBuilder::parseBaseConfiguration checking a different DataSmart instance than the variable was set in. To solve the issue, add a special variable and set it when we find a renamed variable. Check for it in ast.py and bail out if needed. (Bitbake rev: d12400600e30549c88dc9e7883dc3d63b1dc1117) Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache/ConfHandler: Drop TOPDIR/chdir manipulationsRichard Purdie2021-11-151-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | This code has been unchanged since 2006 apart from attempts to optimise performance by minimising chdir() calls. There is no reason the modern bitbake parser should be changing directory all the time. We did have some path assumptions in the mists of time but those were resovled and the code is deterministic and doesn't depend on cwd now for parsing. We can therefore drop the changes in directory. Also, TOPDIR is now being set by cookerdata in all cases so we don't need the fallbacks in this code (which was used to effectively initialise a value). We don't need to change TOPDIR when parsing a recipe, that makes no sense. If we stop all the other messing around, we don't need to expand TMPDIR either. These changes have the potential to break some obscure use cases such as an anonymous function assuming the current working directory, or some case which depends on TOPDIR changing but I believe any such uses should be fixed at this point. (Bitbake rev: add5d488e1d6607a98441836075d01cb1dc9c0fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: Show warnings for append/prepend/remove operators ↵Richard Purdie2021-11-101-0/+4
| | | | | | | | | | | | combined with +=/.= Operations like XXX:append += "YYY" are almost always wrong and this is a common mistake made in the metadata. Show warnings for these usages with a view to making it a fatal error eventually. (Bitbake rev: 8c31e75557dc6a8d8f407b5d24d6327889a3e3b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse_py: Drop deprecated function referenceRichard Purdie2021-09-101-3/+0
| | | | | | | | This is now used from bb.parse everywhere so drop this long deprecated reference. (Bitbake rev: aaa5292ef96ea27f505bc5c5a4b1eb4f497ed061) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Switch to using new override syntaxRichard Purdie2021-08-021-2/+0
| | | | | | | | | | | | | | | | | This change updates the datastore to use the new override syntax using colons instead of underscores exclusively. It is expected layers would have to be converted to work with bitbake after this change. Supporting mixed syntax isn't possible, it is only feasible to have one internal representation of overrides. Whilst we can't warn for every possible override that may be set in the old format, show errors for _append/_prepend/_remove since those should never be present. (Bitbake rev: 7dcf317cc141dc980634f8c18bfa84f83e57206a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: parse/ast: Substitute '~' when naming anonymous functionsPaul Barker2021-07-291-1/+1
| | | | | | | | | | | | | When parsing an anonymous python function, bitbake generates a name for the function based on the full path to the file in which it was found. As not all characters which are valid in file paths are valid in Python function names we have a translation table. However, this translation table was missing an entry for '~'. (Bitbake rev: b201c0b284e25c20685d9d206797c4cc650eeca0) Signed-off-by: Paul Barker <paul@pbarker.dev> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart/parse: Allow ':' characters in variable/function namesRichard Purdie2021-07-203-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is becomming increasingly clear we need to find a way to show what is/is not an override in our syntax. We need to do this in a way which is clear to users, readable and in a way we can transition to. The most effective way I've found to this is to use the ":" charater to directly replace "_" where an override is being specified. This includes "append", "prepend" and "remove" which are effectively special override directives. This patch simply adds the character to the parser so bitbake accepts the value but maps it back to "_" internally so there is no behaviour change. This change is simple enough it could potentially be backported to older version of bitbake meaning layers using the new syntax/markup could work with older releases. Even if other no other changes are accepted at this time and we don't backport, it does set us on a path where at some point in future we could require a more explict syntax. I've tested this patch by converting oe-core/meta-yocto to the new syntax for overrides (9000+ changes) and then seeing that builds continue to work with this patch. (Bitbake rev: 0dbbb4547cb2570d2ce607e9a53459df3c0ac284) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Prevent bitbake from executing event handler for wrong ↵Tomasz Dziendzielski2021-02-161-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | multiconfig target When multiconfig is used bitbake might try to run events that don't exist for specific mc target. In cooker.py we pass `self.databuilder.mcdata[mc]` data that contains names of events' handlers per mc target, but fire_class_handlers uses global _handlers variable that is created during parsing of all the targets. This leads to a problem where bitbake runs event handler that don't exist for a target or even overrides them - if multiple targets use event handler with the same name but different code then only one version will be executed for all targets. See [YOCTO #13071] for detailed bug information. Add mc target name as a prefix to event handler name so there won't be two different handlers with the same name. Add internal __BBHANDLERS_MC variable to have the handlers lists per machine. (Bitbake rev: 5f7fdf7b2d8c59805c8ef4dae84f536baa5e172b) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: logging: Make bitbake logger compatible with python loggerJoshua Watt2021-02-104-5/+5
| | | | | | | | | | | | | | | | | The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] (Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: Don't classify shell functions that names start with ↵Tomasz Dziendzielski2021-02-101-1/+1
| | | | | | | | | | | | | | | | | | "python*" as python function If shell function name starts with 'python' or 'fakeroot' parser wrongly assumes it's python/fakeroot function. [YOCTO #14204] Use regex lookahead assertions to check if 'python' expression is followed by whitespace or '(' and if 'fakeroot' is followed by whitespace. (Bitbake rev: b07b226d5d1b3acd3f76d8365bc8002293365999) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: BBHandler: prompt error when task name contain expressionLee Chee Yang2020-11-081-1/+5
| | | | | | | | | | | | | | Task name contain expresion (eg, do_foo_remove_bar) can cause fatal error. Check for naming before addtask. Prompt with understandable error message when expression found in task name. [YOCTO #8805] (Bitbake rev: 5b4a5bb0960386f9c524c220e43a16e60e38964d) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Allow deltask to take multiple tasknamesRichard Purdie2020-07-212-8/+7
| | | | | | | | | | | | | | | | | | | deltask currently supports only one task to delete but it would be useful if it could support a variable which gets expanded to allow flexibility in the metadata. This is simple to support in bitbake and is how other directives such as inherit operate so adjust the parser/code to handle that. It means that syntax like: EXTRA_NOPACKAGE_DELTASKS = "" deltask ${EXTRA_NOPACKAGE_DELTASKS} is now allowed. (Bitbake rev: 883d926120833c85a16dcf60425dd7af7699046a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event/ast: Add RecipePostKeyExpansion eventRichard Purdie2020-05-271-0/+3
| | | | | | (Bitbake rev: 5c30cbe35e921f118e7da6b804af281627329d77) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: amend code to use proper singleton comparisons where possibleFrazer Clews2020-01-191-10/+10
| | | | | | | | | | | amend the code to handle singleton comparisons properly so it only checks if they only refer to the same object or not, and not bother comparing the values. (Bitbake rev: b809a6812aa15a8a9af97bc382cc4b19571e6bfc) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib: remove unused importsFrazer Clews2020-01-192-6/+0
| | | | | | | | | | removed unused imports which made the code harder to read, and slightly but less efficient (Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697) Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: ConfHandler: Use with to manage filehandle lifetimeOla x Nilsson2019-10-281-23/+23
| | | | | | | (Bitbake rev: 459ad524756a3f9b50feeedf31e33502dceae8d5) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: lib: Cleanup /usr/bin/env pythonRobert Yang2019-06-283-3/+0
| | | | | | | (Bitbake rev: cc712f3257904960247a7532cfc4611f3dccd36c) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-07-09 07:30:53 +0000