summaryrefslogtreecommitdiffstats
path: root/bitbake/bin/bitbake-setup
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: bitbake-setup: Rename bb-layers-relative to bb-layers-file-relativeRichard Purdie2025-11-071-5/+5
| | | | | | | | | | | | | | | The difference between bb-layers and bb-layers-relative is unclear as both are relative paths. Rename one to "file-relative" which makes it clear it is relative to the current file, without becomming a long name. https://lists.openembedded.org/g/bitbake-devel/message/18296 Based on a patch from Alexander Kanavin <alex@linutronix.de> but with different naming. (Bitbake rev: dcb17758b99767ab6da4172cf60eabc9269082dd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: replace {THISDIR} token with a keyword: ↵Alexander Kanavin2025-11-061-12/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | bb-layers-relative {THISDIR} is a special value token that can be used in the list of enabled layers to specify the layer location relative to the confguration file: https://git.openembedded.org/bitbake/commit/?id=b3153be29de8b8570b0c184369bd41f4c646cf92 This replaces the token with an explicit separate keyword for such layers: so that special processing to determine the final value can be avoided, and the feature can be formalized in the json schema: instead of "bb-layers": [ "{THISDIR}/meta-my-project" ] this allows "bb-layers-relative": [ "meta-my-project" Going forward I think we should strive to avoid any further special value tokens. (Bitbake rev: 90da82bd2bfcfd5590c9ae06015737b616074b56) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: capture revisions while checking out layersJohannes Schneider2025-11-061-3/+17
| | | | | | | | | | | | | | | | | | | | | | | | | When initializing a build setup from a conf.json that only sets 'rev' to a tag or branch, the actual revision would not be captured or logged. To capture the current layer state after an 'init' or 'update', the checkout_layers function is extended to store the revision the bb.fetch.Fetch pulled, and write that information into a sources-fixed-revisions.json file. This file can then be fed back into bitbake-setup init as: --sources-overrides This new 'sources-fixed-revisions.json' is written during 'update_build' and stored alongside the 'config-upstream.json' in the config dir. And put with the later under version control by calling 'commit_config" after 'update_build'. The use of 'deepcopy' is necessary to not modify the original input data - which python passes around as reference. (Bitbake rev: 95866ff03f78e987ae7e47daad053bc0f353eea4) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: checkout_layers: construct 'src_uri' separatlyJohannes Schneider2025-11-061-3/+4
| | | | | | | | | | | | | | Construct the 'src_uri' separately, and then pass either variant into one call that creates the Fetch. Making use of format-strings to shorten/simplify the code. Also: using 'proto' instead of 'type' for a variable name, to avoid the protected keyword. (Bitbake rev: 4ad70e05ceca19c1e903dafc33386a82b1176bba) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: rename 'write_config' into 'write_upstream_config'Johannes Schneider2025-11-061-4/+4
| | | | | | | | | | Rename the function to reflect which (of the possibly many) files in the build/config directory it writes to. (Bitbake rev: 7f9cd4c1e91d110085d74f9b9f12884f97f4e8dd) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: fix key-error when changing a new 'section'Johannes Schneider2025-11-061-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Setting a key=value in a new section would raise a key-error when using in-memory settings, e.g: ./bin/bitbake-setup --setting default top-dir-prefix /tmp/bitbake-setup --setting default top-dir-name gs settings foo bar baz Loading settings from /tmp/bitbake-setup/gs/settings.conf Traceback (most recent call last): File "/tmp/bitbake-setup/bitbake/./bin/bitbake-setup", line 853, in <module> main() File "/tmp/bitbake-setup/bitbake/./bin/bitbake-setup", line 838, in main all_settings = merge_settings(builtin_settings, global_settings, topdir_settings, args.setting) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/bitbake-setup/bitbake/./bin/bitbake-setup", line 733, in merge_settings all_settings[section][setting] = value ~~~~~~~~~~~~^^^^^^^^^ KeyError: 'foo' (Bitbake rev: 78ab0d15dff5ccf64b0bf681185370779e6cabaf) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: rename function 'default_settings_path' to ↵Johannes Schneider2025-11-061-8/+8
| | | | | | | | | | | | | | | | | 'topdir_settings_path' Rename the function to align with the corresponding 'global_settings_path' function, and move it down just above the later. This is done to differentiate from the built-in default settings, and to align with the other places in the code that use 'topdir_settings' (or 'global_settings') (Bitbake rev: 3df994f773abbd1d0240e721f5fd29d4b021bfb5) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: rename 'build' -> 'setup'Alexander Kanavin2025-11-041-47/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | After a terminology review by Antonin we brainstormed something less confusing than 'build' and 'build directory' for the place where bitbake-setup clones layers and creates a *bitbake* build directory in. People are bound to get these two confused and mix them up, and 'setup' is much more distinct and aligns nicely with 'bitbake-setup'. It's also not claimed by anything else in OE/Yocto. So before: top-dir -> build-dir (can be several) -> bitbake build dir, layer dir, config dir Now: top-dir -> setup-dir (can be several) -> bitbake build dir, layer dir, config dir This also updates the respective command line options, I understand it's a breaking change, but as before the tweaks are simple and we need to get the terminology right for the users, and now is the time to do it. (Bitbake rev: eeb81a35bf0304451f7612950d5156ea7ff18bad) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: correctly define --global argument for settings ↵Alexander Kanavin2025-11-031-1/+1
| | | | | | | | | | | | | | | | management --global was defined twice (for settings command as a whole, and specifically for set/unset sub-commands), which caused 'settings --global set ...' to be allowed, and at the same time not have any effect. 'settings set --global' is the only valid form after this change. (Bitbake rev: 2f14cf68f6c653b6da90180a99a22c58855b0cef) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Tested-by: Antonin Godard <antonin.godard@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: commandline: use subsubparser for settings ↵Johannes Schneider2025-10-141-19/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | {list,set,unset} Previously the sub-command 'settings' would take any number of arguments and then silently do nothing if the number wasn't three. The help text was also not clear about this, marking the positionals separately as optional: usage: bitbake-setup settings [-h] [--global] [--unset UNSET UNSET] [-l] [section] [key] [value] The '--unset SECTION SETTING' also did not integrate too well, as it had its own positional arguments for section+setting. For a bit more consistency and a explorable help, a sub-subparser is added, that provides the commands: bitbake-setup settings list bitbake-setup settings set foo bar baz bitbake-setup settings unset foo bar with a '--global' that is added from a stand-alone parent parser, so that it shows up in all sub-command help texts. The new help text now reads: usage: bitbake-setup settings [-h] [--global] {list,set,unset} ... and the respective sub commands: usage: bitbake-setup settings list [-h] [--global] usage: bitbake-setup settings set [-h] [--global] <section> <setting> <value> usage: bitbake-setup settings unset [-h] [--global] <section> <setting> (Bitbake rev: 8b582ef8dd0cef0192d4c0104bcd9b5d642d132c) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add 'metavar' for self-descriptive parametersJohannes Schneider2025-10-141-2/+3
| | | | | | | | | | | | | | | | | | | | Add a metavar to the argparse options to have a self-descriptive help text. Otherwise argpase defaults to use the argument name in all-uppercase. Before: usage: bitbake-setup [-h] [-d] [-q] [--color COLOR] [--no-network] [--global-settings GLOBAL_SETTINGS] [--setting SETTING SETTING SETTING] {list,init,status,update,install-buildtools,settings} ... After: usage: bitbake-setup [-h] [-d] [-q] [--color COLOR] [--no-network] [--global-settings PATH] [--setting SECTION SETTING VALUE] {list,init,status,update,install-buildtools,settings} ... (Bitbake rev: 83cecc9356a0684f90249d527fe372298ae92719) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: use args.cmdline_settings for --settingsJohannes Schneider2025-10-141-3/+4
| | | | | | | | | | To align the storage destination with the internally used variable name. This makes room for having another option use 'args.setting' (Bitbake rev: 14d8535309abc78ee30cfdb51bba2e00b474f443) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: init: suggest removing a partially initialized top-dirJohannes Schneider2025-10-141-1/+1
| | | | | | | | | | | | | | In cases where the first call to 'init' failed or was aborted before creating the 'build/init-build-env' has been created, a user can get stuck: a second call to init aborts, suggesting 'status' or 'update' but these to refuse because the --build-dir is not valid. Guide the user by adding a suggestion to start over from scratch. (Bitbake rev: 11b2740c3e19e0c6680229c6bbce3691c73746a8) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: correct 'setting' to 'settings' in a couple of help ↵Alexander Kanavin2025-10-141-2/+2
| | | | | | | | | texts (Bitbake rev: c5e04ce986b680f39f6c7851e4236c4afa4ac3f4) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: further rework the settings handlingAlexander Kanavin2025-10-141-101/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After some further feedback, additional changes are made: 1. 'setting' command is renamed to 'settings' to better reflect that it is an interface to various ways of managing settings. 2. This command now has a -l/--list option to list all settings with their values (same as 'git config -l'). 3. A new level of settings (built-in defaults) is added, and used as a last resort after command line options, top dir settings file and global settings file. 4. This means bitbake-setup does not have to write and use a global settings file, and it no longer does so when initializing a build, avoiding default 'pollution' of ~/.config/bitbake-setup/ which can be problematic or unwelcome. A global settings file is still created if a setting is explicitly requested to be placed into it. 5. 'install-global-settins' is removed as the use case for it (tweak default settings before using them to initialize a build) can be achieved by setting the settings individually. 5. Similarly, a top dir settings file is no longer created by default and only appears if a setting needs to be written into it. 6. Default dl-dir is again created inside a top directory and not in ~/.cache/ to make default builds fully contained in the top directory (which was also asked about). (Bitbake rev: 664f8ec48d42d2ddc5f234c4f7d590fa597f489a) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: rework the settings handlingAlexander Kanavin2025-10-141-147/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the outcome of various discussions, suggestions and pull requests on github. What has specifically changed? 1. The sources for the settings are no longer separated, but are stacked and given priorities, from highest to lowest: a. '--setting section key value' on the command line b. a settings file in the top directory c. a global settings file in ~/.config/bitbake-setup/ (or in a file pointed to by --global-settings) Any setting can be in any of these three locations (other than top dir name and prefix which do not make sense in the settings file in the top directory). 2. A global settings file must contain all of the needed settings, while a settings file in the top directory can be empty (and this is how they are written out if they do not exist). Specifically, both dl-dir and registry settings have been relocated to the global file, and dl-dir defaults to ~/.cache/bitbake-setup/downloads, rather than somewhere in top dir. 3. The file name for both global and top dir settings is now 'settings.conf'. 4. --top-dir-prefix and --top-dir-name options have been removed and superseded by a generic, universal --setting option. 5. 'install-settings' command has been removed, as it is no longer does anything useful, and is superseded by the 'setting' command (see below). 'install-global-settings' has been retained, to be able to have a set of global defaults that can be changed without initializing a build. 6. 'change-setting', 'change-global-setting' and 'install-settings' have all been replaced by a single 'setting' command that mimics 'git config' in its parameters: a. Changing a setting: bitbake-setup setting [--global] default dl-dir /path/to/downloads b. Removing a setting: bitbake-setup setting [--global] --unset default dl-dir (Bitbake rev: 713e7f213c6d4a620be9ce34d5f4396af48e1d69) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: support adding environment-passthroughs to the ↵Johannes Schneider2025-10-141-1/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | init-build-env This patch adds support for extending the BB_ENV_PASSTHROUGH_ADDITIONS environment variable from within the `init-build-env` wrapper script - generated by either oe-core's oe-setup-build, or by `bitbake-setup` - based on per-configuration JSON settings. This enables CI workflows to inject environment-specific data - such as build number, host, build type, or credentials required to fetch from certain SRC_URIs - which cannot be captured via configuration fragments alone. These variables are now handled early in the setup process and exported directly into the build environment. Example: "bb-env-passthrough-additions": [ "ACME_DIR", "ARTIFACTORY_TOKEN", "ARTIFACTORY_USERNAME", "GITHUB_TOKEN", "GITHUB_PROTOCOL", "KEY" ] <snip> the resulting 'init-build-env' would then be: # environment passthrough added by bitbake-setup export BB_ENV_PASSTHROUGH_ADDITIONS=" \ $BB_ENV_PASSTHROUGH_ADDITIONS \ ACME_DIR \ ARTIFACTORY_TOKEN \ ARTIFACTORY_USERNAME \ GITHUB_TOKEN \ GITHUB_PROTOCOL \ KEY" # init-build-env wrapper created by bitbake-setup . /tmp/acme_master-acme-distro_acme-machine_bang/layers/openembedded-core/oe-init-build-env /tmp/bitbake-setup/gs/acme_master-acme-distro_acme-machine_bang/build (Bitbake rev: 782ab99e7a04fba43bdcf5763a6280785944ae3f) Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add support for skipping a fragment selectionAlexander Kanavin2025-10-141-2/+8
| | | | | | | | | | | | | | | | | In autobuilder testing a use case arised where - the available choices in configuration file for choosing a machine are incomplete - putting every possible machine choice into that configuration is undesirable/not possible - autobuilder code can write a machine selection into the bitbake config after the fact. So this --skip-selection option is intended for advanced users that know what they're doing and is generally not recommended as it requires manually tweaking the bitbake config to make it usable. (Bitbake rev: 8cb2372bdad381179969d2ecbba7decaf03a7c5f) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: clarify that default answer to prompts is noPaul Gortmaker2025-10-141-3/+3
| | | | | | | | | | | | | | | | It is common practice to put the default choice in upper case for yes/no interactive prompts, so that when people just hit enter, they know what they are getting. An example that linux users are probably familiar with is "sensors-detect" from the "lm-sensors" package. Unify all the prompts to be the same and indicate that the default answer from hitting enter is a no with an upper case N. No functional changes. (Bitbake rev: 7d6225722e21b116ae164fbaae2a918534a5107b) Signed-off-by: Paul Gortmaker <paulg@kernel.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: allow using {THISDIR}/my-layerYoann Congal2025-10-141-3/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This implement the ability to use "{THISDIR}/my-layer" in the "bb-layers" list. "{THISDIR}" is remplaced by the directory containing the configuration file. In small projects, we try to keep the setup a simple as possible: a single git repo containing both the build confguration (e.g. a bitbake-setup configuration file) and the meta layer with project recipes/machine/distro. This change allows this kind of setup: ├── meta-my-project/ # the project layer └── my-project.conf.json # the bb-setup configuration file by writing, in my-project.conf.json: "bitbake-setup": { "configurations": [{ "bb-layers": [ "{THISDIR}/meta-my-project" Note: in this case meta-my-project is not present as a "source", so, not handled by bb-setup update/status. It is expected of the user to handle this on their own (is our case, a simple git workflow). (Bitbake rev: b3153be29de8b8570b0c184369bd41f4c646cf92) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: dash support for init-build-env scriptYoann Congal2025-10-141-7/+8
| | | | | | | | | | | | | | | | | | | | | | Being minimalist, dash does not support the (non-POSIX) feature of passing an argument while sourcing a script. Like in . <some path>/oe-init-build-env <build dir> With dash, one must use: set <build dir> # puts <build dir> in $1 cd <some path> . ./oe-init-build-env # can only be called from its directory in dash To do this: * Instead of a symlink to oe-init-build-env, keep a symlink to the directory containing it (called "oe-init-build-env-dir") * Generate a init-build-env script that dash can source using the above snippet. (Bitbake rev: 442b41c7949e1522212b66b16811f6b64b089b23) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: suggest "." instead of "source"Yoann Congal2025-10-141-1/+1
| | | | | | | | | | | | "." is in POSIX standard[0], whereas "source" is only supported in more feature-full shells (bash, zsh, ...) [0]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#dot (Bitbake rev: 22c5fe7b2de74841e86d28a81143bd1a717518d9) Signed-off-by: Yoann Congal <yoann.congal@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: improve robustness of loading/writing settingsAlexander Kanavin2025-10-141-3/+3
| | | | | | | | | | | | | | | Particularly: - ensure global settings command line argument is always expanded to full path - ensure any errors that happen when loading settings are reported at that point, otherwise we get an empty dictionary and cryptic key errors later (Bitbake rev: 578afa2f05dfa6727952365918df703875070f64) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add support for specifying branches in repo checkoutsAlexander Kanavin2025-10-141-2/+10
| | | | | | | | | | | | | | | | | | | | Previously bitbake-setup was checking out 'detached commits' using fetcher's nobranch feature, as that is the only option when only a revision is in the config. Branches are optional, but beneficial, as - checkout directory will be on a branch, making it easier for users to understand where they are if they need to make changes (also bitbake will print branch information instead of saying 'HEAD:sha'). - supply chain security! Enforcing a branch means any specified revision has to be on it, and no one can sneak in (accidentally or deliberately!) some dangling commit, or something from their private branch in the same repo. (Bitbake rev: 45ed9b9faebdaa8cb7cc8dd2a6d51ec8eea06e73) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: Improve the already initialized testRichard Purdie2025-10-141-2/+2
| | | | | | | | | If the directory already exists but hasn't been setup, the current test can fail so improve it. (Bitbake rev: dac27bd5acbde1807b9637f809fd0ee5bf424286) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: Switch to internal default registry filesRichard Purdie2025-10-141-1/+1
| | | | | | | | | Switch the url to be the default internal registry rather than a private repo which was intended for testing. (Bitbake rev: e031b75b5b92552d812d2305a35ce90eb6c68b78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: Allow local registry pathsRichard Purdie2025-10-141-4/+9
| | | | | | | | | | It is useful for bitbake-setup to support local paths without access through the fetcher so that internal data to the bitbake repository can be used as a default. (Bitbake rev: ec82a6d402a0bec5704310c66c6f4206c75e4fc9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add 'install-buildtools' commandAlexander Kanavin2025-10-141-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This basically calls install-buildtools from oe-core/poky, but it ensures via command line parameters that the installation location is stable and the downloads are preserved for reproducibility: $ bin/bitbake-setup install-buildtools Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf ====== Buildtools archive is downloaded into /home/alex/bitbake-builds/yocto-master-testing/buildtools-downloads/20250319141333 and its content installed into /home/alex/bitbake-builds/yocto-master-testing/buildtools ... (output from install-buildtools script) ====== It also detects when buildtools are already installed, and will direct users what to do: ====== alex@Zen2:/srv/work/alex/bitbake$ bin/bitbake-setup install-buildtools Loading settings from /home/alex/bitbake-builds/bitbake-setup.conf Buildtools are already installed in /home/alex/bitbake-builds/yocto-master-testing/buildtools. If you wish to use them, you need to source the the environment setup script e.g. $ . /home/alex/bitbake-builds/yocto-master-testing/buildtools/environment-setup-x86_64-pokysdk-linux You can also re-run bitbake-setup install-buildtools with --force option to force a reinstallation ====== This commits includes fixes by Gyorgy Sarvari <skandigraun@gmail.com> https://github.com/kanavin/bitbake/pull/2 (Bitbake rev: 3fe3096847046110c72b23fce37fb4a459b1d748) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-setup: add the initial implementationAlexander Kanavin2025-10-141-0/+809
Preamble ======== The latest iteration of this patchset is available at https://github.com/kanavin/bitbake I recommend taking the patches from there to ensure that you are not trying out outdated code. For the rationale and design guidelines please see this message: https://lists.openembedded.org/g/openembedded-architecture/message/1913 Left out for now but will be done later: - official configuration repository - documentation Amble *scratch* HOWTO ===================== 1. If you don't know where to start, run 'bitbake-setup init'. Bitbake-setup will ask a few questions about available configuration choices and set up a build. Note: 'init' sub-command can also take a path or a URL with a configuration file directly. You can see how those files look like here: https://github.com/kanavin/bitbake-setup-configurations 2. You can then source the bitbake environment and run bitbake to perform builds as usual: $ . /home/alex/bitbake-builds/yocto-master-options-poky-distro_poky-machine_qemux86-64/build/init-build-env Also, subsequent status/update commands will not require a separate --build-dir argument telling bitbake-setup where the build is. 3. To check if the build configuration needs to be updated, run: === $ bin/bitbake-setup status ... Configuration in /home/alex/bitbake-builds/poky-alex/ has not changed. === If the configuration has changed, you will see the difference as a diff. ... - "rev": "akanavin/sstate-for-all" + "rev": "akanavin/bitbake-setup-testing" ... If the configuration has not changed, but layer revisions referred to it have (for example if the configuration specifies a tip of a branch), you will see that too: === ... Layer repository git://git.yoctoproject.org/poky-contrib checked out into /home/alex/builds/poky-alex/layers/poky updated revision akanavin/sstate-for-all from 6b842ba55f996b27c900e3de78ceac8cb3b1c492 to aeb73e29379fe6007a8adc8d94c1ac18a93e68de === 4. If the configuration has changed, you can bring it in sync with: $ bin/bitbake-setup update Note that it will also rename/preserve the existing build/conf directory, and print changes in bitbake configuration (diff of content of build/conf/) if that has changed. I can't at the moment think of anything more clever that is also not much more brittle or complex to implement, but open to suggestions. Terminology =========== - 'top directory' means the place under which bitbake-setup reads and writes everything. bitbake-setup makes a promise to not touch anything outside of that, unless otherwise directed to by entries in settings (currently there is one such setting for fetcher downloads for layers and config registries). Top directory can be selected by an environment variable, a command line option, or otherwise assumed to be ~/bitbake-builds/. If BBPATH is in environment (e.g. we are in a bitbake environment), then the top directory is deduced from that and doesn't need to be specified by hand. - 'settings' means bitbake-setup operational parameters that are global to all builds under a top directory. E.g. the location of configuration registry, or where the bitbake fetcher should place the downloads (DL_DIR setting). Settings are stored in a .conf file in ini format just under the top directory. - 'build' means a tree structure set up by 'bitbake-setup init', consisting of, at least, a layers checkout, and a bitbake build. It maps 1:1 to the json data it was constructed from, which is called 'build configuration'. Build configurations are constructed from generic configurations that may involve making one or more choices about available options in them. Generic configurations are files, URLs or are obtained from git repositories called 'config registries', in which case they can be listed with 'bitbake-setup list'. There can be multiple 'builds' under a top directory. Here are two example generic configurations that showcase this: https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-options.conf.json https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-nested-configs.conf.json - 'bitbake-setup status' will tell if a build is in sync with the generic configuration it was made from. 'bitbake-setup update' will bring a build in sync with a configuration if needed. - 'bitbake build' means a particular sub-tree inside a build that bitbake itself operates on, e.g. what is set in BBPATH/BUILDDIR by oe-init-build-env. conf/* in that tree is 'bitbake configuration'. Bitbake configurations are constructed from templates and fragments, with existing mechanisms provided by oe-core. The configuration file format is specified such that other mechanisms to set up a bitbake build can be added; there was a mention of ability to specify local.conf content and a set of layers directly in a configuration. I think that scales poorly compared to templates and fragments, but I made sure alternative ways to configure a bitbake build are possible to add in the future :) - 'source override' is a json file that can be used to modify revisions and origins of layers that need to be checkout into a build (e.g. when master branches need to be changed to master-next for purposes of testing). Such a file is specified with a command-line option to 'init' and an example can be seen here: https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-next.override.json This commit includes fixes by Ryan Eatmon <reatmon@ti.com> https://github.com/kanavin/bitbake/pull/1 Gyorgy Sarvari <skandigraun@gmail.com> https://github.com/kanavin/bitbake/pull/2 Johannes Schneider <johannes.schneider@leica-geosystems.com> https://github.com/kanavin/bitbake/pull/3 https://github.com/kanavin/bitbake/pull/5 (Bitbake rev: b96154aeb1fc89184ac245e0d68e6e726fe80c04) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>