bitbake: bitbake-setup: add the initial implementation

Preamble
========

The latest iteration of this patchset is available at
https://github.com/kanavin/bitbake
I recommend taking the patches from there to ensure that
you are not trying out outdated code.

For the rationale and design guidelines please see this message:
https://lists.openembedded.org/g/openembedded-architecture/message/1913

Left out for now but will be done later:

- official configuration repository

- documentation

Amble *scratch* HOWTO
=====================

1. If you don't know where to start, run 'bitbake-setup init'.

Bitbake-setup will ask a few questions about available configuration choices and set up a build.

Note: 'init' sub-command can also take a path or a URL with a configuration file directly.
You can see how those files look like here:
https://github.com/kanavin/bitbake-setup-configurations

2. You can then source the bitbake environment and run bitbake to perform builds as usual:

$ . /home/alex/bitbake-builds/yocto-master-options-poky-distro_poky-machine_qemux86-64/build/init-build-env

Also, subsequent status/update commands will not require a separate --build-dir argument telling
bitbake-setup where the build is.

3. To check if the build configuration needs to be updated, run:

===
$ bin/bitbake-setup status
...
Configuration in /home/alex/bitbake-builds/poky-alex/ has not changed.
===

If the configuration has changed, you will see the difference as a diff.
...
  -                "rev": "akanavin/sstate-for-all"
  +                "rev": "akanavin/bitbake-setup-testing"
...

If the configuration has not changed, but layer revisions referred to it have (for example
if the configuration specifies a tip of a branch), you will see that too:

===
...
Layer repository git://git.yoctoproject.org/poky-contrib checked out into /home/alex/builds/poky-alex/layers/poky updated revision akanavin/sstate-for-all from 6b842ba55f996b27c900e3de78ceac8cb3b1c492 to aeb73e29379fe6007a8adc8d94c1ac18a93e68de
===

4. If the configuration has changed, you can bring it in sync with:

$ bin/bitbake-setup update

Note that it will also rename/preserve the existing build/conf directory, and print changes
in bitbake configuration (diff of content of build/conf/) if that has changed. I can't
at the moment think of anything more clever that is also not much more brittle or complex
to implement, but open to suggestions.

Terminology
===========

- 'top directory' means the place under which bitbake-setup reads and
writes everything. bitbake-setup makes a promise to not touch anything outside of
that, unless otherwise directed to by entries in settings (currently
there is one such setting for fetcher downloads for layers and config
registries). Top directory can be selected by an environment variable, a command line option,
or otherwise assumed to be ~/bitbake-builds/. If BBPATH is in environment
(e.g. we are in a bitbake environment), then the top directory is
deduced from that and doesn't need to be specified by hand.

- 'settings' means bitbake-setup operational parameters that are
global to all builds under a top directory. E.g. the location of
configuration registry, or where the bitbake fetcher should place the
downloads (DL_DIR setting). Settings are stored in a .conf file in ini
format just under the top directory.

- 'build' means a tree structure set up by 'bitbake-setup init',
consisting of, at least, a layers checkout, and a bitbake
build. It maps 1:1 to the json data it was constructed from, which is
called 'build configuration'. Build configurations are constructed from
generic configurations that may involve making one or more choices
about available options in them. Generic configurations are files, URLs
or are obtained from git repositories called 'config
registries', in which case they can be listed with 'bitbake-setup
list'. There can be multiple 'builds' under a top directory. Here are
two example generic configurations that showcase this:
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-options.conf.json
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-nested-configs.conf.json

- 'bitbake-setup status' will tell if a build is in sync with
the generic configuration it was made from. 'bitbake-setup update' will bring a build
in sync with a configuration if needed.

- 'bitbake build' means a particular sub-tree inside a build that
bitbake itself operates on, e.g. what is set in BBPATH/BUILDDIR
by oe-init-build-env. conf/* in that tree is 'bitbake configuration'.
Bitbake configurations are constructed from templates and fragments,
with existing mechanisms provided by oe-core. The configuration file
format is specified such that other mechanisms to set up a
bitbake build can be added; there was a mention of ability to specify
local.conf content and a set of layers directly in a configuration. I
think that scales poorly compared to templates and fragments, but I
made sure alternative ways to configure a bitbake build are possible
to add in the future :)

- 'source override' is a json file that can be used to modify revisions
and origins of layers that need to be checkout into a build (e.g.
when master branches need to be changed to master-next for purposes
of testing). Such a file is specified with a command-line option to 'init'
and an example can be seen here:
https://github.com/kanavin/bitbake-setup-configurations/blob/main/yocto-master-next.override.json

This commit includes fixes by
Ryan Eatmon <reatmon@ti.com> https://github.com/kanavin/bitbake/pull/1
Gyorgy Sarvari <skandigraun@gmail.com> https://github.com/kanavin/bitbake/pull/2
Johannes Schneider <johannes.schneider@leica-geosystems.com> https://github.com/kanavin/bitbake/pull/3 https://github.com/kanavin/bitbake/pull/5

(Bitbake rev: bf3542076cdd751da9de8745aa3f351f5c6b0c5f)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

0 files changed, 0 insertions, 0 deletions