summaryrefslogtreecommitdiffstats
path: root/bitbake
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: bitbake-user-manual: Removed and replaced broken linkKristi Rifenbark2017-07-041-2/+2
| | | | | | | | | | | | | Fixes [YOCTO #11675] THere was a "hambedded" link that was broken. The link was intended to give additional information on adding layers. Replaced with link about layers and replaced the wording. (Bitbake rev: 877a1f476212dc151b74ce0f1febdc48956ef0e9) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: Replaced bad linkKristi Rifenbark2017-07-041-1/+1
| | | | | | | | | | | | | Fixes [YOCTO #11675] There was a "hambedded" link that was broken. The link was intended to show the bitbake.conf file. I replaced it with the link to show the actual bitbake.conf file. (Bitbake rev: 3b8d101e564488ff1f5b27744404f9743da2b97d) Signed-off-by: Kristi Rifenbark <kristi.rifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: npm fetcher: fix unknown variable name.Ismo Puustinen2017-06-291-1/+1
| | | | | | | | | 'mirrortarball' is supposed to be a local variable to the function. (Bitbake rev: a457cbfb1f20a47db3978290921d0708cd96bd70) Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cache: don't insert PN into PACKAGESRoss Burton2017-06-281-2/+2
| | | | | | | | | | | | | | | | | | | | The cache code currently inserts PN into the package list if it isn't already present. Whilst this ensures that the package list contains something which is important for native recipes that don't set PACKAGES, it causes confusing behaviour where a normal recipe doesn't have PN in PACKAGES: for example adding dhcp to IMAGE_INSTALL will parse successfully but fail at rootfs time as the dhcp recipe doesn't generate a dhcp package. Solve this by only adding PN to the cache's package list if the package list is empty. This results in the package list for recipes such as DHCP being correct, but native recipes continue to have just PN in the list as before. [ YOCTO #5533 ] (Bitbake rev: df31a88786ce5bd7708ff14e1379dc2a58a8c0cf) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: test 'commit' first in get_vcs_referenceDavid Reyna2017-06-281-2/+2
| | | | | | | | | | | | | | The 'commit' value should be tested and used first when resolving the ref for a layer, since that is an explicit override in each layer index 'layeritem' record. If should take precedence over more-global 'branch' and 'release' values, instead of being last. [YOCTO #11515] (Bitbake rev: b303365ca3a1ba5fb8c6839180f1ec26df60c6ce) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: large package set breaks sqlite queryDavid Reyna2017-06-281-4/+9
| | | | | | | | | | | | | | If you build a project with a large package set, you will get a crash in "views.py" when the dashboard attempts to fetch the package set to calculate the package count and size. This is a sqlite limitation, and it fails with as few as 1220 packages. [YOCTO #11717] (Bitbake rev: 02cb2b7f7ff594de75a404396f39a2428750c798) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: Add distro selection supportDavid Reyna2017-06-2812-3/+325
| | | | | | | | | | | | | | Add the ability to select a distro in the project page, based on values from the Layer Index. Add a distro selection page with the add layer feature, based on the add machine page. [YOCTO #10632] (Bitbake rev: a156a4eff67cdc3943494f5be72b96e3db656250) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: git clone progress barDavid Reyna2017-06-286-1/+98
| | | | | | | | | | | | | | If a project has a lot of additional layers, the build may appear to hang while those layers are checked out. This patch adds a clone progress bar that is visible before the parsing progress appears. [YOCTO #9916] (Bitbake rev: 0c94d947b74c4dee23d7b9d255facd3cf839ccbe) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: address Django-1.10 API deprecationsDavid Reyna2017-06-2810-82/+105
| | | | | | | | | | | | | | | | | | | | | | | | | There are four main API deprecations in Django-1.10: (a) String view arguments to url() must be replaced by the explicit class reference (b) New TEMPLATES stucture in settings.py consolidates TEMPLATE_DIRS, TEMPLATE_CONTEXT_PROCESSORS, TEMPLATE_LOADERS, TEMPLATE_STRING_IF_INVALID, and TEMPLATE_DEBUG (c) patterns() wrapper in url() is removed, with urlpatterns now a simple list (d) NoArgsCommand in commands() must be replace by BaseCommand, and handle_noargs() changed to handle() Also, the Django version checker must be updated to accept two digit sub-version numbers (e.g. "1.8" < "1.10") [YOCTO #11684] (Bitbake rev: e4c7a94fac7a53fc146387a57e5a09b9ec3caca0) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-selftest: add bb.tests.event to bitbake-selftestJair Gonzalez2017-06-231-0/+1
| | | | | | | | | | | | | Tests for bitbake event module were created on bb.tests.event. This change is to include them on the default test list in bitbake-selftest script. [YOCTO #10368] (Bitbake rev: 22cc318be1aff16f1b653ff0650fe71d0e9f60bb) Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests: create unit tests for event moduleJair Gonzalez2017-06-231-0/+377
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a new unit test module (bb.tests.event) for bitbake event. It includes the following items: - Client and server stubs setup - Testing the module's main functions including: - get_class_handlers - set_class_handlers - clean_class_handlers - enable_threadlock - disable_threadlock - get_handlers - set_handlers - execute_handler - fire_class_handlers - print_ui_queue - fire_ui_handlers - fire - fire_from_worker - register - remove - register_UIHhandler - unregister_UIHhandler - Testing event handling using: - class Event(object) - class OperationStarted(Event) - class OperationCompleted(Event) - class OperationProgress(Event) - class ConfigParsed(Event) [YOCTO #10368] (Bitbake rev: 0be3ad391adc73cc0dff81bd0ed7874f2c6a00be) Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: remove mapping for deleted event handlersJair Gonzalez2017-06-231-1/+5
| | | | | | | | | | | | | | | | | The current remove method for class event handlers does not update the event mapping after an event handler is deleted from the main dictionary. This change enhances the remove method by also updating the event mapping accordingly. This was detected after creating the bb.tests.event module. [YOCTO #10368] (Bitbake rev: 839f83dea1e99d1d182b138e5aea1df1c45a92fc) Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch: fix handling of files with incorrect checksums from a premirrorJoshua Lock2017-06-221-0/+9
| | | | | | | | | | | Ensure that when an item fetched from a premirror has an invalid checksum the fetcher falls back to the usual logic of trying the upstream and any configured mirrors. (Bitbake rev: 022adb30dbb0df764c9fb515918cb9a88e4f8d6f) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: drop some unused eventsPaul Eggleton2017-06-221-21/+0
| | | | | | | | | | These were all used by support code for the Hob UI which has been removed - nothing will currently fire or respond to these events. (Bitbake rev: 03ba8db0595723286ad462fa197f862e1efdf0de) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: noweb should init databaseDavid Reyna2017-06-221-15/+35
| | | | | | | | | | | | | | | | | | | | When the 'noweb' option for Toaster is used, perform the database check/create if the Toaster database does not yet exist. This will allow Toaster to not fail if the first use is with 'noweb'. This avoids potentially clashing database updates if there are multiple overlaping 'noweb' sessions (for example with a CI system). If the user wished to update the database, they can either use the explicit "lsupdate" command or (re)start a web hosted Toaster session (which is gated by the webserver's PID). [YOCTO #11378] (Bitbake rev: 910b96b9894c712aa32b5d4dadda88b766d86e35) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: get_last_build_id not called correctlyDavid Reyna2017-06-221-6/+6
| | | | | | | | | | | | | The method "get_last_build_id" missing the "()" in several calls. This has been failing silently with Django 1.8 but intermittently crash with Django 1.9. [YOCTO #11570] (Bitbake rev: c29e137a6e65317b8a36804126511c2137878715) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: add getMessage to MockEventDavid Reyna2017-06-221-0/+6
| | | | | | | | | | | | | The MockEvent needs to not only stand in for Toaster and Bitbake quick events, it also needs to stand in for LogRecord, and for that it needs to provide the new getMessage method. [YOCTO #11440] (Bitbake rev: d1ac359d460b1abe9815f323b3fd4cd0231cde6c) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: fail on layers with sub-layerDavid Reyna2017-06-221-3/+8
| | | | | | | | | | | | | | | | | | | | | For layers directories with sub-layers (for example meta-intel), Toaster breaks when trying to exact match a dependency event for that sub-layer against the top level layer directory paths. Given that top the level layer directory paths are unique, adding a test to see if the dependency path is a subset of a top level path is also unique. Also, since a warning was issued the processing should not fail on the assert but instead should continue and gather the recipes and tasks. [YOCTO #11149] (Bitbake rev: 6f1a42af19220a6a8d75790893da995e84ecd7aa) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: add ID's to build menu linksDavid Reyna2017-06-221-10/+10
| | | | | | | | | | | | | Add IDs to the build page menu lines, for example "Tasks", "Recipes", "Time", "CPU Time", "Disk I/O", and so forth. This will support test automation. [YOCTO #11337] (Bitbake rev: 49d11490fad683a96f71de21f55035398ad8247a) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: add ID's to navigation linksDavid Reyna2017-06-221-3/+5
| | | | | | | | | | | | Add IDs to the navigation bar links "All builds", "All projects", and "Documentation. This will support test automation. [YOCTO #11335] (Bitbake rev: 5cbf6f67994fc42dae72ece191e5c9630fc4b5af) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual: Updated BBLAYERS_FETCH_DIR variable descriptionScott Rifenbark2017-06-151-2/+0
| | | | | | | | | | | | | Just discovered a minor issue in the bitbake manual - under the variable glossary entry for BBLAYERS_FETCH_DIR we say there's a default value of ${COREBASE} but that's not actually true in bitbake itself - there's no default in bitbake, that default is set in OE-Core. We can keep this in the YP ref manual but in the bitbake manual that sentence should be removed. (Bitbake rev: 7b4e81772ed8789521661a3b32b576c91276ae34) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: ensure graceful exit after exception during BuildCompleted ↵2.4_M1Paul Eggleton2017-06-141-2/+4
| | | | | | | | | | | | | | | | handler If an event handler for bb.event.BuildCompleted fails, we still need to call finishAsyncCommand() or else BitBake will just exit immediately without showing any error summary, or worse in the case of memory resident mode BitBake will hang and if you Ctrl+C to break out, the command won't be marked as finished which means that no further commands will be able to be executed until the server is manually restarted. (Bitbake rev: 5639faa3eef55cc476a82e810e61ca228cbdf221) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: fix always loading cache on every UI start with memresPaul Eggleton2017-06-141-5/+12
| | | | | | | | | | | | | | | | | | | | | The main point of memory resident bitbake is to avoid loading data unnecessarily on every bitbake invocation. Unfortunately the code that updated options from the UI was simply treating the fact that either of the "prefile" or "postfile" options were in the list of options passed in as an indication that the configuration was invalid, which was bad because these are always passed in. We only need to mark the configuration as invalid and thus reload it (and thus reload the cache) if the option value has actually changed. At the same time, the recently handled "tracking" option needs to be treated in a similar manner since the configuration needs to be reparsed if that has changed. Also, add a few extra debug messages to aid debugging this code in future. (Bitbake rev: 18dfd144d0da6ff662308ce4f9d135dc11412133) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: multiconfig fixJuro Bystricky2017-06-141-1/+1
| | | | | | | | | | Differentiate between fn and taskfn in "execute". This was somehow missed in "fakeroot" handling. (Bitbake rev: d848bff7cf78f63986467b51f701a998a480eb25) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake:process: flush stderr/stdout to logJuro Bystricky2017-06-141-36/+43
| | | | | | | | | | | | | | | | | When a process terminates, some messages may still remain in stdout or stderr and do not make it into the log file. In addition, the messages that do make it to the log file may end up in the log file in incorrect order. This patch flushes all messages into the log file after the process terminates. Some additional log flushing is also needed to keep the various messages showing up in the log file in proper order. [YOCTO#10785] (Bitbake rev: 1f6e6aa8262369eafc3bbf9f01f8d981f90becdf) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler.py: allow require or include with multiple parametersPatrick Ohly2017-06-131-8/+12
| | | | | | | | | | | | | "inherit" already allows inheriting more than one class in a single statement. The same also makes sense for "include" and "require", because then one can generate a list of files to be included dynamically also for the case that more than one file needs to be included. (Bitbake rev: 8d0a76f5a595dddf16b7268bae2c00ef5f568316) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler.py: allow require or include without parameterPatrick Ohly2017-06-131-0/+4
| | | | | | | | | | | | | | | | Writing .bbappends that only have an effect when some configuration variable like DISTRO_FEATURES is changed becomes easier when allowing "include" or "require" without a parameter. The same was already allowed for "inherit". Then one can write in a .bbappend: require ${@bb.utils.contains('DISTRO_FEATURES', 'foo', 'bar.inc', '', d)} (Bitbake rev: 8b39c6361758b96fce50a53a6dba8008cd7e6433) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Implement LAYERSERIES_COMPAT and LAYERSERIES_CORENAMESRichard Purdie2017-06-121-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, someone trying a master branch of any given layer may or may not find it works with the version of OE-Core that they have picked. This patchset introduces LAYERSERIES_CORENAMES and LAYERSERIES_COMPAT_<layername> with the intention of allowing layers to indicate which versions of the core project they're compatible with. In general the master branch will use the next release codename as the value of LAYERSERIES_CORENAMES, meaning that the waterfall model of layer releases is supported with layers updating their LAYERSERIES_COMPAT_ values as they branch for release. Both variables support multiple strings and a given layer is accepted if there is overlap of any one value. This means a layer can be compatible with multiple core versions. Setting LAYERSERIES_COMPAT_ will be required by the Yocto Project Compatible v2 standard but the system will not error if its not set at this point. This should make it clear when a given layer is unmaintained and untested with new releaes of OE-Core, a common user pain point. (Bitbake rev: 92c49b9b1a16dfd35444db8143bd4cae4cda70cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-user-manual-metadata.xml: fix a typoMing Liu2017-06-121-1/+1
| | | | | | | | | 'buildCompleted' should be 'BuildCompleted'. (Bitbake rev: 9bd96cf938f746ace8d157f6b1b6c2550798085f) Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: prevent extra logger from being enabled for tinfoilPaul Eggleton2017-06-122-2/+10
| | | | | | | | | | | | | | | tinfoil sets up its own logger by default, but if and when we initialise the UI (by default knotty) will also set one up, leading to duplicated messages specifically from tasks. To avoid this, rather than adding some kind of parameter, just check if there is already a logger outputting to stdout/stderr and if so, skip adding our own. Part of the fix for [YOCTO #11275]. (Bitbake rev: 66d866745f35468d1540a793d07e3a401298b84b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: fix duplication of log messagesPaul Eggleton2017-06-122-3/+4
| | | | | | | | | | | | | | | Adding an additional logger in setup_bitbake() interacts poorly with the logger we have added by default in tinfoil's constructor, with the result that messages may be doubled or even tripled in tinfoil-using scripts. Disable adding this one when calling setup_bitbake() from tinfoil to avoid this problem. Part of the fix for [YOCTO #11275]. (Bitbake rev: 8a5bae76f91f2411187c638a42fa3c762052cf11) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command / cooker: drop Hob support commands and functionsPaul Eggleton2017-06-122-253/+0
| | | | | | | | | | | | | | | | | | | Drop a number of the commands and support functions that were created solely to support functionality in the now-removed Hob UI. In most cases we now have this functionality elsewhere e.g. to modify config files and recipes we have bb.utils.edit_metadata_file() and friends, and in OE we have oe.recipeutils.patch_recipe_file() and friends which build on top of the former. Additionally, some of it represented pretty egregious incursion of OE metadata-specific references into BitBake code. For now I have left in the find*File functions and commands as they are relatively generic and possibly still useful, but they might be removed in future. (Bitbake rev: a322f13183c66a28d93cc4bc3d839d95fc1d90f4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: add BB_CMDLINE to enable access to UI command line with memresPaul Eggleton2017-06-123-3/+9
| | | | | | | | | | | | | | | | | | | | | | | In OpenEmbedded's buildhistory class we want access to the bitbake command line that launched the build, and up to now we were simply using sys.argv from within the event handler to get that. Unfortunately that doesn't work in memory resident mode, since the event handler is naturally executing within the server and thus will give you the command that launched the bitbake server which is much less interesting. Add a dynamic variable BB_CMDLINE to provide access to this, set from sys.argv within the UI process in updateToServer(). (Note that BB_CMDLINE isn't currently passed through to the worker, so this is only really readable from event handlers plus any explicit getVariable calls - in theory an observe-only UI could read it for example.) Part of the fix for [YOCTO #11634]. (Bitbake rev: 8ad31a3eff5cfcb7b1c462578a582dafcbc426d4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: set config valid flags in initConfigurationData()Paul Eggleton2017-06-121-6/+2
| | | | | | | | | | | | | | If we set these flags here then we can not only tidy up some of the code that calls initConfigurationData() (and reset() which simply calls the former), we also avoid calling initConfigurationData() an extra time unnecessarily during startup (since setting baseconfig_valid = False in updateConfigOpts() also resulted in initConfigurationData() being called from updateCacheSync() at the end of the command.) (Bitbake rev: 74d2ef99a920aa9e70470ba8e600ea973a3c672a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: fix file changes not triggering task re-execution with memresPaul Eggleton2017-06-121-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | If you build a recipe, modify one of the files referred to in SRC_URI, then build it again, it should re-run do_fetch and the tasks that depend upon it to incorporate the modified file. With memory resident mode this was not working unless you restarted the server, because the mtime of the file was cached and thus we never checked the actual file. Clear out the mtime caches in buildTargets() to ensure this isn't an issue. I did a minor refactoring to break this out to its own function since we were already clearing out the build mtime cache here. (The test I used for this was . ./oe-init-build-env-memres, bitbake -c fetch mdadm, modify the comments at the top of one of the patches referred to in the recipe, and then bitbake -c fetch mdadm again and it should re-execute the fetch task ("... 0 didn't need to be re-run".) Fixes [YOCTO #10732]. (Bitbake rev: fe4f3487368bc35043a1ba464e1c69d2c9cbefc6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: fix variable history not showing in bitbake -e with memresPaul Eggleton2017-06-121-1/+1
| | | | | | | | | | | | | | | | | | CookerConfiguration sets the "tracking" option to True when -e is specified in order to have history tracking enabled in the datastore so that we can show variable history (which isn't enabled by default for performance reasons), however in memory resident mode this wasn't doing anything because it was happening on the UI side only. We do have a mechanism for updating the cooker configuration in the server, but the tracking option wasn't being included in the list of options we updated, so we just need to add this option to fix the issue. Fixes [YOCTO #10730]. (Bitbake rev: 57d4977555cf892b15dd0302dfe261fe37d49327) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Add support for BBFILES_DYNAMICPatrick Ohly2017-06-091-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BBFILES_DYNAMIC can be used to activate content only when some other layers are present. The other layers are identified by the collections that they define. The main use case is to avoid .bbappends without the corresponding .bb file in layers that want to modify other layers via .bbappends without introducing a hard dependency on those other layers. .bb files could also be handled via BBFILES_DYNAMIC. Entries in BBFILES_DYNAMIC must have the form <collection name>:<filename pattern>. Example usage: BBFILES_DYNAMIC += " \ clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \ core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \ " Parsing is aborted when invalid entries are found with an error message like this: ERROR: BBFILES_DYNAMIC entries must be of the form <collection name>:<filename pattern>, not: /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend Based on a patch by Richard Purdie. (Bitbake rev: 04f8bd50aa04b12cf91dd6a3154527ad2c24695c) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-layers: check layer dependencies before addingChang Rebecca Swee Fun2017-06-062-4/+23
| | | | | | | | | | | | | | | | | | | In the original implementation, "bitbake-layers add-layers <layer>" succeeded without error checking. This will further introduce failures in recipe parsing only when "bitbake" command is executed. Adding a meta layer without its dependency layer(s) should failed and exit the process gracefully. Added extra argument "-F" to force add a layer without checking layer dependency. [YOCTO #10913] (Bitbake rev: 705ab252e631903e6d2e46202b419a9e8adcd861) Signed-off-by: Phoong Stanley Cheong Kwan <stanley.cheong.kwan.phoong@intel.com> Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tinfoil: allow extra features to be passed to prepareAndy Voltz2017-06-051-1/+4
| | | | | | | (Bitbake rev: 2e35de1f19dc73a61a18a3eb186efede078d597d) Signed-off-by: Andy Voltz <andy.voltz@timesys.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: fix undefined dep in generatePkgDepTreeDataAndy Voltz2017-06-051-2/+2
| | | | | | | | | | The loop for populating the depends of the tree data should use dep instead of item. (Bitbake rev: 37c298b27e3f72b21513124237cfeffb22c0e2f0) Signed-off-by: Andy Voltz <andy.voltz@timesys.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Fix return value checks from subprocess.call()'sMikko Rapeli2017-06-052-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Python function subprocess.call() returns the return value of the executed process. If return values are not checked, errors may go unnoticed and bad things can happen. Change all callers of subprocess.call() which do not check for the return value to use subprocess.check_call() which raises CalledProcessError if the subprocess returns with non-zero value. https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module All users of the function were found with: $ git grep "subprocess\.call" | \ egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call' Tested similar patch on top of yocto jethro. Only compile tested core-image-minimal on poky master branch. (Bitbake rev: d2cf67bcaf001acb6be8fc5884fb450649849847) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ConfHandler: Require whitespace between export and variable nameOla x Nilsson2017-06-052-2/+23
| | | | | | | (Bitbake rev: 22bb7c9270f02ddae72e13d849375feee5f4a98b) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Make calc_taskhash match get_taskhash for file checksumsRichard Purdie2017-06-051-1/+2
| | | | | | | | | | | The code in these two functions is meant to be equivlanet in behaviour but isn't. Add in code to ensure files that don't exist are handled consistently by both functions. Users did report being able to generate tracebacks otherwise. (Bitbake rev: c11b4832879354b387a8c9c6a3096f2d621afeac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for removing arbitrary revs for shallowChristopher Larson2017-06-022-2/+82
| | | | | | | | | | | | | | | | | | | | In certain cases, it's valuable to be able to exert more control over what history is removed, beyond srcrev+depth. As one example, you can remove most of the upstream kernel history from a kernel repository, keeping predominently the non-publically-accessible content. If the repository is private, the history in that repo couldn't be restored via `git fetch --unshallow`, but upstream history could be. Example usage: # Remove only these revs, not at a particular depth BB_GIT_SHALLOW_DEPTH_pn-linux-foo = "0" BB_GIT_SHALLOW_REVS_pn-linux-foo = "v4.1" (Bitbake rev: 97f856f0455d014ea34c28b1c25f09e13cdc851b) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitannex: add support for shallow mirror tarballsChristopher Larson2017-06-022-3/+37
| | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating annex in that case. Also include annex files in the shallow tarball. (Bitbake rev: ca0dd3c95502b22c369fbf37f915f45e02c06887) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/gitsm: add support for shallow mirror tarballsChristopher Larson2017-06-022-10/+38
| | | | | | | | | | | | | When we're building from a shallow mirror tarball, we don't want to do anything with ud.clonedir, as it's not being used when we unpack. As such, disable updating the submodules in that case. Also include the repositories in .git/modules in the shallow tarball. It does not actually make the submodule repositories shallow at this time. (Bitbake rev: 6c0613f1f2f9d4f009545f82a9173e80396f9d34) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for keeping extra refs for shallowChristopher Larson2017-06-022-1/+63
| | | | | | | | | | | | | | | | | | | | By default, all unused refs (branches & tags) are removed from the repository, as shallow processing scales with the number of refs it has to process. Add the ability to explicitly specify additional refs to keep. This is particularly useful for recipes with custom checkout processes, or whose git-based versioning requires a tag be available (i.e. for `git describe --tags`). The new `BB_GIT_SHALLOW_EXTRA_REFS` variable is a space-separated list of refs, fully specified, and support wildcards. Example usages: BB_GIT_SHALLOW_EXTRA_REFS = "refs/tags/v1.0" BB_GIT_SHALLOW_EXTRA_REFS += "refs/heads/*" (Bitbake rev: 1771934cd9f8b5847c6fcae0a906fb99d6b0db16) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: support per-branch/per-url depths for shallowChristopher Larson2017-06-022-23/+73
| | | | | | | | | | | | | | | | Allow the user to explicitly adjust the depth for named urls/branches. The un-suffixed BB_GIT_SHALLOW_DEPTH is used as the default. Example usage: BB_GIT_SHALLOW_DEPTH = "1" BB_GIT_SHALLOW_DEPTH_doc = "0" BB_GIT_SHALLOW_DEPTH_meta = "0" (Bitbake rev: 9dfc517e5bcc6dd203a0ad685cc884676d2984c4) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/git: add support for shallow mirror tarballsChristopher Larson2017-06-022-15/+417
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support to the git fetcher for fetching, using, and generating mirror tarballs of shallow git repositories. The external git-make-shallow script is used for shallow mirror tarball creation. This implements support for shallow mirror tarballs, not shallow clones. Supporting shallow clones directly is not really doable for us, as we'd need to hardcode the depth between branch HEAD and the SRCREV, and that depth would change as the branch is updated. When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow mirror tarball. If the shallow mirror tarball cannot be fetched, it will try to fetch the full mirror tarball and use that. If a shallow tarball is to be used, it will be unpacked directly at `do_unpack` time, rather than extracting it to DL_DIR at `do_fetch` time and cloning from there, to keep things simple. There's no value in keeping a shallow repository in DL_DIR, and dealing with the state for when to convert the clonedir to/from shallow is not worthwhile. To clarify when shallow is used vs a real repository, a current clone is preferred to either tarball, a shallow tarball is preferred to an out of date clone, and a missing clone will use either tarball (attempting the shallow one first). All referenced branches are truncated to SRCREV (that is, commits *after* SRCREV but before HEAD are removed) to further shrink the repository. By default, the shallow construction process removes all unused refs (branches/tags) from the repository, other than those referenced by the URL. Example usage: BB_GIT_SHALLOW ?= "1" # Keep only the top commit BB_GIT_SHALLOW_DEPTH ?= "1" # This defaults to enabled if both BB_GIT_SHALLOW and # BB_GENERATE_MIRROR_TARBALLS are enabled BB_GENERATE_SHALLOW_TARBALLS ?= "1" (Bitbake rev: 5ed7d85fda7c671be10ec24d7981b87a7d0d3366) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: git-make-shallow: add script to make a git repo shallowChristopher Larson2017-06-022-0/+292
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This script will be used by the git fetcher to create shallow mirror tarballs. usage: git-make-shallow [-h] [--ref REF] [--shrink] REVISION [REVISION ...] Remove the history of the specified revisions, then optionally filter the available refs to those specified. positional arguments: REVISION a git revision/commit optional arguments: -h, --help show this help message and exit --ref REF, -r REF remove all but the specified refs (cumulative) --shrink, -s shrink the git repository by repacking and pruning While git does provide the ability to clone at a specific depth, and fetch all remote refs at a particular depth, the depth is across all branches/tags, and doesn't provide the flexibility we need, hence this script. Refs (branches+tags) can be filtered, as the process of history removal scales up rapidly with the number of refs. Even the existing `git fetch --depth=` is extremely slow on an upstream kernel repository with all the branches and tags kept. This uses the same underlying mechanism to implement the history removal which git itself uses (.git/shallow), and the results, when configured similarly, are in line with the results git itself produces with `fetch --depth`. (Bitbake rev: 0254020f0e1911c0eaf99111b91828d2a74a4ee1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-13 15:54:43 +0000