summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/cookerdata.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: cookerdata/cooker: Restore original datastore upon client disconnectRichard Purdie2017-07-211-0/+9
| | | | | | (Bitbake rev: 67ae612ab890965357d24a9bd35610cf813b79ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server: Rework the server API so process and xmlrpc servers coexistRichard Purdie2017-07-211-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the way bitbake server works quite radically. Now, the server is always a process based server with the option of starting an XMLRPC listener on a specific inferface/port. Behind the scenes this is done with a "bitbake.sock" file alongside the bitbake.lock file. If we can obtain the lock, we know we need to start a server. The server always listens on the socket and UIs can then connect to this. UIs connect by sending a set of three file descriptors over the domain socket, one for sending commands, one for receiving command results and the other for receiving events. These changes meant we can throw away all the horrid server abstraction code, the plugable transport option to bitbake and the code becomes much more readable and debuggable. It also likely removes a ton of ways you could hang the UI/cooker in weird ways due to all the race conditions that existed with previous processes. Changes: * The foreground option for bitbake-server was dropped. Just tail the log if you really want this, the codepaths were complicated enough without adding one for this. * BBSERVER="autodetect" was dropped. The server will autostart and autoconnect in process mode. You have to specify an xmlrpc server address since that can't be autodetected. I can't see a use case for autodetect now. * The transport/servetype option to bitbake was dropped. * A BB_SERVER_TIMEOUT variable is added which allows the server to stay resident for a period of time after the last client disconnects before unloading. This is used if the -T/--idle-timeout option is not passed to bitbake. This change is invasive and may well introduce new issues however I believe the codebase is in a much better position for further development and debugging. (Bitbake rev: 72a3dbe13a23588e24c0baca6d58c35cdeba3f63) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Add a function to find TOPDIRRichard Purdie2017-07-211-0/+21
| | | | | | | | | | | | Finding the top level build directory is currently hard and relies on having a complete cooker being setup. Add a helper function which does the same thing without all the extra overhead. This is needed to be able to locate the bitbake lockfile and hence the socket for connecting clients in the new server model. (Bitbake rev: d196afe68032898c31a8599ca7d3ceba58d96b0a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: fix indentation and trailing spacesPaul Eggleton2017-07-181-5/+5
| | | | | | | | | Fix one incorrect indentation and several lines with trailing spaces. (Bitbake rev: 4dbf4f1907a32f303c2e9acf2f27817cf3c9ec38) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add MultiConfigParsed eventRichard Purdie2017-07-081-0/+2
| | | | | | | | | | There are some cases where the metadata needs to be aware a multiconfig build is happening and have access to the multiconfig data stores to merge data into the common build. This adds such an event allowing access to these datastores. (Bitbake rev: 160e47f5df90850e64dcb857c81a5039abc9235f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Implement LAYERSERIES_COMPAT and LAYERSERIES_CORENAMESRichard Purdie2017-06-121-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, someone trying a master branch of any given layer may or may not find it works with the version of OE-Core that they have picked. This patchset introduces LAYERSERIES_CORENAMES and LAYERSERIES_COMPAT_<layername> with the intention of allowing layers to indicate which versions of the core project they're compatible with. In general the master branch will use the next release codename as the value of LAYERSERIES_CORENAMES, meaning that the waterfall model of layer releases is supported with layers updating their LAYERSERIES_COMPAT_ values as they branch for release. Both variables support multiple strings and a given layer is accepted if there is overlap of any one value. This means a layer can be compatible with multiple core versions. Setting LAYERSERIES_COMPAT_ will be required by the Yocto Project Compatible v2 standard but the system will not error if its not set at this point. This should make it clear when a given layer is unmaintained and untested with new releaes of OE-Core, a common user pain point. (Bitbake rev: 92c49b9b1a16dfd35444db8143bd4cae4cda70cc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: add BB_CMDLINE to enable access to UI command line with memresPaul Eggleton2017-06-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | In OpenEmbedded's buildhistory class we want access to the bitbake command line that launched the build, and up to now we were simply using sys.argv from within the event handler to get that. Unfortunately that doesn't work in memory resident mode, since the event handler is naturally executing within the server and thus will give you the command that launched the bitbake server which is much less interesting. Add a dynamic variable BB_CMDLINE to provide access to this, set from sys.argv within the UI process in updateToServer(). (Note that BB_CMDLINE isn't currently passed through to the worker, so this is only really readable from event handlers plus any explicit getVariable calls - in theory an observe-only UI could read it for example.) Part of the fix for [YOCTO #11634]. (Bitbake rev: 8ad31a3eff5cfcb7b1c462578a582dafcbc426d4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: fix variable history not showing in bitbake -e with memresPaul Eggleton2017-06-121-1/+1
| | | | | | | | | | | | | | | | | | CookerConfiguration sets the "tracking" option to True when -e is specified in order to have history tracking enabled in the datastore so that we can show variable history (which isn't enabled by default for performance reasons), however in memory resident mode this wasn't doing anything because it was happening on the UI side only. We do have a mechanism for updating the cooker configuration in the server, but the tracking option wasn't being included in the list of options we updated, so we just need to add this option to fix the issue. Fixes [YOCTO #10730]. (Bitbake rev: 57d4977555cf892b15dd0302dfe261fe37d49327) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Add support for BBFILES_DYNAMICPatrick Ohly2017-06-091-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | BBFILES_DYNAMIC can be used to activate content only when some other layers are present. The other layers are identified by the collections that they define. The main use case is to avoid .bbappends without the corresponding .bb file in layers that want to modify other layers via .bbappends without introducing a hard dependency on those other layers. .bb files could also be handled via BBFILES_DYNAMIC. Entries in BBFILES_DYNAMIC must have the form <collection name>:<filename pattern>. Example usage: BBFILES_DYNAMIC += " \ clang-layer:${LAYERDIR}/bbappends/meta-clang/*/*/*.bbappend \ core:${LAYERDIR}/bbappends/openembedded-core/meta/*/*/*.bbappend \ " Parsing is aborted when invalid entries are found with an error message like this: ERROR: BBFILES_DYNAMIC entries must be of the form <collection name>:<filename pattern>, not: /work/my-layer/bbappends/meta-security-isafw/*/*/*.bbappend /work/my-layer/bbappends/openembedded-core/meta/*/*/*.bbappend Based on a patch by Richard Purdie. (Bitbake rev: 04f8bd50aa04b12cf91dd6a3154527ad2c24695c) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: add option to run all tasks in specific build targetMatthew McClintock2017-03-131-0/+1
| | | | | | | | | | | | | | | | | | | | For example: $ bitbake core-image-minimal --runall unpack ... NOTE: Tasks Summary: Attempted 326 tasks of which 88 didn't need to be rerun and all succeeded. $ bitbake core-image-minimal --runall patch NOTE: Tasks Summary: Attempted 453 tasks of which 332 didn't need to be rerun and all succeeded. This can replace fetchall as well: $ bitbake core-image-minimal --runall fetch NOTE: Tasks Summary: Attempted 135 tasks of which 119 didn't need to be rerun and all succeeded. (Bitbake rev: 7c0fa6ba66cdb956b37d94055307cde857633df9) Signed-off-by: Matthew McClintock <msm-oss@mcclintock.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Add BB_LIMITEDDEPS supportRichard Purdie2017-01-201-0/+1
| | | | | | | | | | | When we're running with bitbake -b, BB_TASKDEPDATA is incorrect and limited. We really need a way to know this from the metadata and this new variable provides this in worker context. This means existing code can stop having to guess. (Bitbake rev: 05763bc886024dcce2ce6b3060fb00abf79a9402) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Convert multiconfig to use BB_CURRENT_MCRichard Purdie2016-12-221-2/+3
| | | | | | | | | | | | | | | | | People are struggling with multiconfig as the point the conf file is injected into the data store is not what people expect. We can't really use a post config since that is too late and we can't really use a pre config file since that is too early. In OE terms, we need something right around the local.conf point so it behaves in a similar way. A way to handle this is to set the new variable BB_CURRENT_MC to be the currently selected multiconfig, then the metadata itself can choose when to inject the approriate configuration. (Bitbake rev: 518b9015c2be8d3894277a8e54890d6f04d656c0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: provide a means to shut down from the client in memres modePaul Eggleton2016-12-141-0/+1
| | | | | | | | | | In memory resident mode we don't really want to actually shut down since it's only the client going away. (Bitbake rev: 74db369c46043116359101cab70486afd82372c0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVarFlag callsJoshua Lock2016-11-301-1/+1
| | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (Bitbake rev: c19baa8c19ea8ab9b9b64fd30298d8764c6fd2cd) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: remove True option to getVar callsJoshua Lock2016-11-301-5/+5
| | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata.py: Catch BBHandledException, preventing a backtrace in ↵Mark Hatle2016-09-211-1/+1
| | | | | | | | | | | | | an event The event handling 'Exception' was catching and triggering a backtrace. This trace was obscuring any errors from an event handler that had raised the BBHandledException, which should indicate do not print additional information. (Bitbake rev: 51ca5193a5674b27d816140b0254f485912177a2) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: allow multiple passes of config re-parsingMarkus Lehtonen2016-09-161-1/+8
| | | | | | | | | [YOCTO #10188] (Bitbake rev: 07a03a1290fd206df2b40ffc28381b5b3c10ba4a) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: fire ConfigParsed event after re-parseMarkus Lehtonen2016-09-161-0/+1
| | | | | | | | | [YOCTO #10188] (Bitbake rev: ec1c951a4ee0c33acdde29e578f79ad719a34aca) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata/ast: Fail gracefully if event handler function is not foundMarkus Lehtonen2016-09-021-0/+3
| | | | | | | | | [YOCTO #10186] (Bitbake rev: 107c47c4e6de6a596cf1aeca5c18dbc1c5b44dc4) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Initial multi-config supportRichard Purdie2016-08-181-25/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the notion of supporting multiple configurations within a single build. To enable it, set a line in local.conf like: BBMULTICONFIG = "configA configB configC" This would tell bitbake that before it parses the base configuration, it should load conf/configA.conf and so on for each different configuration. These would contain lines like: MACHINE = "A" or other variables which can be set which can be built in the same build directory (or change TMPDIR not to conflict). One downside I've already discovered is that if we want to inherit this file right at the start of parsing, the only place you can put the configurations is in "cwd", since BBPATH isn't constructed until the layers are parsed and therefore using it as a preconf file isn't possible unless its located there. Execution of these targets takes the form "bitbake multiconfig:configA:core-image-minimal core-image-sato" so similar to our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND. Implementation wise, the implication is that instead of tasks being uniquely referenced with "recipename/fn:task" it now needs to be "configuration:recipename:task". We already started using "virtual" filenames for recipes when we implemented BBCLASSEXTEND and this patch adds a new prefix to these, "multiconfig:<configname>:" and hence avoid changes to a large part of the codebase thanks to this. databuilder has an internal array of data stores and uses the right one depending on the supplied virtual filename. That trick allows us to use the existing parsing code including the multithreading mostly unchanged as well as most of the cache code. For recipecache, we end up with a dict of these accessed by multiconfig (mc). taskdata and runqueue can only cope with one recipecache so for taskdata, we pass in each recipecache and have it compute the result and end up with an array of taskdatas. We can only have one runqueue so there extensive changes there. This initial implementation has some drawbacks: a) There are no inter-multi-configuration dependencies as yet b) There are no sstate optimisations. This means if the build uses the same object twice in say two different TMPDIRs, it will either load from an existing sstate cache at the start or build it twice. We can then in due course look at ways in which it would only build it once and then reuse it. This will likely need significant changes to the way sstate currently works to make that possible. (Bitbake rev: 5287991691578825c847bac2368e9b51c0ede3f0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Simplify prefiles/postfilesRichard Purdie2016-08-181-3/+5
| | | | | | | | | The current codepaths are rather confusing. Stop passing these as parameters and use the ones from when the object is created. (Bitbake rev: 8c992c148d9619b10eeae8bbd9376ecf408037a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: print an error if layer dir does not existMarkus Lehtonen2016-06-031-0/+4
| | | | | | | | | | | Makes it easier for user to identify problems, e.g. typos, in BBLAYERS. [YOCTO #9507] (Bitbake rev: 32c9689e4b492dc5821749e284e397d717af2a6c) Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Provide LAYERDIR_RE for layer.confChristopher Larson2016-05-191-2/+7
| | | | | | | | | | | | | This variable is a regex-escaped version of LAYERDIR, for safer use in BBFILE_PATTERN, so as to avoid issues with regex special characters in the layer path. [YOCTO #8402] (Bitbake rev: 72900522778b6ff08b135bf8bb97dff3f1a20bd9) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Fix log messages and ensure we exitRichard Purdie2016-05-131-1/+2
| | | | | | | | | The string formatting wasn't correct and we should exit if we hit errors here similar to the other exception handlers. (Bitbake rev: b90a16408a5c45ce5312384f278e19d09f8dda4d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata.py: remove slash in the endRobert Yang2016-03-311-0/+2
| | | | | | | | | | | | | | | It's very possible that we added layer as: BBLAYERS += "/path/to/meta/" then there would be warning: WARNING: No bb files matched BBFILE_PATTERN_core '^/path/to/meta//' This patch can fix the problem. (Bitbake rev: 2b1cb21d18fb18399e682021b866babeced9a4aa) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Improve handling of ParseErrorRichard Purdie2016-03-291-1/+3
| | | | | | | | | | | | | | | | If local.conf contains an invalid line, e.g.: APPEND += " igor" (note the leading space) then nasty tracebacks are shown which confuse the user. Change so the parse error is simply shown without a traceback, improving the user experience. [YOCTO #9332] (Bitbake rev: 148aa1fb45dcb37a756a08301a7daf270e753180) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main/runqueue: Add --setscene-only option to bitbakeRichard Purdie2016-01-111-0/+1
| | | | | | | | | | | | | Its turning out that we really need a way to have bitbake just run the setscene tasks but not any real tasks, particularly for SDK operations. Add an option for this since its pretty straight forward. This allows various nasty workarounds in OE-Core to be removed. (Bitbake rev: e4a2aafa1650a227a04d92a8a0b31efaed2c310e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: ast/event/utils: Improve tracebacks to include file and line ↵Richard Purdie2015-12-181-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | numbers more correctly Currently bitbake tracebacks can have places where the line numbers are inaccurate and filenames may be missing. These changes start to try and correct this. The only way I could find to correct line numbers was to compile as a python ast, tweak the line numbers then compile to bytecode. I'm open to better ways of doing this if anyone knows of any. This does mean passing a few more parameters into functions, and putting more data into the data store about functions (i.e. their filenames and line numbers) but the improvement in debugging is more than worthwhile). Before: ---------------- ERROR: Execution of event handler 'run_buildstats' failed Traceback (most recent call last): File "run_buildstats(e)", line 43, in run_buildstats(e=<bb.build.TaskStarted object at 0x7f7b7c57a590>) NameError: global name 'notexist' is not defined ERROR: Build of do_patch failed ERROR: Traceback (most recent call last): File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task return _exec_task(fn, task, d, quieterr) File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task event.fire(TaskStarted(task, logfn, flags, localdata), localdata) File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire fire_class_handlers(event, d) File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers execute_handler(name, handler, event, d) File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler ret = handler(event) File "run_buildstats(e)", line 43, in run_buildstats NameError: global name 'notexist' is not defined ---------------- After: ---------------- ERROR: Execution of event handler 'run_buildstats' failed Traceback (most recent call last): File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats(e=<bb.build.TaskStarted object at 0x7efe89284e10>): if isinstance(e, bb.build.TaskStarted): > trigger = notexist pn = d.getVar("PN", True) NameError: global name 'notexist' is not defined ERROR: Build of do_package failed ERROR: Traceback (most recent call last): File "/media/build1/poky/bitbake/lib/bb/build.py", line 560, in exec_task return _exec_task(fn, task, d, quieterr) File "/media/build1/poky/bitbake/lib/bb/build.py", line 497, in _exec_task event.fire(TaskStarted(task, logfn, flags, localdata), localdata) File "/media/build1/poky/bitbake/lib/bb/event.py", line 170, in fire fire_class_handlers(event, d) File "/media/build1/poky/bitbake/lib/bb/event.py", line 109, in fire_class_handlers execute_handler(name, handler, event, d) File "/media/build1/poky/bitbake/lib/bb/event.py", line 81, in execute_handler ret = handler(event) File "/media/build1/poky/meta/classes/buildstats.bbclass", line 143, in run_buildstats trigger = notexist NameError: global name 'notexist' is not defined ---------------- (Bitbake rev: 1ff860960919ff6f8097138bc68de85bcb5f88b0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: preserve pre and post configsEd Bartosh2015-10-271-0/+2
| | | | | | | | | | | | | | | | | | | Additional config files passed to bitbake server with --read and --postread options are rewritten by client bitbake even if it doesn't use those options. This is a show stopper for toaster as toaster command line builds are based on the assumption that server is aware of toster configs, provided by --postread option. This behaviour is fixed by preserving values of --read and --postread options when bitbake server starts and restoring them if client bitbake doesn't explicitly specify them. (Bitbake rev: 02c64f7487ca8ec5d32c440f5002c4b8f64b76a6) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Rename BBPKGS -> BBTARGETSRichard Purdie2015-10-011-2/+2
| | | | | | | | | | | | | | | | | | BBPKGS is a confusing name from before we tried to straighten out our terminology. Its also a mostly unknown variable that isn't in wide use. I've been asked about it recently and before people start relying more heavily on it, I'd like to rename it BBTARGETS which better describes what it does. Its not currently in the manuals, I'd prefer to document it under the better name. I've not provided any migration path for the variable since I believe its unused currently. It allows the targets to built to be specified from a conf file in addition to those on the commandline. (Bitbake rev: f60c6a2172bceeb5682dcb738a02c4bf26176566) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.cookerdata: include useful traceback for ExpansionError/ParseErrorChristopher Larson2015-09-041-1/+10
| | | | | | | | | | Show the user only the portion of the traceback which was from the metadata, nothing from bitbake's internal calls. (Bitbake rev: c45054aef03393fa0bf70e853ddcfc55988493cf) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: ensure prefile/postfile can work in memory resident modePaul Eggleton2015-08-191-1/+2
| | | | | | | | | | | | The prefile/postfile options weren't working in memory resident mode because they weren't being passed through to the server, so ensure that they do get passed through and that the server is reset when the values come through. (Bitbake rev: a3f7dc042fc7b1c308bfd248431930eb8ba50326) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.cookerdata: don't show traceback for ParseError/ExpansionErrorChristopher Larson2015-08-011-2/+5
| | | | | | | | | | | Tracebacks are of extremely limited usefulness in this context. The exceptions carry the necessary context already, and the user doesn't care about the calls in bitbake internals that led to an expansion or parse failure. (Bitbake rev: 9b95fa94eaae452ac7814f1e67c2f7a6314c52f1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Add explict getVar param for (non) expansionRichard Purdie2015-06-231-3/+3
| | | | | | | | | | | | | | Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Allow ~ in bblayersEd Bartosh2015-04-191-0/+3
| | | | | | | | | | Implemented processing of ~ in bblayer's paths if HOME environment variable is approved. (Bitbake rev: 3b8a656d3ccb543c32696229184ebf12237ad38e) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bin/bitbake: Create bitbake_main APIEd Bartosh2015-03-251-3/+3
| | | | | | | | | | Moved most of functionality of bin/bitbake to lib/bb/main.py to be able to call bitbake from python code. (Bitbake rev: d377f7f88d73f4e5d2dffef03d6acee809827ac6) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command/cooker/knotty: Fix memres handling of command environment ↵Richard Purdie2015-01-211-2/+2
| | | | | | | | | | | | | | | | changes If the environment changes, we need memory resident bitbake to adapt to those changes. This adds in functionality to handle this alongside the configuration option handling code. This means that the common usage: MACHINE=X bitbake Y now works with the memory resident server. (Bitbake rev: 4d1343010da757a0c126bc22475354da44aaf8e3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: add option to write offline event log fileAlexandru DAMIAN2014-12-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a "-w/--write-log" option to bitbake that writes an event log file for the current build. The name of the file is passed as a parameter to the "-w" argument. If the parameter is the empty string '', the file name is generated in the form bitbake_eventlog_DATE.json, where DATE is the current date and time, with second precision. The "-w" option can also be supplied as the BBEVENTLOG environment variable. We add a script, toater-eventreplay, that reads an event log file and loads the data into a Toaster database, creating a build entry. We modify the toasterui to fix minor issues with reading events from an event log file. Performance impact is undetectable under no-task executed builds. (Bitbake rev: 1befb4a783bb7b7b387d4b5ee08830d9516f1ac2) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Ensure commandline parameters are updated in memres serverRichard Purdie2014-09-231-0/+11
| | | | | | | | | | | | | | | | When using options like -k, -f, -v and so on with the memory resident server, they'd currently only be set on the initial values passed to the original command. This ensures they now match those specified on the commandline for the options where this makes sense. To make this work, a command to update the options on the server side is required so this is added. [YOCTO #5292] (Bitbake rev: 1c75cc4d0c8b606c1fe76e6bf60bf6a32298b105) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Improve error handlingRichard Purdie2014-06-011-2/+5
| | | | | | | | | | | | | If we see errors during parsing, firstly its bad to show a traceback for an expansion error so lets suppress this. Secondly, raise a BBHandledException instead of a SystemExit to show we've informed the user about the condition (printing a traceback in the default unknown case). (Bitbake rev: e01988d9a1b7c40e31161c6ce7b85c4405671068) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Force -S option to take a parameterRichard Purdie2014-03-271-1/+1
| | | | | | | | | | | | | There is no easy way to make this change. We really need parameters for the -S (dump signatures) handling code. Such a parameter can then be used within the codebase to handle the signatures in different ways. For now, "none" is the recommended default and "printdiff" will execute the new (and more expensive) comparison algorithms. (Bitbake rev: b9873588696507dfb6aade6821f6f75cb9a19e0a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake.lock: Add host:port to bitbake.lock for memres serverJason Wessel2013-12-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The idea is to build on the --status-only option for bitbake and expose a mechanism where the oe init scripts can easily switch between memres server and the non-memres server. In the case of the standard oe init script the following can shut down the server: if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only if [ $? = 0 ] ; then echo "Shutting down bitbake memory resident server with bitbake -m" BBSERVER=`cat bitbake.lock` bitbake -m fi fi A similar function can be used to automatically detect if the server is already running for the oe memres init script. This new functionality allows for the memres init script to be started in a new shell and connect up to an alaready running server without seeing the error of trying to start the server multiple times. (Bitbake rev: b1803958de8d7c3c3279841e38604a08dc2316cc) Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker,xmlrpc,servers: implement CookerFeaturesAlexandru DAMIAN2013-09-221-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implementing feature set selection that allows a client to enable specific features in the server at connection time. Only enabling of features is supported, as there is no way to safely remove data loaded into the cooker. Once enabled, a feature will remain enabled for the life of the cooker. Client-server connection now supports specifying the feature set required by the client. This is implemented in the Process server using a managed proxy list, so the server cooker will now load dynamically needed features based on what client connects to it. In the XMLRPC server the feature set is requested by using a parameter for registerUIHandler function. This allows observer-only clients to also specify features for the server. The server code configuration now is completly separated from the client code. All hardcoding of client knowledge is removed from the server. The extra_caches is removed as the client can now specify the caches it needs using the feature. The UI modules now need to specify the desired featureSet. HOB is modified to conform to the featureSet specification. The only feature available is CookerFeatures.HOB_EXTRA_CACHES which forces loading the bb.cache_extra:HobRecipeInfo class. (Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Allow bblayers.conf to be found using BBPATHRichard Purdie2013-08-301-8/+15
| | | | | | | | | | | | | | | It should be possible to run a build anywhere on the filesystem and have bitbake find the correct build directory if its set somehow. The BBPATH variable makes perfect sense for this usage. Therefore use any available value of BBPATH to search for conf/bblayers.conf before walking the parent directory structure. This restores the option of being able to run bitbake from anywhere if the user has set things up to operate in that environment. (Bitbake rev: e86336b3fe245bc97fe74c9b9d6a21d38a536fb7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Set TOPDIR when using bblayers.confRichard Purdie2013-08-301-0/+3
| | | | | | | | | | | | | By definition, bblayers.conf is at the top of the build tree. We'd like to support running bitbake anywhere within that build tree but TOPDIR gets set to wherever cwd is. Change the code to reset TOPDIR to the top of the build directory. This shouldn't break anything but does make the system more usable. (Bitbake rev: b266db27de0bba19a418e4d42e870649136b116b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Improve message if BBPATH is unset and bblayers.conf ↵Richard Purdie2013-06-281-1/+6
| | | | | | | | | | | | | | | not found If BBPATH isn't set and bblayers.conf isn't found, improve the message shown to the user to help their understanding of what the problem might be. [YOCTO #3271] (Bitbake rev: 0e639f5cbc813c8d4719019cfdd4287e9a429610) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> A
* bitbake: bitbake: Add event mask flag supportBogdan Marinescu2013-06-141-1/+1
| | | | | | | | | | | | | | | Add a flag to event handlers which lists the events a given handler wishes to process. By default event handlers recieve all events but this means we can stop running code in many cases if we know it doesn't want the event. This is part of the fix for YOCTO #3812, but implements filtering only for class event handlers; the other part (events filter for UIs) will be the subject of a different patch. (Bitbake rev: 074003a4e7530a72863b9c685fc5c31b0f08c039) Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Split runqueue to use bitbake-workerRichard Purdie2013-06-141-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a pretty fundamental change to the way bitbake operates. It splits out the task execution part of runqueue into a completely separately exec'd process called bitbake-worker. This means that the separate process has to build its own datastore and that configuration needs to be passed from the cooker over to the bitbake worker process. Known issues: * Hob is broken with this patch since it writes to the configuration and that configuration isn't preserved in bitbake-worker. * We create a worker for setscene, then a new worker for the main task execution. This is wasteful but shouldn't be hard to fix. * We probably send too much data over to bitbake-worker, need to see if we can streamline it. These are issues which will be followed up in subsequent patches. This patch sets the groundwork for the removal of the double bitbake execution for psuedo which will be in a follow on patch. (Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cookerdata: Fix tinfoil by adding missinge extra_caches defaultRichard Purdie2013-05-301-0/+1
| | | | | | (Bitbake rev: d513153cac283aa4ec37135a9190f7a091b6c44b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: move extra cache collection out of cookerAlexandru DAMIAN2013-05-301-1/+0
| | | | | | | | | | | | | | The collection of the extra caching data should not be performed by the cooker, but supplied to it. This patch will also streamline the code for launching servers without a UI attached. Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com> (Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-14 15:42:33 +0000