summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/cooker.py
Commit message (Collapse)AuthorAgeFilesLines
...
* cooker: show a useful message for ParsingFailureChris Larson2011-06-081-3/+6
| | | | | | | (Bitbake rev: a41c217c6e9195f8b9ea2de6e1d335b10b904558) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Fix debug message argumentsRichard Purdie2011-06-081-1/+1
| | | | | | | | (From Poky rev: 06bb59d) (Bitbake rev: b420e865f6dfb04b58c1dbc5f1b5332b137f49ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Cleanup bitbake server init process to be clearer to followRichard Purdie2011-06-081-6/+5
| | | | | | | | Create a standard format server class instance with method calls for each step in the server setup. There should be enough hooks for each of the different server types. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cache: Implement multiple extra cache fields request supportLiping Ke2011-06-071-8/+8
| | | | | | | | | | | | | | This patch is to support extra cache. If user needs to request extra cache fields besides CoreRecipeInfo fields, just add a new XXXRecipeInfo class definition as Hob Does. Currently supported Extra RecipeInfo name is an array. We can support multiple extra cache fields at the same time besides CoreRecipeInfo which is needed by all clients. (Bitbake rev: 077657e50ad032c0fa876bf54e9802af2686e0fb) Signed-off-by: Liping Ke <liping.ke@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Introduce new param caches_array into Cache impl.Liping Ke2011-06-071-8/+44
| | | | | | | | | | | | | | When using hob ui interface, we need extra cache fields. We will save ui required extra cache fields into a separate cache file. This patch introduce this caches_array parameter. It will be used in the extra cache implementation (following patch). Caches_array at least contains CoreRecipeInfo. If users need extra cache fields support, such as 'hob', caches_array will contain more relevant elements such as HobRecipeInfo. (Bitbake rev: d50389ae692377c957afec7c846fc2ce2c070a09) Signed-off-by: Liping Ke <liping.ke@intel.com>
* bitbake: use layer priority when applying bbappendsPaul Eggleton2011-06-011-10/+14
| | | | | | | | | | | | | If the priority of a layer has been specified with BBFILE_PRIORITY_ then use that to sort the list of BBFILES entries, which will affect the order in which .bbappend files are applied. Fixes [YOCTO #1125] (Bitbake rev: a8ab0af776ba20c83832215054180fbd15c274c0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/codeparser: Improve cache handlingRichard Purdie2011-05-271-0/+1
| | | | | | | | | | | | | | | | | | | The current codeparser cache handling hurts performance badly even over a couple of cores and certainly on many core systems, it can spent huge amounts of time in the codeparser cache save functions. This patch reworks the cache handling so that each parsing thread saves out its own "differences" file compared to any existing core cache and then the main bitbake thread picks these up and merges things back together. This was tested on systems with small and large numbers of cores and was found to perform orders of magnitude better in all cases despite the more complex code. (Bitbake rev: 9f27563d66523f5af1028f173d53ee75e0877d46) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Ensure BBFILES is processed in orderRichard Purdie2011-05-271-3/+8
| | | | | | | | | | | | | | | The files found by collect_bbfiles should be processed in order but due to being processed using python's set(), the order was not being preserved. Use a list instead as whilst the code is slightly more ugly, order is preserved. Addresses [YOCTO #1100] Acked-by: Darren Hart <dvhart@linux.intel.com> (Bitbake rev: c12dd868368bdc3a4f800e075a30c67edca28d47) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Fix -b option regexp handlingRichard Purdie2011-05-271-7/+6
| | | | | | | | | | | bitbake -b core-image was showing no matches when it should list all targets containing the fragment "core-image". This patch only calls os.path.abspath() on things that look like paths and passed the path around more consistently to various functions to get this use case working again. (Bitbake rev: 6e9119c42d3f4cb3dab3d25856eee8b4bf4ad94f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker.py: Don't show spurious warnings for collections of .bbappend filesRichard Purdie2011-05-251-1/+18
| | | | | | | | | | | | | | Seeing warnings like: WARNING: No bb files matched BBFILE_PATTERN_yocto '^/xxx/meta-yocto/' are not encouraging to users and we shouldn't show these if we found .bbappend files (but no .bb files). This change stops these warnings from appearing. (Bitbake rev: 48899fe7b3791dd897968f44c317e98bad14e146) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker: fix -b with BBCLASSEXTENDChris Larson2011-05-061-7/+10
| | | | | | | (Bitbake rev: 58bdaeb679d3c84cda827a33d09ce543547c45b4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cache.py: Ensure skipped recipes make it into the cache to avoid ↵Richard Purdie2011-05-051-2/+1
| | | | | | reparsing Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker/codeparser: Ensure the code parser cache is saved for each ↵Richard Purdie2011-05-051-4/+1
| | | | | | | | | | | | | | parsing process Before this change, the codeparser cache was only being saved for the main server process. This is suboptimal as it leaves code being re-evaluated at task execution time and increases parse time. We use the multiprocess Finalize() functionality to ensure each process saves out its cache. We need to update the cache save function to be multiprocess friendly with locking. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* universe target: add a new target to collect all recipe targetsSaul Wold2011-05-031-0/+8
| | | | | | | | | This new universe target is not intended to be used for compiling or building everything, it use is for sanity checking and other tasks that need to find all targets. This does not exclude any broken or virtual targets. Signed-off-by: Saul Wold <sgw@linux.intel.com>
* bitbake/cooker: don't error in prepareTreeData for unbuildable targetsJoshua Lock2011-03-141-1/+3
| | | | | | | Set abort to False in prepareTreeData so that unbuildable targets do not raise an exception. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/cooker: don't drop possible_world ref countJoshua Lock2011-02-241-4/+0
| | | | | | | | We need this if we want to run the buildWorldTargetList function more than once, for example in a UI where we can change the MACHINE and DISTRO as much as we like before triggering a build. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/[cooker|cache]: cache summary, license and group. Add to targets treeJoshua Lock2011-02-241-0/+6
| | | | | | | | | Add summary, license and group metadata to RecipeInfo and the cache. Unfortunately this impacts parse speed but gives us a much richer set of metadata to expose through UI's which can be accessed via the generateTargetsTree command. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/cooker: reduce code duplicationJoshua Lock2011-02-241-26/+11
| | | | | | | | Move runqueua and taskdata initialisation into a new function, prepareTreeData(), so that generateDepTreeData() and generateTargetsTreeData() are not duplicating the same logic. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/cooker: add generateTargetsTree methodJoshua Lock2011-02-241-3/+64
| | | | | | | | | | The generateTargetsTree() command needs to return a model which includes more metadata than the one generated by generateDepTree(). This patch adds a new method generateTargetsTreeData() to the cooker, based on generateDepData(), and switches generateTargetsTree() to use it. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake: implement command to find configuration files for a config variableJoshua Lock2011-02-241-0/+24
| | | | | | | | | Some configuration variables (MACHINE, MACHINE-SDK and DISTRO) set which confguration files bitbake should use. The added command , findConfigFiles, enables a UI to query which files are suitable values for a specified parameter. Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake: implement command to get all possible targets and their dependenciesJoshua Lock2011-02-241-0/+25
| | | | | | | | | | Add a new command generateTargetsTree() which returns a dependency tree of possible targets (tasks and recipes) as well as their dependency information. Optional parameter 'klass' also ensures any recipes which inherit the specified class path (i.e. 'classes/image.bbclass') are included in the model Signed-off-by: Joshua Lock <josh@linux.intel.com>
* bitbake/cooker: Fix parsing failure zombie problemRichard Purdie2011-02-231-0/+12
| | | | | | | | | When parsing if a SystemExit event is triggered, it causes the parsing thread to exit and the main process hangs waiting for it to finish indefintely. Add code to catch BaseExceptions and raise these with the main process gracefully instead of just hanging indefinitely with zombie processes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker: Ensure delays are accurately transfered to the idle loops ↵Richard Purdie2011-02-151-2/+2
| | | | | | from runqueue Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Fix merge conflict with upstream which was preventing ↵Richard Purdie2011-01-071-6/+5
| | | | | | clean bitbake shutdown Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-7/+4
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move LAYERDIR expansion hack into DataSmartChris Larson2011-01-041-13/+1
| | | | | | | (Bitbake rev: 40778a6e9e82c7ea4673a74fc19574430fa63e8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Show the user progress when loading the cacheBob Foerster2011-01-041-0/+1
| | | | | | | | (Bitbake rev: bdd7813d8eecf7b6b636322e748ca6bf69118513) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't swallow recipe parse errorsChris Larson2011-01-041-2/+6
| | | | | | | (Bitbake rev: 1f5d49ce64e5e1c7705edbfa3c8e19649c21edd2) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Rename command events, adjust compareRevisionsChris Larson2011-01-041-4/+0
| | | | | | | | | | | | - Moved the logic for comparing revisions from cooker into command - Removed 'Cooker' from the event names - Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to be a subclass of CommandExit (Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: add shutdown/stop methodsChris Larson2011-01-041-0/+6
| | | | | | | (Bitbake rev: fc64eff03fe1f41e59128cb3df0ef2161e24a8cb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: merge cookerState and cookerActionChris Larson2011-01-041-22/+11
| | | | | | | (Bitbake rev: c7c8945ef7ca9465312e630b7fa5f0a87ac8b6c7) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: use a pool, abort on first parse errorChris Larson2011-01-041-84/+60
| | | | | | | (Bitbake rev: 9caf65e79f95fe0045e727391e974c4c1e7411ff) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix the <100 recipe progress fixChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: 424428a764651183218f9cc93bc05496867aa5de) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't choke on <100 recipes to parseChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: 2be5169a01e57bcfcb1f75cbd6bac3bf0f230edd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: no cached in progressbar and add ETAChris Larson2011-01-041-15/+9
| | | | | | | | | | | | Rather than updating the progress bar based on the recipe being processed (whether cached or parsed), consider only parsed recipes. This reduces the instability in progress rate introduced by the cached entries, and allows the ETA to be resurrected and be a bit more useful. (Bitbake rev: 618480f7739f6ae846f67a57bee5a78efb37839d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: pass back child exceptions to the serverChris Larson2011-01-041-6/+12
| | | | | | | (Bitbake rev: 0f68f8bcd0e0aa944f76f88a4a85c9bcc1e42bee) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: ensure that the cache sync completesChris Larson2011-01-041-2/+7
| | | | | | | | | | | Without explicitly joining the thread, it's possible for the process to end (e.g. after a bitbake -p) and kill off the thread without waiting for it to exit cleanly. So, register the thread join with atexit. (Bitbake rev: 97ce57e6f860d3e6f34cc7a603ed1eeac4f423d3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: get number of threads in constructorChris Larson2011-01-041-3/+3
| | | | | | | (Bitbake rev: e7fd259d1528f48b0812fb7b0a54dd6d5d22f4a9) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't add info for skipped recipesChris Larson2011-01-041-2/+3
| | | | | | | (Bitbake rev: ac596d55de4ad86e1cdc84dabfea1350dd54efd0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: show progress bar before initializing the cacheChris Larson2011-01-041-13/+16
| | | | | | | | | | | | | This ensures that the time spent loading the cache from disk occurs with the progress bar up. Though the progress bar stays at 0% during this period, I think this is an improvement over the multi-second stall which occurred previously before the progress bar came up. Ideally, we'd integrate cache loading from disk into the progress display, but this is a first step. (Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: save progress chunk value (total/100)Chris Larson2011-01-041-1/+2
| | | | | | | (Bitbake rev: 09333737cbeeb9875d938521ddcd519fc808bcc3) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: stop loading the cache for -bChris Larson2011-01-041-25/+28
| | | | | | | | | | | Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: don't fire unnecessary parse progress eventsBob Foerster2011-01-041-1/+3
| | | | | | | (Bitbake rev: 7cf7d5f14405681496fced3640a50a20ef1acac1) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: sync the cache file to disk in the backgroundChris Larson2011-01-041-15/+16
| | | | | | | | | | | | | | | | This version uses a thread rather than a process, to avoid problems with waitpid handling. This gives slightly less overall build time reduction than the separate process for it did (this reduces a -c compile coreutils-native by about 3 seconds, while the process reduced it by 7 seconds), however this time is quite insignificant relative to a typical build. The biggest issue with non-backgrounded syncing is the perceived delay before work begins, and this resolves that without breaking anything, or so it seems. (Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Implement parallel parsing supportChris Larson2011-01-041-38/+97
| | | | | | | | | | | This utilizes python's multiprocessing module. The default number of threads to be used is the same as the number of available processor cores, however, you can manually set this with the BB_NUMBER_PARSE_THREADS variable. (Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cache: create and use a RecipeInfo classChris Larson2011-01-041-2/+1
| | | | | | | | | | | | | | | | | | This class holds the particular pieces of information about a recipe which are needed for runqueue to do its job. By using it, I think we improve code clarity, reduce method sizes, reduce overuse of primitive types, and prepare for parallel parsing. In addition, this ditches the leaky abstraction whereby bb.cache attempted to hide the difference between cached data and a full recipe parse. This was a remnant from the way things used to be done, and the code using it had to know the difference anyway. If we choose to reimplement caching of the full recipes, we can do it in bb.parse, in a completely transparent way. (Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Prefer xrange over range for small performance gain.Bob Foerster2011-01-041-1/+1
| | | | | | | | | | | | | | range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Pre-explode rundeps/runrecs in CacheDataChris Larson2011-01-041-2/+2
| | | | | | | (Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix GraphViz .dot output for rdepends and rrecsChris Larson2011-01-041-2/+2
| | | | | | | | | | | Use bb.utils.explode_deps to break up the rdepends and rrecommends strings. This fixes the same issue which was fixed by a number of patches floating around, but uses explode_deps rather than regular expressions. (Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Resurrect display of failed filesChris Larson2011-01-041-0/+4
| | | | | | | | | | This was inadvertantly removed when trying to reduce the amount of duplicated information the user sees when a failure occurs. (Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-24 01:21:31 +0000