summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/runqueue.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: Update users of getVar/setVar to use the data store functions directlyRichard Purdie2011-11-271-13/+13
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Fix debug message to reference the correct taskRichard Purdie2011-11-241-1/+2
| | | | | | (Bitbake rev: 035c673c463ca450245acf824e7b7e8f889bdc89) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Ensure we fully process the covered listRichard Purdie2011-11-231-3/+4
| | | | | | | | | | | | | | | The existing looping code can mask an existing "found = True" by forcing it to False each time. This can lead to dependency lists not being fully searched and results in dependency errors. An exmaple of this was the autobuilder building linux-yocto from sstate but then rebuilding some of the recipe's tasks for no apparent reason. Separating the logic into two variables solves this problem since any "found = True" value is now always preserved. (Bitbake rev: 61017fc5d30b7a13308d038872ec92efc1a84cef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Ensure setscene tasks don't break dependency orderRichard Purdie2011-11-231-4/+7
| | | | | | | | | | | | | | | If A depends upon B which depends upon C and the setscene for B succeeds but C is going to get rebuilt, we should wait for C to try and build A but currently we don't. This is due to the timing of when we run the task_skip() as this triggers other tasks to become buildable. This patch moves the timing of that call to a more appropriate place allowing dependencies to behave as expected. (Bitbake rev: b7114d8e5d9b0720339bd5d24d243c0f2a7c1f3b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Add BB_SETSCENE_VERIFY_FUNCTION hookRichard Purdie2011-11-221-11/+9
| | | | | | | | | | | | | | | | | The OE metadata has cases where it needs to prevent some setscene functions from running. An example of this is where we know a task is going to run do_configure (which would clean out do_populate_sysroot) and hence we don't want do_populate_sysroot_setscene to run. This change adds in a hook so that the metadata can allow any such policy decision to filter back up to bitbake. It removes the existing code which attempted to do this in a generic way but failed. (Bitbake rev: 2f6d987d9957a5d713ce119c24c2e87540611f53) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Sort the list of skipped tasks as it makes searching ↵Richard Purdie2011-11-211-1/+1
| | | | | | | | | | the list easier when debugging (From Poky rev: 5de8a495fba657e1febc616bbc737a8136cc88f9) (Bitbake rev: 110f6cccbcc5907e15262c05d5c47da101e1a47d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix incorrect task number reference in debug messageRichard Purdie2011-11-211-1/+1
| | | | | | | | (From Poky rev: 45887bbd5479041be05b914668f14de6ec9b9831) (Bitbake rev: dc4439ca8c7db7ceee78bd0552f65ceddcff17a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/runqueue: avoid marking runtime dependencies as coveredPaul Eggleton2011-11-151-1/+6
| | | | | | | | | | | | | | | | | | | | | | The code which populates setscene_covered list was adding a task to the covered list if all of the tasks that depend upon it were also covered; however, this means that tasks that would have installed "runtime" dependencies were being marked as covered also, e.g. gmp-native and mpfr-native are needed by gcc-cross at runtime since they are shared libraries that gcc links to, but their do_populate_sysroot tasks were being marked as covered, resulting in failures later on if gcc-cross was available from sstate but mpfr-native and gmp-native weren't. Since we currently have no real way to handle runtime dependencies for native packages, add a workaround which avoids marking tasks as covered if one or more of their revdeps are from a different recipe. Fixes [YOCTO #1536]. (Bitbake rev: e492eb4dc9016cd0bed194377c6f2b85cf0ad113) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add FAKEROOTNOENV variableRichard Purdie2011-11-101-0/+6
| | | | | | | | | | | | | | | | | Add a FAKEROOTNOENV which does the opposite of the FAKEROOTENV variable and is data loaded into the environment for tasks without the fakeroot flag. The intent here is to provide a way to control the environment when we aren't needing fakeroot context which allows us to unload the preload from memory entirely and gain a valuable speedup. I'm not 100% happy with needing this at the bitbake level, particularly with the cache hit but it does give a valuable speedup. (Bitbake rev: 05c29ab0b2ad3c521414cabb6a92bca15c6e919c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Ensure fakeroot variables are reflected in the datastoreRichard Purdie2011-09-091-0/+1
| | | | | | | | | | Without this, variables can be set to one thing in one part of the environment and something different in another part. This change ensures the datastore and the environment are consistent. (Bitbake rev: 459addf13721a6847406f215650fa1882fb83ea9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Ensure task environment is correctRichard Purdie2011-09-091-1/+10
| | | | | | | | | | | | | | | | | | This fixes two problems: a) Variables which were in the parent environment but not set as "export" variables in the datastore could end up in the task environment b) oe.environ.update() can't cope with the generator returned by bb.data.exported_vars() Whilst the updated code isn't as neat, it does do the expected thing, sets the environment correctly and stops unwanted values leaking into the task environment. (Bitbake rev: 67e5e23034c5ec2b9efcca935242830306c0048d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fixup remaining bb.msg.domain usersRichard Purdie2011-08-151-10/+10
| | | | | | (Bitbake rev: d5abdacaf9ac604ef8d8c1bafb9b30617827cb4f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Add task specific stamp file supportRobert Yang2011-06-281-0/+10
| | | | | | | | | | | | | | | | | | This patch, based on proof of concept code from Richard adds code to bitbake to allow individual tasks to optionally specify their stamp file using the stamp-base flag. This takes the same form as the STAMP variable but can be specified on a per task basis. Code is also added to runqueue to ensure that if two tasks share the same stamp file, only one will be executed at once. A significant usecase for this code is to share source code (${S}) between recipes where separate build directories (${B}) are used. (Bitbake rev: 41bef02bef8379590ba012319aebe05068a8081e) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Add umask task controlMark Hatle2011-06-241-0/+11
| | | | | | | | | | | | | | | | The umask for a task can now be set as: task[umask] = 022 task[umask] = '022' If specified as a text string, it must be octal. (This is due to recipe parsing where it's always set to a string.) [RP tweaked to use None instead of -1] (Bitbake rev: a5caaaaba8f0db1af5d8f2e610021d6d4b56894e) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-1/+1
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data/runqueue: Sync up with upstream to clean up environment ↵Richard Purdie2011-06-081-10/+1
| | | | | | variable handling Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Wrap parsing with exception trappingRichard Purdie2011-06-081-24/+24
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Misc sync with upstreamRichard Purdie2011-06-081-5/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Move BB_WORKERCONTEXT to a place where anonymous python ↵Richard Purdie2011-05-311-1/+2
| | | | | | can see it Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure existing setscene stamp files are taken into accountRichard Purdie2011-05-271-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | JaMa reported issues where bitbake would rebuild things instead of using the existing built tasks. This was tracked to a case where: a) rm_work is uses b) A depends on B c) B has a version change (e.g. PR bump) and A *and* B would then rebuild. It turns out that rm_work was correctly turning stamp files into the correct _setscene varients but bitbake was then ignoring them during setscene processing. If the correct sstate checksumed files didn't exist, everything would seemingly rebuild. The fix is to check for existing *_setscene stamps and if present, honour them. If "basichash" is enabled, the hash is included with the stamps so everything should then function as intended. (Bitbake rev: 0a4d857aabc86b973170ba9ce32b6b449a4e2165) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Remove old log file handling remnantsRichard Purdie2011-05-051-6/+0
| | | | | | These lines date from earlier code and are no longer required. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure fakeroot env variables make it to the child processRichard Purdie2011-03-311-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify fakeroot environment handlingChris Larson2011-03-311-15/+17
| | | | | | | (Bitbake rev: 88f0d1db88bdba95e488fba34f40dc0b72a85c33) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix more incorrect usages of 'is'Chris Larson2011-03-311-2/+2
| | | | | | | (Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: fix clash when setscene & real tasks done in same buildPaul Eggleton2011-03-291-0/+19
| | | | | | | | | | | | | | | | | If a build causes a real task to be run when the setscene task has already run then it was possible for dependent packages to be rebuilding at the same time as a rebuild of the packages they depended on, resulting in failures when files were missing. This change looks in the setscene covered list and removes anything where a dependency of the real task is going to be run (e.g. do_install is going to be run even though the setscene equivalent of do_populate_sysroot has already been run). As an additional safeguard we also delete the stamp file for the setscene task under these circumstances. Fixes [YOCTO #792] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Revert "bitbake/gcc: Enable a shared common source tree"Richard Purdie2011-03-291-8/+0
| | | | | This reverts commit 12b163dbd81cafafec1ebe3c4039c65af60ee261 as it was an unintended change for master
* bitbake/gcc: Enable a shared common source treeRichard Purdie2011-03-281-0/+8
| | | | | | | | | | | | | | | | | | | | | This patch is a quick proof of concept to show how source code could be shared between recipes which use ${B} to have a separate build directory compared to source directory ${S}. Issues: a) gcc uses sed and creates config files against ${S} which means the directory should not be shared. Need to change the way that works. b) Could be extended to cover eglibc except there is a patch applied against nativesdk versions which again makes the source incompatible. c) Need to clean up the layout in work-shared and make a directory level deeper to ensure patch separation. d) clean task does not remove stamps Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: show correct task name for setscene task failurePaul Eggleton2011-03-151-3/+11
| | | | | | | | If a setscene task failed previously it was showing an incorrect task name in the error line. This patch ensures we show the correct name, also including the "_setscene" suffix. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* bitbake/cache/runqueue.py: Move workload for recipe parsing to the child processRichard Purdie2011-02-281-12/+23
| | | | | | | | | | | | | | | Parsing the recipe in the parent before forking off the child worker can mean the parent doesn't hit the idle loop and becomes a bottleneck when lauching many short lived processes. The reason we need this in the parent is to figure out the fakeroot environmental options. To address this, add the fakeroot variables to the cache and move recipe loadData into the child task. For a poky-image-sato build this results in about a 2 minute speedup (1.8%). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Drop PATH override. This was needed for the exec() change ↵Richard Purdie2011-02-211-2/+0
| | | | | | but not now we use fork() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify RunQueueStats.copyChris Larson2011-02-211-4/+1
| | | | | | | (Bitbake rev: 778571f155139fcf43d0e30e875c39d6592ae03f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: pass a copy of the RunQueueStats to eventsChris Larson2011-02-211-1/+9
| | | | | | | | | | This avoids cases where the stats are modified after the event is fired but before it's dispatched to the UI. (Bitbake rev: 1954f182687a0bd429175dda87f05d8a94bb403a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Move SIGCHLD handler to wrap none server sleep callRichard Purdie2011-02-181-7/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure child has the default SIGCHLD handler restoredRichard Purdie2011-02-161-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we successfully fork off a task and there are more ↵Richard Purdie2011-02-151-0/+2
| | | | | | processes we can start do so immediately Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we have successfully collected a pid value return ↵Richard Purdie2011-02-151-0/+1
| | | | | | True to ensure we don't sleep in the main loop Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Add a SIGCHLD handler to ensure we wake from sleeps for ↵Richard Purdie2011-02-151-0/+6
| | | | | | SIGCHLD events Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Avoid starvation of events to the serverRichard Purdie2011-01-281-1/+1
| | | | | | | | | | | | | | The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since most new LogRecord events are larger than this it leads to a build up of data which is only processed slowly, leading to a bottleneck and a slow down of all bitbake processes. Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging this. A large value has been left in for the read() command just to ensure some fairness amongst process handling if a task tries to log truly huge amounts of data to the server, or goes crazy and ensures the main loop doesn't stall. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Introduce stamp-extra-info task flag into stamp filenamesDongxiao Xu2011-01-181-6/+6
| | | | | | | | | | | For certain tasks, we need additional information in build stamp file other than the task name and file name. stamp-extra-info is introduced as a task flag which is appended to the stamp file name. [Code simplifcations/tweaks from Richard] Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Renable check_stamp_fn() for now since we still fork for task ↵Richard Purdie2011-01-101-14/+10
| | | | | | execution (as per bitbake-upstream) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Simplify hashdata storage since we're not using an ↵Richard Purdie2011-01-101-17/+14
| | | | | | external script for tasks anymore, in line with bitbake-upstream Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the ↵Richard Purdie2011-01-101-5/+6
| | | | | | BasicHash siggen code Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Drop unneeded import and improve quieterrors handling as per ↵Richard Purdie2011-01-101-3/+5
| | | | | | upstream bitbake Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update event queue handling to match upstream bitbakeRichard Purdie2011-01-101-4/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Sync with changes in upstream bitbakeRichard Purdie2011-01-101-90/+80
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Remove duplicate TaskStarted event fire()Richard Purdie2011-01-071-2/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignoredRichard Purdie2011-01-061-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake build/siggen/runqueue: Fix stampfile parametersRichard Purdie2011-01-061-6/+6
| | | | | | | | The current parameters are not useful to the stampfile generator function as they can't uniquely define a task. This updated things so the parameters can identify unique tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-11/+14
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue: fix typo in documentation of calculate_task_weightsBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 5cde120003af97a5cf5c6eff2a02bb1480f9414b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-30 10:30:04 +0000