summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/runqueue.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: runqueue: Fix sceneQueueEvent to use the correct hashesRichard Purdie2014-03-251-1/+1
| | | | | | | | | | The runqueue should be using the "realtask" ID to lookup the task hash, not the "task" ID. This patch resolves corruption issues where incorrect task hashes were displayed within toaster. (Bitbake rev: 84be1a27f89d1bf63c21f06d831df0a66a5db860) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Remove use of waitpid on worker processesRichard Purdie2014-03-191-16/+12
| | | | | | | | | | | | | Use of waitpid on the worker processes is a bad idea since it conflicts directly with subprocess internals. Instead use the poll() method and returncode to determine if the process has exitted, if it has, we can shut down the system. This should resolve the hangs once and for all, famous last words. (Bitbake rev: 60969cd62e21e7d4af161bf8504b7643a879c73f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Revert child signal handler for nowRichard Purdie2014-03-191-28/+17
| | | | | | | | | | | | | | | | | | | | | | | | | We're running into processes using 100% cpu. It appears theses are locked in a subprocess.poll() type loop where the process has exited but the code is looping as its not handling the ECHILD error. http://bugs.python.org/issue14396 http://bugs.python.org/issue15756 This is likely due to one or both of the above bugs. The question is what actually grabbed the child exit code as it wasn't this code. Its likely there is therefore some other code racing and taking that code, it may be some kind of race like: http://hg.python.org/cpython/rev/767420808a62/ where the fix effectively catches the childs codes in a different part of the system. We could try and get everyone onto python 2.7.4 where the above bugs are fixed however for now its safer to admit defeat and go back to polling explictly for our worker exit codes. (Bitbake rev: 5b9a099ec2a1dc954b614e12a306595f55b6a99e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Don't catch all child return codesRichard Purdie2014-03-191-3/+4
| | | | | | | | | | Catching all child exit status values is a bad idea. Setting an http sstate mirror is a great way to view that spectacularly break things. The previous change did have good code changes so don't revert those parts. (Bitbake rev: fa7ffb62d510ac1124ae7e08fa4d190a710f5b54) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Really fix sigchld handlingRichard Purdie2014-03-181-14/+14
| | | | | | | | | | | | | | | | | | | There are several problems. Firstly, a return value of "None" can mean there is a C signal handler installed so we need to better handle that case. signal.SIG_DFL is 0 which equates to false so we also need to handle that by testing explicitly for None. Finally, the signal handler *must* call waitpid on all child processes else it will just get called repeatedly, leading to the hanging behaviour we've been seeing. The solution is to only error for the worker children, we warn about any other stray children which we'll have to figure out the sources of in due course. Hopefully this patch gets things working again properly though. (Bitbake rev: 973876c706f08735c1b68c791a5a137e5f083dd2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Ensure handler does not recurseRichard Purdie2014-03-181-1/+1
| | | | | | | | | Failures on the autobuilder look like this handler is recursing. That shouldn't be possible but it doesn't hurt to code as such. (Bitbake rev: e39e85803cbe1ef9413a118868c19087c0546d01) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: More carefully handle the sigchld handlerRichard Purdie2014-03-181-2/+4
| | | | | | | | | | | | We've noticed hanging processes which appear to be looping around waitpid. Its possible multiple calls to teardown are causing problem or in theory multiple registrations (although the code should not allow that). Regardless, put better guards around signal handler registration. (Bitbake rev: 79acfb0853aa3215215cee89a945f8e97b0a8fae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Don't error if we never setup workersRichard Purdie2014-03-171-1/+2
| | | | | | | | | | If we didn't setup any workers (such as bitbake -S), this would error since we're trying to set a signal handler to None. This patch avoids that problem. (Bitbake rev: ce17478c8197abf178c00774f5bbe23fd4375ee2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Improve sigchld handlerRichard Purdie2014-03-121-5/+10
| | | | | | | | | | | | | | | | | | | The sigchld handler was reaping any processes and this was leading to confusion with any other process handling code that could be active. This patch: a) Ensures we only read any process results for the worker processes we want to monitor b) Ensures we pass the event to any other sigchld handler if it isn't an event we're interested in so the functions are properly chained. Together this should resolve some of the reports of unknown processes people have been reporting. (Bitbake rev: fe8baaa2f533db7a1b7203476c675588923d8d45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Use SIGCHLD instead of polling waitpidRichard Purdie2014-03-101-14/+19
| | | | | | | | | Instead of a significant number of calls to waitpid, register a SIGCHLD handler instead. (Bitbake rev: 76029d08ad56a0a264ff9738a0336971a455b7f5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: providers/runqueue/taskdata: Optimise logger.debug callsRichard Purdie2014-03-101-1/+1
| | | | | | | | | | | | | A run of "bitbake bash -c unpack" when the task has already been completed resulted in about 9000 calls to logger.debug(). With this patch which comments out some noisy/less usefull logging and moves other logging calls outside loops, this number is reduced to 1000 calls. This results in cleaner logs and gives a small but measurable 0.15s speedup. The log size dropped from 900kb to 160kb. (Bitbake rev: d2677f084fe1d8846db77d89ef5e6ffb18dc171a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: Gracefully handle a missing worker processRichard Purdie2014-03-101-7/+14
| | | | | | | | | | If the worker has already gone missing (e.g. SIGTERM), we should gracefully handle the write failures at exit time rather than throwing ugly tracebacks. (Bitbake rev: 1b1672e1ceff17fc4ff8eb7aa46f59fce3593873) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: Handle worker disappearing gracefullyRichard Purdie2014-03-101-5/+22
| | | | | | | | | | | If the worker (or fakeworker) process disappears for some reason, the system doesn't currently even notice. To fix this, we call waitpid periodically, looking for exit events of our children. If these occur, we can gracefully shutdown the server. (Bitbake rev: ee28ddadaa7ef91e4d4b7d22fc267382aaad6d01) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix typoRichard Purdie2014-03-061-1/+1
| | | | | | | | | | slef.self is clearly meant to be self, fix typo. Otavio spotted and reported, thanks. (Bitbake rev: 316daad7928a58cdfc42e27b20e739f4dd74a02a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Catch ValueError from pickle.loadsMartin Jansa2014-02-241-2/+8
| | | | | | | | | | * exception like this keeps spinning quite quickly generating GBs of logs better to kill it asap and show invalid pickle (Bitbake rev: a69eb4c12c71bba9d742c4e5578f25c388d9f825) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix silly variable overlapRichard Purdie2014-02-151-3/+3
| | | | | | | | | | | | A previous commit of mine used the target variable for two different uses resulting in a lot more sstate being installed than is needed. Fix the variable to use two different names and unbreak the setscene behaviour. (Bitbake rev: f975ca2cf728561bd6317ed8f76303598546113a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Ensure we do run 'masked' setscene tasks if specified as ↵Richard Purdie2014-02-131-4/+12
| | | | | | | | | | | | | | | | | | targets If you specify multiple targets on bitbake's commandline and some of them are setscene tasks which are "masked" by other tasks they may not get run. For example <image>:do_rootfs <kernel>:do_populate_sysroot the rootfs tasks "masks" the populate_sysroot task so bitbake would currently decide not to run it. In this case, we do really want it to be run. The fix is not to skip anything which has been given as an explict target. (Bitbake rev: 0753899d1e855795cc18671357609a86f169b379) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix setscene hard dependency problemsRichard Purdie2014-02-111-5/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051 added in the idea of hard dependencies such as the case a setscene has a hard dependency on pseudo-native and that dependency wasn't available from sstate for some reason. Unfortunately the implementation was a bit too enthusiastic, causing rebuilds of things when it wasn't necessary. A test case was: bitbake quilt-native bitbake quilt-native -c clean bitbake <some-image> and then you'd watch quilt-native get rebuilt for no good reason. The clue to the problem is in the for loop where it never depends on the item being iterated over. The fix is to include the exact list of hard dependencies rather than guessing. With these changes, the use case above works, the one in the original commit also works. This patch also adds in or cleans up various pieces of logging to allow issues like this to be more easily debugged in future. (Bitbake rev: 81bd475585ff1b44b390036b1eca0feae7c149eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix race against tasks sharing stamp filesRichard Purdie2014-01-311-1/+4
| | | | | | | | | | Shared work directories work by assuming bitbake will not run more than one task with a specific stamp name. Recent runqueue optimisations accidentally broke this meaning there could be races. This fixes the code. (Bitbake rev: b1628b1a260ddf43fc9985535b1ddcfcebbb1e5b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Simplify pointless len() usageRichard Purdie2014-01-281-1/+1
| | | | | | (Bitbake rev: 1f2bdd1b99075babe8dba91478cfc5d3501676cb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Only attempt to print closest matching task if there is a ↵Richard Purdie2014-01-271-6/+5
| | | | | | | | match (Bitbake rev: 1dbf400c662354b7826b2b97ee2e3e6d11af9fd2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Further extend bitbake -S output to view signature ↵Richard Purdie2013-12-201-2/+45
| | | | | | | | | | | | differences Based upon the list of difference starting points, we can use the siggen.find_siginfo() function call and the difference printing code to provide a list of differences between the current build target and whatever can be obtained from the sstate cache. (Bitbake rev: 7a77861feb62750ef166d2d1e89ed1f444ca8dc7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix data being written into siginfo/sigdata filesRichard Purdie2013-12-201-15/+2
| | | | | | | | | | | | | | The way hash_deps was being generated was different to the way siggen generated the data internally which lead to seemingly different sigdata/siginfo files for the same checksum. The -S output correct but the files written during builds contained superflous data which would look like a difference. This patch removes the badly duplicated data and uses it from the source which ensures its consistent. (Bitbake rev: e6d5e925c402cd2cc7ee034e9de4cc6df8944a34) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Add output for -S option for listing the changepoints ↵Richard Purdie2013-12-181-0/+77
| | | | | | | | | | | | | | | compared with an sstate cache Its useful to understand where the delta starts against an existing sstate cache for a given target. Adding this to the output of the -S option seems like a natural fit. We use the hashvalidate function to figure this out and assume it can find siginfo files for more than just the setscene tasks. (Bitbake rev: c18b8450640ebfd55a2b35b112959f9ea3e0a700) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Share BB_TASKDEPDATA with tasksRichard Purdie2013-11-261-4/+26
| | | | | | | | | | | | | | | | | Currently tasks have no knowledge of which other tasks they depend upon. This makes it impossible to do at least two things which would be desirable/interesting: a) Have the ability to create per recipe sysroots b) Allow the aclocal files to be present only for the entries in DEPENDS (directly and indirectly) By exporting task data through this new variable, tasks can inspect their dependencies and then take actions based upon this. (Bitbake rev: 84f1dde717dac22435005b79d03ee0b80a3e8e62) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Optimise next_buildable_task()Richard Purdie2013-11-261-17/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This unlikely looking function was found to be eating a lot of CPU time since it gets called once per trip through the idle loop if we're not running a maximum number of processes. This was particularly true in world builds of 13,000 tasks. Calling the computation code is pretty pointless because until some other task finishes nothing is going to become available to build. We can know when things become available so this patch teaches the scheduler this knowledge. It also: * skips any coputation when nothing can be built * if there is only one available item to build, ignore the priority map * precomputes the stamp filenames, rather than doing it every time * saves the length of the array rather than calculating it each time (the extra function overhead is significant) Timing wise, initially, 5000 iterations through here was 20s, with the patch 200000 calls takes the same time. The end result is that builds get up and running faster. (Bitbake rev: 4841c1d37c503a366f99e3a134dca7440e3a08ea) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/bitbake-worker: Fix dry run fakeroot issuesRichard Purdie2013-11-241-1/+1
| | | | | | | | | | | | | | When using the dry run option (-n), bitbake would still try and fire a specific fakeroot worker. This is doomed to failure since it might well not have been built. Add in some checks to prevent the failures. [YOCTO #5367] (Bitbake rev: f34d0606f87ce9dacadeb78bac35879b74f10559) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix hole in setsceneverify skipped task logicRichard Purdie2013-11-201-2/+19
| | | | | | | | | | | | | | | | | | | | | | We have do_bundle_initramfs which is a task inserted after compile and before build. It is not covered by sstate. If we run a build with a valid sstate cache present, the setsceneverify function realises it will rerun the do_compile step (due to the bundle_initramfs task) and hence marks do_populate_sysroot to rerun. do_install, a dependency of do_populate_sysroot is left as marked as covered by sstate. What we need to do is traverse the dependency tree for any setsceneverify invalided task and ensure any dependencies are also invalidated. We can stop at any point we reach another setscene task though. This means the do_populate_sysroot task has the data from do_install available and doesn't crash. (Bitbake rev: f21910157d873c030b149c4cdc5b57c5062ab5a6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: add task hash to Queue eventsAlexandru DAMIAN2013-09-221-0/+5
| | | | | | | | | | Adding the sstate-related hash for all runqueue and scenequeue tasks, as it's needed in the WebHob data. (Bitbake rev: b6e2ce1cf7a0ede890f08fabf536a556dc4263c5) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: build, runqueue: adds info to the *runQueue* eventsAlexandru DAMIAN2013-09-221-0/+10
| | | | | | | | | | | | | | | | | | | This patch adds task identifying information for all runQueue and sceneQueue events, and for bb.build.Task* events. This will allow matching event to specific tasks in the UI handlers processing these events. Adds RunQueueData functions to get the task name and task file for usage with the runQueue* events. Adds taskfile and taskname properties to bb.build.TaskBase. Adds taskfile and taskname properties to the *runQueue* events (Bitbake rev: b4a5e4be50d871a80dbe0993117d73f5ad82e38f) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: cooker,runqueue: send the task dependency treeAlexandru DAMIAN2013-09-221-0/+5
| | | | | | | | | | | | | | | Adding a CookerFeature that allows UIs to enable receving a dependency tree once the task data has been computed and the runQueue is ready to start. This will allow the clients to display dependency data in an efficient manner, and not recompute the runqueue specifically to get the dependency data. (Bitbake rev: 75466a53b6eece5173a9bfe483414148e4c06517) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: add runQueueTaskSkipped eventAlexandru DAMIAN2013-09-171-3/+12
| | | | | | | | | | Adding a runQueueTaskSkipped to notify that the tasks that are not run either because they are set-scened or they don't need an update (timestamp was ok). (Bitbake rev: cf4a0c7aa82090876ae652b611acfab3ce2688f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: add sceneQueueTaskCompleted eventAlexandru DAMIAN2013-09-101-0/+6
| | | | | | | | | | | Adding an event to be fired when a scene task is completed. It is analogous to the run task completed event, and has been missing for some reason. (Bitbake rev: 73b8f4d3fbeaf1b330a66d76012d0a5cef8dbe2d) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: check whether multiple versions of the same PN are due ↵Robert Yang2013-09-091-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to be built There would be an race issue if we: $ bitbake make-3.81 make-3.82 This because they are being built at the same time which would cause unexpected problems, for example: [snip] ERROR: Package already staged (/path/to/tmp/sstate-control/manifest-qemux86-make.populate-sysroot)?! ERROR: Function failed: sstate_task_postfunc [snip] Or there would be python's strack trace such as: [snip] *** 0004: mfile = open(manifest) 0005: entries = mfile.readlines() 0006: mfile.close() 0007: 0008: for entry in entries: Exception: IOError: [Errno 2] No such file or directory: xxx [snip] [YOCTO #5094] We can quit earlier to avoid this kind of issue when two versions of the same PN are going to be built since this isn't supported. (Bitbake rev: ab377c00c33a2d296bfda1b0b6c2a62b29d1004f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Ensure ${DATE} and ${TIME} are consistentPeter Kjellerstedt2013-09-061-0/+2
| | | | | | | | | | | | | | | Due to the worker split the ${DATE} and ${TIME} variables could end up with different values for different workers. E.g., a task like do_rootfs that is run within a fakeroot environment had a slightly different view of the time than another task that was not fakerooted which made it impossible to correctly refer to the image generated by do_rootfs from the other task. (Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: ensure BUILDNAME is available during executionPaul Eggleton2013-09-041-0/+1
| | | | | | | | | | | | | | | BUILDNAME is set from cooker by default, so since the worker split it will not be set when executing functions. In OpenEmbedded this results in /etc/version (which is populated from BUILDNAME) not having any content. Pass this variable value through to the worker explicitly to fix the issue. Fixes [YOCTO #4818]. (Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix scenequeue to pass file descriptors, not a floatRichard Purdie2013-09-021-1/+1
| | | | | | | | This was missed off in a previous patch. (Bitbake rev: ad7664edd40fa46e6f6fec2144403e3b6fc3a639) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on ↵Richard Purdie2013-09-011-2/+10
| | | | | | | | | | | | | | | | | | | | | | fds, not time.sleep() The existing backend server implementations were inefficient since they were sleeping for the full length of the timeouts rather than being woken when there was data ready for them. It was assumed they would wake and perhaps did when we forked processes directory but that is no longer the case. This updates both the process and xmlrpc backends to wait using select(). This does mean we need to pass the file descriptors to wait on from the internals who know which these file descriptors are but this is a logical improvement. Tests of a pathaolgical load on the process server of ~420 rapid tasks executed on a server with BB_NUMBER_THREAD=48 went from a wall clock measurement of the overall command execution time of 75s to a much more reasonable 24s. (Bitbake rev: 9bee497960889d9baa0a4284d79a384b18a8e826) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: report close matches for an invalid task namePaul Eggleton2013-08-161-1/+7
| | | | | | | | | | Help to pick up mistakes such as "bitbake -c cleanstate xyz" (instead of "bitbake -c cleansstate xyz".) (Bitbake rev: 15c3db1cffdffd85641c6b12e77f19ce7a553472) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: add warning if invalidating invalid taskPetter Mabäcker2013-07-291-0/+3
| | | | | | | | | | | | | Add a warning if 'bitbake -C' is executed with a task that does not exist. Fixes [YOCTO #4877] (Bitbake rev: 6459c1d0eb8f1007246df36149e2496ee531e25f) Signed-off-by: Petter Mabäcker <petter@technux.se> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue/build: Add recideptask flagRichard Purdie2013-06-201-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | Currently, tasks like fetchall are slightly broken since if a recipe has specific [depends] which occur after do_fetch and add items not listed in DEPENDS and RDEPENDS, they are not caught by recrdeptask. We've gone around in circles on this issue (e.g http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/runqueue.py?id=5fa6036d49ed7befe6ad50ec95c61a50aec48195 ) and in many cases the behaviour of recrdepends is correct but tasks like fetchall need the other behaviour. To address this we add a recideptask flag which can be used in conjuction with the recrdeptask flag to specify which task to to the inspection upon. This means entries like do_rootfs[depends] which have do_fetch tasks are caught and run. I'm not 100% happy with needing another flag but I don't see any rational way to get the correct behaviour in all cases without it. [YOCTO #4597] (Bitbake rev: f8c9b292b02ce2c28741b74901205f5e5807ca87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Improve handling of failing setscene tasks with hard ↵Richard Purdie2013-06-141-3/+7
| | | | | | | | | | | | | | | | | | | | | | | dependencies If a setscene task has a hard dependency on a task like pseudo-native, its expected that the setscene task will not run unless the dependency is met. This adds code to ensure that is the case, otherwise a bug would show up with a usecase like: bitbake gnome-common bitbake pseudo-native -c cleansstate bitbake gnome-common -c clean bitbake gnome-common With the double wrapper script environment, we'd not see issues like this as it would be masked. The problem theoretically affects code like useradd too as well as anything using a sstate postinstall. (Bitbake rev: c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Adapt autostart to bitbake-workerRichard Purdie2013-06-141-0/+1
| | | | | | | | | | With the change to bitbake-worker we need to ensure the workers know how to contact the PR service, the magic 0 port and singleton is no longer enough. (Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Spawn a separate worker for fakeroot tasksRichard Purdie2013-06-141-17/+58
| | | | | | (Bitbake rev: 860ec42b220b7ed3f3bbe52c3546bba66644eac8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Abstract the start and teardown worker functionsRichard Purdie2013-06-141-17/+31
| | | | | | | | | We're going to need a fakeroot/pseudo version of the worker so abstract the code to start the worker process. (Bitbake rev: b5d0f12f9df3ab211700473ed145ee6fbd9ca8e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Move the bitbake-worker execution to a higher levelRichard Purdie2013-06-141-59/+55
| | | | | | | | | | The worker was being executed by each execution queue so would get constructed twice for each build. This is wasteful so move execution to the main runqueue so we only have to start the worker once. (Bitbake rev: 8117f8480125b121b2b5ac0afc31b108d9e670ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Split runqueue to use bitbake-workerRichard Purdie2013-06-141-178/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a pretty fundamental change to the way bitbake operates. It splits out the task execution part of runqueue into a completely separately exec'd process called bitbake-worker. This means that the separate process has to build its own datastore and that configuration needs to be passed from the cooker over to the bitbake worker process. Known issues: * Hob is broken with this patch since it writes to the configuration and that configuration isn't preserved in bitbake-worker. * We create a worker for setscene, then a new worker for the main task execution. This is wasteful but shouldn't be hard to fix. * We probably send too much data over to bitbake-worker, need to see if we can streamline it. These are issues which will be followed up in subsequent patches. This patch sets the groundwork for the removal of the double bitbake execution for psuedo which will be in a follow on patch. (Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Split data from configurationRichard Purdie2013-06-071-6/+6
| | | | | | | | | | | The reasons for combining these objects is ancient history, it makes sense to clean things up and separate them out now. This follows on logically from the configuration cleansups and leads well into the bitbake-worker changes. (Bitbake rev: 89ffd62661ebcf2a97ce0c8dfd5e4d5bfbe27de7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: Ensure export flag is set for fakeroot environment ↵Richard Purdie2013-05-221-0/+1
| | | | | | | | | | | | variables This means the variables show up in the shell execution "run" files since its useful to know what the fakeroot environment is and how to set it up manually. (Bitbake rev: bdf437747b664479acde6deaa9096e2a6bcdf483) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Separate out collections handling code into its own classRichard Purdie2013-05-121-2/+2
| | | | | | | | | | The Cooker class is too large and needs to be split up into different functional units. Splitting out the collections code into its own class seems like a good place to start to try and disentangle things. (Bitbake rev: ca1fcbb6e214c155a05328779d3d326e10c5eac0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-13 17:59:07 +0000