summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/runqueue.py
Commit message (Collapse)AuthorAgeFilesLines
* bitbake/runqueue.py: Remove old log file handling remnantsRichard Purdie2011-05-051-6/+0
| | | | | | These lines date from earlier code and are no longer required. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure fakeroot env variables make it to the child processRichard Purdie2011-03-311-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify fakeroot environment handlingChris Larson2011-03-311-15/+17
| | | | | | | (Bitbake rev: 88f0d1db88bdba95e488fba34f40dc0b72a85c33) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix more incorrect usages of 'is'Chris Larson2011-03-311-2/+2
| | | | | | | (Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: fix clash when setscene & real tasks done in same buildPaul Eggleton2011-03-291-0/+19
| | | | | | | | | | | | | | | | | If a build causes a real task to be run when the setscene task has already run then it was possible for dependent packages to be rebuilding at the same time as a rebuild of the packages they depended on, resulting in failures when files were missing. This change looks in the setscene covered list and removes anything where a dependency of the real task is going to be run (e.g. do_install is going to be run even though the setscene equivalent of do_populate_sysroot has already been run). As an additional safeguard we also delete the stamp file for the setscene task under these circumstances. Fixes [YOCTO #792] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Revert "bitbake/gcc: Enable a shared common source tree"Richard Purdie2011-03-291-8/+0
| | | | | This reverts commit 12b163dbd81cafafec1ebe3c4039c65af60ee261 as it was an unintended change for master
* bitbake/gcc: Enable a shared common source treeRichard Purdie2011-03-281-0/+8
| | | | | | | | | | | | | | | | | | | | | This patch is a quick proof of concept to show how source code could be shared between recipes which use ${B} to have a separate build directory compared to source directory ${S}. Issues: a) gcc uses sed and creates config files against ${S} which means the directory should not be shared. Need to change the way that works. b) Could be extended to cover eglibc except there is a patch applied against nativesdk versions which again makes the source incompatible. c) Need to clean up the layout in work-shared and make a directory level deeper to ensure patch separation. d) clean task does not remove stamps Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: show correct task name for setscene task failurePaul Eggleton2011-03-151-3/+11
| | | | | | | | If a setscene task failed previously it was showing an incorrect task name in the error line. This patch ensures we show the correct name, also including the "_setscene" suffix. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* bitbake/cache/runqueue.py: Move workload for recipe parsing to the child processRichard Purdie2011-02-281-12/+23
| | | | | | | | | | | | | | | Parsing the recipe in the parent before forking off the child worker can mean the parent doesn't hit the idle loop and becomes a bottleneck when lauching many short lived processes. The reason we need this in the parent is to figure out the fakeroot environmental options. To address this, add the fakeroot variables to the cache and move recipe loadData into the child task. For a poky-image-sato build this results in about a 2 minute speedup (1.8%). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Drop PATH override. This was needed for the exec() change ↵Richard Purdie2011-02-211-2/+0
| | | | | | but not now we use fork() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify RunQueueStats.copyChris Larson2011-02-211-4/+1
| | | | | | | (Bitbake rev: 778571f155139fcf43d0e30e875c39d6592ae03f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: pass a copy of the RunQueueStats to eventsChris Larson2011-02-211-1/+9
| | | | | | | | | | This avoids cases where the stats are modified after the event is fired but before it's dispatched to the UI. (Bitbake rev: 1954f182687a0bd429175dda87f05d8a94bb403a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Move SIGCHLD handler to wrap none server sleep callRichard Purdie2011-02-181-7/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure child has the default SIGCHLD handler restoredRichard Purdie2011-02-161-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we successfully fork off a task and there are more ↵Richard Purdie2011-02-151-0/+2
| | | | | | processes we can start do so immediately Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we have successfully collected a pid value return ↵Richard Purdie2011-02-151-0/+1
| | | | | | True to ensure we don't sleep in the main loop Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Add a SIGCHLD handler to ensure we wake from sleeps for ↵Richard Purdie2011-02-151-0/+6
| | | | | | SIGCHLD events Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Avoid starvation of events to the serverRichard Purdie2011-01-281-1/+1
| | | | | | | | | | | | | | The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since most new LogRecord events are larger than this it leads to a build up of data which is only processed slowly, leading to a bottleneck and a slow down of all bitbake processes. Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging this. A large value has been left in for the read() command just to ensure some fairness amongst process handling if a task tries to log truly huge amounts of data to the server, or goes crazy and ensures the main loop doesn't stall. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Introduce stamp-extra-info task flag into stamp filenamesDongxiao Xu2011-01-181-6/+6
| | | | | | | | | | | For certain tasks, we need additional information in build stamp file other than the task name and file name. stamp-extra-info is introduced as a task flag which is appended to the stamp file name. [Code simplifcations/tweaks from Richard] Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Renable check_stamp_fn() for now since we still fork for task ↵Richard Purdie2011-01-101-14/+10
| | | | | | execution (as per bitbake-upstream) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Simplify hashdata storage since we're not using an ↵Richard Purdie2011-01-101-17/+14
| | | | | | external script for tasks anymore, in line with bitbake-upstream Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the ↵Richard Purdie2011-01-101-5/+6
| | | | | | BasicHash siggen code Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Drop unneeded import and improve quieterrors handling as per ↵Richard Purdie2011-01-101-3/+5
| | | | | | upstream bitbake Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update event queue handling to match upstream bitbakeRichard Purdie2011-01-101-4/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Sync with changes in upstream bitbakeRichard Purdie2011-01-101-90/+80
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Remove duplicate TaskStarted event fire()Richard Purdie2011-01-071-2/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignoredRichard Purdie2011-01-061-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake build/siggen/runqueue: Fix stampfile parametersRichard Purdie2011-01-061-6/+6
| | | | | | | | The current parameters are not useful to the stampfile generator function as they can't uniquely define a task. This updated things so the parameters can identify unique tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-11/+14
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue: fix typo in documentation of calculate_task_weightsBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 5cde120003af97a5cf5c6eff2a02bb1480f9414b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue: use fewer newlines in error pathBernhard Reutner-Fischer2011-01-041-2/+2
| | | | | | | (Bitbake rev: 7498466f0e42beea6f5f411209a892b636c7783c) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* logger usage cleanupChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: 976e4f84a8147ad762442df7ff4820611a21d227) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Use os.devnull, not /dev/nullChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: ce5ab0fc524a1c2c48c4c39d6fc8aae23019207b) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Transfer noexec runqueue messages to the UIRichard Purdie2011-01-041-10/+7
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move the runq task start/failed messages to the UIChris Larson2011-01-041-11/+4
| | | | | | | (Bitbake rev: ab831e867f09b47001cb8da2f8f060e04febf237) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: stop loading the cache for -bChris Larson2011-01-041-2/+2
| | | | | | | | | | | Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Prefer xrange over range for small performance gain.Bob Foerster2011-01-041-27/+27
| | | | | | | | | | | | | | range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix exit code display for task failureChris Larson2011-01-041-2/+4
| | | | | | | | | | | Per the python documentation, os.waitpid returns the exitcode shifted up by 8 bits, and we weren't compensating, resulting in a display of 'failed with 256' when a worker process exits with a code of 1. (Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* FIXUP: runqueue bb.debug refsRichard Purdie2011-01-041-2/+2
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Switch bitbake internals to use logging directly rather than bb.msgChris Larson2011-01-041-46/+44
| | | | | | | | | | | | | | We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug (Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Use logging in the knotty ui, and pass the log record across directlyChris Larson2011-01-041-1/+9
| | | | | | | | | | | | | | | | | | This kills firing of Msg* events in favor of just passing along LogRecord objects. These objects hold more than just level and message, but can also have exception information, so the UI can decide what to do with that. As an aside, when using the 'none' server, this results in the log messages in the server being displayed directly via the logging module and the UI's handler, rather than going through the server's event queue. As a result of doing it this way, we have to override the event handlers of the base logger when spawning a worker process, to ensure they log via events rather than directly. (Bitbake rev: c23c015cf8af1868faf293b19b80a5faf7e736a5) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Split out 'find next buildable task' into a separate generator functionChris Larson2011-01-041-7/+13
| | | | | | | | | | It needs to be a generator, so scheduler subclasses have the option to skip buildable tasks and return a later one. (Bitbake rev: a8c61e41bc6277222e4cde667ad0b24bd1597aa0) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Let the RunQueueScheduler control the number of active tasksChris Larson2011-01-041-12/+8
| | | | | | | (Bitbake rev: 1387423e747f59866fd1cb99a7d90605e668823f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Add the ability to use runqueue schedulers from the metadataChris Larson2011-01-041-7/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | If you create a runqueue scheduler class in a python module, available in the usual python search path, you can now make it available to bitbake via the BB_SCHEDULERS variable, and the user can then select it as they select any other scheduler. Example usage: In a test.py I placed appropriately: import bb.runqueue class TestScheduler(bb.runqueue.RunQueueScheduler): name = "myscheduler" In local.conf, to make it available and select it: BB_SCHEDULERS = "test.TestScheduler" BB_SCHEDULER = "myscheduler" (Bitbake rev: 4dd38d5cfb80f9bb72bc41a629c3320b38f7314d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix IndexError encountered when a 'depends' flag references a nonexistant taskChris Larson2011-01-041-1/+3
| | | | | | | (Bitbake rev: 4b0fd70539e73d99282fa89d47ad2d5f642ca4f4) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* When shutting down, send SIGTERM, not SIGINTChris Larson2010-12-311-2/+2
| | | | | | | | | | | | SIGINT should be from the user, not a script. It also doesn't work as reliably to shut down processes, as it's not always interpreted as a termination request. In addition, it causes KeyboardInterrupt exceptions in the worker processes, which can interfere with our exception handling. (Bitbake rev: e5f6e0e9de4c6d1dfdd269d2bf7f83c00c415a27) Signed-off-by: Chris Larson <clarson@kergoth.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue.py: set BB_TASKHASH approriately after moving to fork()Kevin Tian2010-12-211-1/+2
| | | | | | | | | In previous exec() model, cooker is re-initialized from scratch with environmental variable exported accordingly. Now in fork() model, environmental variables are not exported again, and thus original method to export BB_TASKHASH doesn't apply now which breaks all sstate packages. Now we can set data variable directly instead. Signed-off-by: Kevin Tian <kevin.tian@intel.com>
* bitbake/runqueue.py: Somehow the python environment mapping is failing so do ↵Richard Purdie2010-12-211-2/+8
| | | | | | it manually Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake Revert bitbake exec() and go back to fork() for performace wins ↵Richard Purdie2010-12-171-54/+89
| | | | | | (first draft) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake/runqueue: Revert previous setscene noexec changes and ensure ↵Richard Purdie2010-12-171-1/+3
| | | | | | setscene stamp exists for setscene noexec tasks Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-06-29 02:25:44 +0000