summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/runqueue.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Add FAKEROOTNOENV variableRichard Purdie2011-11-101-0/+6
| | | | | | | | | | | | | | | | | Add a FAKEROOTNOENV which does the opposite of the FAKEROOTENV variable and is data loaded into the environment for tasks without the fakeroot flag. The intent here is to provide a way to control the environment when we aren't needing fakeroot context which allows us to unload the preload from memory entirely and gain a valuable speedup. I'm not 100% happy with needing this at the bitbake level, particularly with the cache hit but it does give a valuable speedup. (Bitbake rev: 05c29ab0b2ad3c521414cabb6a92bca15c6e919c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Ensure fakeroot variables are reflected in the datastoreRichard Purdie2011-09-091-0/+1
| | | | | | | | | | Without this, variables can be set to one thing in one part of the environment and something different in another part. This change ensures the datastore and the environment are consistent. (Bitbake rev: 459addf13721a6847406f215650fa1882fb83ea9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Ensure task environment is correctRichard Purdie2011-09-091-1/+10
| | | | | | | | | | | | | | | | | | This fixes two problems: a) Variables which were in the parent environment but not set as "export" variables in the datastore could end up in the task environment b) oe.environ.update() can't cope with the generator returned by bb.data.exported_vars() Whilst the updated code isn't as neat, it does do the expected thing, sets the environment correctly and stops unwanted values leaking into the task environment. (Bitbake rev: 67e5e23034c5ec2b9efcca935242830306c0048d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fixup remaining bb.msg.domain usersRichard Purdie2011-08-151-10/+10
| | | | | | (Bitbake rev: d5abdacaf9ac604ef8d8c1bafb9b30617827cb4f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Add task specific stamp file supportRobert Yang2011-06-281-0/+10
| | | | | | | | | | | | | | | | | | This patch, based on proof of concept code from Richard adds code to bitbake to allow individual tasks to optionally specify their stamp file using the stamp-base flag. This takes the same form as the STAMP variable but can be specified on a per task basis. Code is also added to runqueue to ensure that if two tasks share the same stamp file, only one will be executed at once. A significant usecase for this code is to share source code (${S}) between recipes where separate build directories (${B}) are used. (Bitbake rev: 41bef02bef8379590ba012319aebe05068a8081e) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Add umask task controlMark Hatle2011-06-241-0/+11
| | | | | | | | | | | | | | | | The umask for a task can now be set as: task[umask] = 022 task[umask] = '022' If specified as a text string, it must be octal. (This is due to recipe parsing where it's always set to a string.) [RP tweaked to use None instead of -1] (Bitbake rev: a5caaaaba8f0db1af5d8f2e610021d6d4b56894e) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* make exception handling syntax consistentScott Garman2011-06-151-1/+1
| | | | | | | | | | Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/data/runqueue: Sync up with upstream to clean up environment ↵Richard Purdie2011-06-081-10/+1
| | | | | | variable handling Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Wrap parsing with exception trappingRichard Purdie2011-06-081-24/+24
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/cooker.py: Misc sync with upstreamRichard Purdie2011-06-081-5/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Move BB_WORKERCONTEXT to a place where anonymous python ↵Richard Purdie2011-05-311-1/+2
| | | | | | can see it Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure existing setscene stamp files are taken into accountRichard Purdie2011-05-271-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | JaMa reported issues where bitbake would rebuild things instead of using the existing built tasks. This was tracked to a case where: a) rm_work is uses b) A depends on B c) B has a version change (e.g. PR bump) and A *and* B would then rebuild. It turns out that rm_work was correctly turning stamp files into the correct _setscene varients but bitbake was then ignoring them during setscene processing. If the correct sstate checksumed files didn't exist, everything would seemingly rebuild. The fix is to check for existing *_setscene stamps and if present, honour them. If "basichash" is enabled, the hash is included with the stamps so everything should then function as intended. (Bitbake rev: 0a4d857aabc86b973170ba9ce32b6b449a4e2165) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Remove old log file handling remnantsRichard Purdie2011-05-051-6/+0
| | | | | | These lines date from earlier code and are no longer required. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure fakeroot env variables make it to the child processRichard Purdie2011-03-311-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify fakeroot environment handlingChris Larson2011-03-311-15/+17
| | | | | | | (Bitbake rev: 88f0d1db88bdba95e488fba34f40dc0b72a85c33) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix more incorrect usages of 'is'Chris Larson2011-03-311-2/+2
| | | | | | | (Bitbake rev: a26a2f548419af0e971ad21ec0a29e5245fe307f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: fix clash when setscene & real tasks done in same buildPaul Eggleton2011-03-291-0/+19
| | | | | | | | | | | | | | | | | If a build causes a real task to be run when the setscene task has already run then it was possible for dependent packages to be rebuilding at the same time as a rebuild of the packages they depended on, resulting in failures when files were missing. This change looks in the setscene covered list and removes anything where a dependency of the real task is going to be run (e.g. do_install is going to be run even though the setscene equivalent of do_populate_sysroot has already been run). As an additional safeguard we also delete the stamp file for the setscene task under these circumstances. Fixes [YOCTO #792] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Revert "bitbake/gcc: Enable a shared common source tree"Richard Purdie2011-03-291-8/+0
| | | | | This reverts commit 12b163dbd81cafafec1ebe3c4039c65af60ee261 as it was an unintended change for master
* bitbake/gcc: Enable a shared common source treeRichard Purdie2011-03-281-0/+8
| | | | | | | | | | | | | | | | | | | | | This patch is a quick proof of concept to show how source code could be shared between recipes which use ${B} to have a separate build directory compared to source directory ${S}. Issues: a) gcc uses sed and creates config files against ${S} which means the directory should not be shared. Need to change the way that works. b) Could be extended to cover eglibc except there is a patch applied against nativesdk versions which again makes the source incompatible. c) Need to clean up the layout in work-shared and make a directory level deeper to ensure patch separation. d) clean task does not remove stamps Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: show correct task name for setscene task failurePaul Eggleton2011-03-151-3/+11
| | | | | | | | If a setscene task failed previously it was showing an incorrect task name in the error line. This patch ensures we show the correct name, also including the "_setscene" suffix. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* bitbake/cache/runqueue.py: Move workload for recipe parsing to the child processRichard Purdie2011-02-281-12/+23
| | | | | | | | | | | | | | | Parsing the recipe in the parent before forking off the child worker can mean the parent doesn't hit the idle loop and becomes a bottleneck when lauching many short lived processes. The reason we need this in the parent is to figure out the fakeroot environmental options. To address this, add the fakeroot variables to the cache and move recipe loadData into the child task. For a poky-image-sato build this results in about a 2 minute speedup (1.8%). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Drop PATH override. This was needed for the exec() change ↵Richard Purdie2011-02-211-2/+0
| | | | | | but not now we use fork() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: simplify RunQueueStats.copyChris Larson2011-02-211-4/+1
| | | | | | | (Bitbake rev: 778571f155139fcf43d0e30e875c39d6592ae03f) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: pass a copy of the RunQueueStats to eventsChris Larson2011-02-211-1/+9
| | | | | | | | | | This avoids cases where the stats are modified after the event is fired but before it's dispatched to the UI. (Bitbake rev: 1954f182687a0bd429175dda87f05d8a94bb403a) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Move SIGCHLD handler to wrap none server sleep callRichard Purdie2011-02-181-7/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Ensure child has the default SIGCHLD handler restoredRichard Purdie2011-02-161-0/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we successfully fork off a task and there are more ↵Richard Purdie2011-02-151-0/+2
| | | | | | processes we can start do so immediately Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: When we have successfully collected a pid value return ↵Richard Purdie2011-02-151-0/+1
| | | | | | True to ensure we don't sleep in the main loop Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue: Add a SIGCHLD handler to ensure we wake from sleeps for ↵Richard Purdie2011-02-151-0/+6
| | | | | | SIGCHLD events Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Avoid starvation of events to the serverRichard Purdie2011-01-281-1/+1
| | | | | | | | | | | | | | The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since most new LogRecord events are larger than this it leads to a build up of data which is only processed slowly, leading to a bottleneck and a slow down of all bitbake processes. Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging this. A large value has been left in for the read() command just to ensure some fairness amongst process handling if a task tries to log truly huge amounts of data to the server, or goes crazy and ensures the main loop doesn't stall. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Introduce stamp-extra-info task flag into stamp filenamesDongxiao Xu2011-01-181-6/+6
| | | | | | | | | | | For certain tasks, we need additional information in build stamp file other than the task name and file name. stamp-extra-info is introduced as a task flag which is appended to the stamp file name. [Code simplifcations/tweaks from Richard] Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Renable check_stamp_fn() for now since we still fork for task ↵Richard Purdie2011-01-101-14/+10
| | | | | | execution (as per bitbake-upstream) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Simplify hashdata storage since we're not using an ↵Richard Purdie2011-01-101-17/+14
| | | | | | external script for tasks anymore, in line with bitbake-upstream Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the ↵Richard Purdie2011-01-101-5/+6
| | | | | | BasicHash siggen code Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue.py: Drop unneeded import and improve quieterrors handling as per ↵Richard Purdie2011-01-101-3/+5
| | | | | | upstream bitbake Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update event queue handling to match upstream bitbakeRichard Purdie2011-01-101-4/+3
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Sync with changes in upstream bitbakeRichard Purdie2011-01-101-90/+80
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Remove duplicate TaskStarted event fire()Richard Purdie2011-01-071-2/+0
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignoredRichard Purdie2011-01-061-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake build/siggen/runqueue: Fix stampfile parametersRichard Purdie2011-01-061-6/+6
| | | | | | | | The current parameters are not useful to the stampfile generator function as they can't uniquely define a task. This updated things so the parameters can identify unique tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Sync a load of whitespace and other non-functionality changes with ↵Richard Purdie2011-01-041-11/+14
| | | | | | bitbake uptream Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue: fix typo in documentation of calculate_task_weightsBernhard Reutner-Fischer2011-01-041-1/+1
| | | | | | | (Bitbake rev: 5cde120003af97a5cf5c6eff2a02bb1480f9414b) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* runqueue: use fewer newlines in error pathBernhard Reutner-Fischer2011-01-041-2/+2
| | | | | | | (Bitbake rev: 7498466f0e42beea6f5f411209a892b636c7783c) Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* logger usage cleanupChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: 976e4f84a8147ad762442df7ff4820611a21d227) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Use os.devnull, not /dev/nullChris Larson2011-01-041-1/+1
| | | | | | | (Bitbake rev: ce5ab0fc524a1c2c48c4c39d6fc8aae23019207b) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* bitbake: Transfer noexec runqueue messages to the UIRichard Purdie2011-01-041-10/+7
| | | | Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Move the runq task start/failed messages to the UIChris Larson2011-01-041-11/+4
| | | | | | | (Bitbake rev: ab831e867f09b47001cb8da2f8f060e04febf237) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* cooker: stop loading the cache for -bChris Larson2011-01-041-2/+2
| | | | | | | | | | | Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. (Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Prefer xrange over range for small performance gain.Bob Foerster2011-01-041-27/+27
| | | | | | | | | | | | | | range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. (Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0) Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Fix exit code display for task failureChris Larson2011-01-041-2/+4
| | | | | | | | | | | Per the python documentation, os.waitpid returns the exitcode shifted up by 8 bits, and we weren't compensating, resulting in a display of 'failed with 256' when a worker process exits with a code of 1. (Bitbake rev: 90c2b6cb24dc9c82f0a9aa9d23f2d1ed2e6ff301) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-26 10:11:53 +0000