summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: Revert "bb.fatal: Raise a BBHandledException instead of exiting"Richard Purdie2013-09-171-1/+2
| | | | | | | | | Sanity test failures are no longer fatal with this change so whilst its the right idea, the code paths need more work. This reverts commit a50017ba71250e1710a6425b60ac7e3f03d88295. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bb.fatal: Raise a BBHandledException instead of exitingRichard Purdie2013-09-171-2/+1
| | | | | | | | | | With new bitbake UIs having the cooker exit at 'random' points in the codebase is problematic. This patch raises an exception which matches the siutation instead. (Bitbake rev: a50017ba71250e1710a6425b60ac7e3f03d88295) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Optimise flag exclusion list handlingRichard Purdie2013-09-171-5/+4
| | | | | | | | | Move the variable lookup to the outer loop for performance, replacing a now unneeded parameter (after the previous changes). (Bitbake rev: 8a59a8707cecbde257fca169775ce8ff7709928b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Optimise flag lookup in build_dependenciesRichard Purdie2013-09-171-2/+5
| | | | | | | | | When looking up flag variable dependencies, large chunks of the function aren't needed. Optimise the function flow accordingly for speed. (Bitbake rev: 1bf3aee698ad35f6815ea2c75471a96511a29d55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Optimise build_dependencies a littleRichard Purdie2013-09-171-11/+10
| | | | | | | | | Instead of multiple calls to getVarFlag, make one call to getVarFlags, only expanding the flags that need to be expanded. This improves performance. (Bitbake rev: eba1e9545cc933820d40de96f023b2307b3c4d0b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Cache the fact a variable accesses another even if its ↵Richard Purdie2013-09-171-1/+1
| | | | | | | | | | | | unset If a variable references another but it isn't set at present, the reference wasn't stored. It really should be marked as a reference and the higher level dependency code can handle as appropriate. (Bitbake rev: b05b748b2153c941b95cd36fb22aaafc4dbf3791) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Allow flags to use the expand cacheRichard Purdie2013-09-171-1/+3
| | | | | | (Bitbake rev: a0122ab80df21597291ff32ff7fbaa4de0347a6f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Allow expansion of flags in getVarFlagsRichard Purdie2013-09-171-2/+3
| | | | | | | | | | Allow a list of flags to expand to be passed into getVarFlags. This is useful within bitbake itself to optimise performance of the dependency generation code. (Bitbake rev: a3ae7efdf750fc5bb9ff5a75defbcfdab1912dbe) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Add explict None checksAlexandru DAMIAN2013-09-171-2/+2
| | | | | | | | | | | Simple if xxx checks end up calling len(xxx). We're interested in the specific case of None which means we can break out the iterator much earlier after the first item. This adds in the specific tests for None in what is a hot path in the data store code which gives small performance gains. (Bitbake rev: a4d81e44a7cd3dafb0bf12f7cac5ff511db18e60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Cache an list of export variablesRichard Purdie2013-09-172-1/+8
| | | | | | | | | Compute a cache of the list of potential export variables so that we don't have to compute the list from scratch. (Bitbake rev: f41f46f7eaa6889edeb3a4e4ddedc07084686c60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Use direct iteration, not keys()Richard Purdie2013-09-171-2/+2
| | | | | | | | | | | Profiling shows the creation of keys() has overhead and we're better using an iterator rather than the memory associated with the huge list of keys when iterating the whoe datastore. We minimise the number of times we do this to twice only per recipe. (Bitbake rev: e63448d9ee331b0f45fb9a0197d0dbee49eb2fa0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: use the expand_cache in VariableParseRichard Purdie2013-09-171-0/+5
| | | | | | | | | | When in VariableParse, use the expand_cache if possible rather than looking up data. Ultimately it would come from the same place but this short cuts a heavily used code block for speed improvements. (Bitbake rev: f682b8b83d21d576160bac8dc57c4c989b4dc555) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Improve variable expansion regexpRichard Purdie2013-09-171-1/+1
| | | | | | | | | | | Debugging showed the variable expansion regexp was catching python expressions (starting with @). Since these are caught by their own dedicated regexp, stop matching these for the plain variable expansion for small performance improvements. (Bitbake rev: c630d564285f55f9db10c18269bd310df797430e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: siggen: Use lookup cache exclusivelyRichard Purdie2013-09-171-12/+3
| | | | | | | | | | All the values we need are already guaranteed to be in the lookupcache so rather than fetch variables again, just use the cache. This gives a small performance improvement and simplifies the code. (Bitbake rev: 8ffaba61da7f195d7c3b64dce35b6a56272aecae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data: Be explicit in data_db checkRichard Purdie2013-09-171-1/+1
| | | | | | | | | | | The if statement current causes the size of parent to be calcuated which is like a len() operation on a datastore. Since we're only interested whether the value is none, checking explictly for this gives a small performance gain. (Bitbake rev: 43a245bde318545ea75ca4ce7894395c1cf9b32a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Allow profiling of the parser in profile modeRichard Purdie2013-09-171-2/+22
| | | | | | (Bitbake rev: f8a6e4caed4dc3dcf207aecc4ea5f438027da8be) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Avoid duplication for taskdata creationAlexandru DAMIAN2013-09-171-40/+27
| | | | | | | | | | | | Clean-up to avoid duplication and promote code reuse to factor taskdata creation into a common function. [RP: minor tweaks] (Bitbake rev: 468c221449290c4f196e87f7d8e23fcd7db86135) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: add runQueueTaskSkipped eventAlexandru DAMIAN2013-09-171-3/+12
| | | | | | | | | | Adding a runQueueTaskSkipped to notify that the tasks that are not run either because they are set-scened or they don't need an update (timestamp was ok). (Bitbake rev: cf4a0c7aa82090876ae652b611acfab3ce2688f7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Add logfile to add TaskBase eventsRichard Purdie2013-09-161-10/+6
| | | | | | | | | | | We add the path to the logfile for all Task events except TaskInvalid so that we can trace back the logfile locations at some future point. TaskInvalid doesn't ever have a logfile. (Bitbake rev: 8344d84c609446f59f9619cc7ca0d693b7e2bbd6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: updateCache should rause exceptions, not sys.exitRichard Purdie2013-09-141-2/+2
| | | | | | | | | | | Exiting from the server is antisocial, instead we should raise an exception. This will correctly fail the current command and reset the server state. We use the handled exception since for these conditions to occur, something was already displayed to the user. (Bitbake rev: dacc94bcace85a2e95aee2dccd8e680c59e4545f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Cleanup error/interruption handlingRichard Purdie2013-09-141-3/+4
| | | | | | | | | | Only display a CommandFailed ERROR: message if there is an error to display. Only display an errors summary if we actually displayed errors. (Bitbake rev: 568ea00acd226d48e725bb01d4f8c410ed1eaa61) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: command: Treat empty messages as failures, not CommandCompletedRichard Purdie2013-09-141-1/+1
| | | | | | | | | Empty messages should trigger CommandFailed, not CommandCompleted as otherwise the exit code will be incorrect. (Bitbake rev: 70a8ead31f9ffc987d9c6db61a926f7a9af8f8b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/command: Add finishcommand to reset cooker stateRichard Purdie2013-09-142-1/+4
| | | | | | | | | | | | | After running a command on the server, it needs to reset to the initial state. This ensures that subsequent clients start from a known state and notice any configuration changes. Ultimately we may want to do more than this buts a good start and better than nothing. (Bitbake rev: dd15648fc2654b8d7c3e00ea7ab3dbf04f24f24b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Rename confusing 'stop' state to 'forceshutdown'Richard Purdie2013-09-148-19/+19
| | | | | | | | | | | | The shutdown state causes the server to finish what its doing, stop was them meant to completely stop it. It doesn't mean the server is stopped though. Renaming the current stop event for forceshutdown gives more meaning to what it actually does. The stopped namespace then becomes available to indicate a completely stopped server. (Bitbake rev: 12e9d33bfae5294e3870dfd1202f63383ad05e92) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Clean up init/reset configuration codeRichard Purdie2013-09-141-25/+24
| | | | | | | | | | | Currently the cooker event data isn't rebuilt upon reset and the cache configuration cannot be changed after init. These are both bad things and this patch refactors the init/reset code so that it is possible to reconfigure the server. (Bitbake rev: 1193b8d76fcb6cb87e9ec135a2514370d7dd90ac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Drop obsolete worker testRichard Purdie2013-09-141-5/+1
| | | | | | | | | This call only ever happens in cooker context now so we can drop the nasty worker check from here. (Bitbake rev: bc0b30199a8e3624c5b9914430adbcc7c6bd4497) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: xmlrpc: delete unused code BitBakeUIEventServerAlexandru DAMIAN2013-09-131-45/+0
| | | | | | | | | | | | | BitBakeUIEventServer is an unused class that pushes UI events over a separate thread. The current version of XMLRPC server works just fine with the classic UI event handlers, so this class is not needed. (Bitbake rev: 8e8e17631d790271b1be747c4b45059ec38ab606) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Account for changes in append/prepend/remove in the ↵Richard Purdie2013-09-121-3/+6
| | | | | | | | | | | | | | | config hash bitbake wasn't reparsing when _remove items were added to its configuration and equally, appends/prepends were also being badly tracked. This change enrures these variables are accounted for in the configuration hash. [YOCTO #5172] (Bitbake rev: 62914f9208ef2427a34daa523af857f4027900eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: utils: Fix BB_PRESERVE_ENVRichard Purdie2013-09-111-0/+2
| | | | | | | | | | | BB_PRESERVE_ENV wasn't working since data.inheritFromOS wasn't getting a correct list of keys to import into the data store. This fixes things so it does add all environment variables into the data store when BB_PRESERVE_ENV is used. (Bitbake rev: 843e9339c5ee3c99657a40a0e2c7dbd777b6ef06) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: runqueue: add sceneQueueTaskCompleted eventAlexandru DAMIAN2013-09-101-0/+6
| | | | | | | | | | | Adding an event to be fired when a scene task is completed. It is analogous to the run task completed event, and has been missing for some reason. (Bitbake rev: 73b8f4d3fbeaf1b330a66d76012d0a5cef8dbe2d) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue.py: check whether multiple versions of the same PN are due ↵Robert Yang2013-09-091-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to be built There would be an race issue if we: $ bitbake make-3.81 make-3.82 This because they are being built at the same time which would cause unexpected problems, for example: [snip] ERROR: Package already staged (/path/to/tmp/sstate-control/manifest-qemux86-make.populate-sysroot)?! ERROR: Function failed: sstate_task_postfunc [snip] Or there would be python's strack trace such as: [snip] *** 0004: mfile = open(manifest) 0005: entries = mfile.readlines() 0006: mfile.close() 0007: 0008: for entry in entries: Exception: IOError: [Errno 2] No such file or directory: xxx [snip] [YOCTO #5094] We can quit earlier to avoid this kind of issue when two versions of the same PN are going to be built since this isn't supported. (Bitbake rev: ab377c00c33a2d296bfda1b0b6c2a62b29d1004f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: providers.py: enhance the runtime debug degbug messgaeRobert Yang2013-09-091-1/+1
| | | | | | | | | | | | The runtime provider debug message is the same as the build time debug message, make them different would be better. [YOCTO #5067] (Bitbake rev: 92b624cbc2711d3d859994099fb63918dfd0031a) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Ensure data is committedRichard Purdie2013-09-082-1/+6
| | | | | | | | | | | | In exclusive mode, we need to complete the transaction for writes to make it to the database. Therefore add sync calls to ensure this happens. Autocommit mode is significantly (100 times) slower so caching the data is of significant benefit. (Bitbake rev: 4e55f7821786a59c2cd7dbd8bfa2a22f5f196e99) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: knotty: Reduce refresh of footerRichard Purdie2013-09-061-2/+4
| | | | | | | | | | | When displaying larger number of events the client can get caught up in displaying the footer, then immediately overwriting it. To avoid this, wait for pauses in the event stream before displaying the footer to give a slightly more friendly feel to the UI. (Bitbake rev: 5d706c7cd6ee8d83b67ff18312d4c8119bea8878) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: Ensure ${DATE} and ${TIME} are consistentPeter Kjellerstedt2013-09-061-0/+2
| | | | | | | | | | | | | | | Due to the worker split the ${DATE} and ${TIME} variables could end up with different values for different workers. E.g., a task like do_rootfs that is run within a fakeroot environment had a slightly different view of the time than another task that was not fakerooted which made it impossible to correctly refer to the image generated by do_rootfs from the other task. (Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hob: remove custom image from the images list when not neededCristiana Voicu2013-09-061-0/+4
| | | | | | | | | | | | | Selected custom image recipes should be cleared from the combo the moment you change your selection. The idea is to always perform the selection of those images in the same way (i.e through the "Select from my image recipes" option). [YOCTO #5001] (Bitbake rev: 94483ee5ae9f4051bccd660c4718c36564e17161) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hob: limit the description size when a custom image is savedCristiana Voicu2013-09-061-2/+11
| | | | | | | | [YOCTO #5003] (Bitbake rev: 9aec9ee41d4d893325d9bf92b8a53f2e68e4973d) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: data_smart: Move getVar expand cache handing to fix _remove operationsRichard Purdie2013-09-061-7/+6
| | | | | | | | | | | | | DISTRO_FEATURES_remove = "opengl" wasn't working as expected. The reason turned out the be the indirect reference to opengl and the fact _remove was operating on unexpanded data. This patch rearranges some code to ensure we operate on expanded data by moving the expand cache handing into getVarFlags instead of getVar. (Bitbake rev: 181899bd9665f74f8d1b22d2453616ad30d26d9e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: hob: remove PACKAGE_INSTALL variable setting from hobCristiana Voicu2013-09-041-3/+1
| | | | | | | | | | | | | | Before saving the packages for a custom images in a .bb file, the packages were saved in bitbake memory. Now all the variables are saved in conf file, so saving PACKAGE_INSTALL is not needed anymore. Moved were LINGUAS_INSTALL is set, because both conditions are for testing if a custom image is saved. [YOCTO #5101] (Bitbake rev: 8757f962b92e7668f40d2d8bd9e762b152f91f7b) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-worker: ensure BUILDNAME is available during executionPaul Eggleton2013-09-041-0/+1
| | | | | | | | | | | | | | | BUILDNAME is set from cooker by default, so since the worker split it will not be set when executing functions. In OpenEmbedded this results in /etc/version (which is populated from BUILDNAME) not having any content. Pass this variable value through to the worker explicitly to fix the issue. Fixes [YOCTO #4818]. (Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake/cooker: fix some calls of cookerdata.findConfigFile methodCristiana Voicu2013-09-021-3/+3
| | | | | | | | | Cookerdata.findconfigFile method has a new parameter. Changed some calls. (Bitbake rev: dce0f9d4afe0986e2dd0146944fc4ac9dde275e4) Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Fix scenequeue to pass file descriptors, not a floatRichard Purdie2013-09-021-1/+1
| | | | | | | | This was missed off in a previous patch. (Bitbake rev: ad7664edd40fa46e6f6fec2144403e3b6fc3a639) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/xmlrpc: stop server on client exitAlexandru DAMIAN2013-09-011-0/+5
| | | | | | | | | | | | If the server only expects a single client to connect (i.e. no bind parameter set, so there is no way for the clients to get the server port), stop the server after the first client exits. (Bitbake rev: eb6bae56f62082bf147045311154cbae4bca0f4c) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv/serv: Settle on two threads for optimal performanceRichard Purdie2013-09-011-14/+34
| | | | | | | | | | | | Using the threading mixin class resulted in large amounts of memory being used by the PR server for no good reason. Using a receiver thread and a thread to do the actual database operations on a single connection gives the same performance with a much saner memory overhead so switch to this. (Bitbake rev: e08455d5f3b8e96765942b9c3b9767c30650557d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: build: Fix profile file namesRichard Purdie2013-09-011-1/+1
| | | | | | | | | | | | | Using the basename of the .bb file is not unique, for example xxx-native and xxx can overwrite each other. If this happens whilst running, you can get odd backtraces as one file is parsed as another tries to write out new data. Avoid issues by using PN for the output filename instead. (Bitbake rev: c9534f8e59d44b885334607ed90a3be2e492ec69) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: serv/db: Don't use BEGIN/COMMITRichard Purdie2013-09-011-13/+3
| | | | | | | | | | Since we don't support using multiple servers on the same database file, don't use the BEGIN/COMMIT syntax and allow writes to the database to work ~100 times faster with no transaction locking. (Bitbake rev: 42144a54979658f93fbbb43f7e271c1fff4d88ff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: serv/db: Take an excluside lock on the databaseRichard Purdie2013-09-011-1/+1
| | | | | | | | | We only support one server using the database at a time so take an exclusive lock and avoid later lock overhead. (Bitbake rev: e3e39be6f2d063858c92971ce8ccd89c95d4f26d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: serv/db: Fix looping upon database locked issuesRichard Purdie2013-09-011-3/+4
| | | | | | | | | | | | | If the database is locked we will get an immediate error indicating so, there is no retry timeout. The looping code is therefore useless, the loop count is near instantly exceeded. Using a time based retry means we can wait a sensible time, then gracefully exit. (Bitbake rev: 9f9e6d87007ea87e62495705464f4232c996a165) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process, server/xmlrpc, runqueue: Use select.select() on ↵Richard Purdie2013-09-013-21/+28
| | | | | | | | | | | | | | | | | | | | | | fds, not time.sleep() The existing backend server implementations were inefficient since they were sleeping for the full length of the timeouts rather than being woken when there was data ready for them. It was assumed they would wake and perhaps did when we forked processes directory but that is no longer the case. This updates both the process and xmlrpc backends to wait using select(). This does mean we need to pass the file descriptors to wait on from the internals who know which these file descriptors are but this is a logical improvement. Tests of a pathaolgical load on the process server of ~420 rapid tasks executed on a server with BB_NUMBER_THREAD=48 went from a wall clock measurement of the overall command execution time of 75s to a much more reasonable 24s. (Bitbake rev: 9bee497960889d9baa0a4284d79a384b18a8e826) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv: Allow 'table is locked' matching for retry loopRichard Purdie2013-08-301-1/+1
| | | | | | | | | Try and avoid errors like "ERROR: database table is locked: PRMAIN_nohist" by retrying if we see the string "is locked". (Bitbake rev: 1a175b51f80d13f747b653d29e9c0d2201b5109c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-12 08:00:05 +0000