summaryrefslogtreecommitdiffstats
path: root/bitbake/lib
Commit message (Collapse)AuthorAgeFilesLines
* bitbake: main: Attempt to gain bitbake.lock rather than just waitingRichard Purdie2017-08-241-1/+9
| | | | | | | | | Rather than just waiting for 5s, try and get the lockfile. If we gain the lock, we know we're ready to retry and can skip any remaining timeout. (Bitbake rev: 8a60106c6f7d586c793b965c5e9460b6016fab15) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Clean up connection retry logicRichard Purdie2017-08-242-7/+14
| | | | | | | | | | | | | | | | | | Its possible for a connection to connect to the server as its shutting down but before its removed the socket file. This patch: a) Removes the socket file earlier to avoid connections. b) Handles EOFError in initial connections gracefully. These occur if the socket is closed during the server shutdown. c) Ensure duplicate events aren't shown on the console. This makes debugging these issues very very confusing. With these changes the backtrace that was concerning users is hidden and the server works as expected with a reconnect when it catches it in a bad state. (Bitbake rev: f45196cf84669723382730944dddc7eaf50826f2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: don't mandate path element in encodeurl()Paul Eggleton2017-08-231-6/+5
| | | | | | | | | | | | | URLs do not have to have a path; currently our npm URLs don't, so encodeurl() needs to handle if the path element isn't specified. This fixes errors using OpenEmbedded's devtool add / recipetool create on an npm URL after OE-Core revision ecca596b75cfda2f798a0bdde75f4f774e23a95b that uses decodeurl() and encodeurl() to change URL parameter values. (Bitbake rev: d5cab2dbf5682d2fd08e58316a3bf39a10f63df2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake: ui: Sort 'Dependent tasks' in taskexpJussi Kukkonen2017-08-231-1/+3
| | | | | | | | | | The underlying model is already sorted for use in the other view, add a sorting model for the 'Dependent Tasks' view. (Bitbake rev: 27ca94c33234f0ef9753f8285213dde2871a3fcf) Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: support custom Layer Index URL and fixture overrideDavid Reyna2017-08-232-2/+9
| | | | | | | | | | | | | | | | | | | | Toaster needs to allow the custom fixture file to specific an alternate Layer Index URL for users that host their own Layer Index Server via the 'CUSTOM_LAYERINDEX_SERVER' in 'custom.xml'. Toaster also needs to allow the ability to completely override the default fixture files, otherwise that content can leak into the custom environment (by default the custom fixture is an overlay that cannot remove existing values from the default fixture) via the 'CUSTOM_XML_ONLY' value in 'setting.xml'. [YOCTO #11938] (Bitbake rev: ac29d4a9078494544a2627d8b6b021096b49cb34) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Toaster: custom start and stop actionsDavid Reyna2017-08-231-0/+49
| | | | | | | | | | | | | | Toaster needs the ability to allow custom extensions to execute when Toaster is started and stopped. Toaster will look for a custom extension script in the fixtures directory and execute any applicable hooks. [YOCTO #11938] (Bitbake rev: 12a73f6914488029f2b9dd680e004fba7dde41af) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: custom image updates and original creationDavid Reyna2017-08-232-38/+62
| | | | | | | | | | | | | | | | | | | | | When Toaster removes a package from a custom image, it must also always remove the advised reverse-dependent recipes. Similarly, when adding a package it must always add its advised the packages it depends on. This code must be un-indented so that it applies to all respective added or removed packages. Toaster normally waits until a new custom image is built before creating the custom layer and the recipe. However, an intermediate different build can fail because the recipe has already been added to the project, so the image's default recipe must be created when the image is created. [YOCTO #11915] (Bitbake rev: bcd68fd7231f166baff875fb88e4f9ce0d9bf91d) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Ensure we call select() to know which fds to readRichard Purdie2017-08-221-1/+1
| | | | | | | | | | | | | | | | | | | There is an interesting bug in the current code where a sync command is not seen until the current async command completes, by which time the UI may have shut down. The reason is that if there are idle commands, we may not end up sleeping in the select call at all, partiularly under heavy load like parsing. Fix this by calling select with a zero timeout so that we see active fds and know to read from them. This fixes various problems toaster was having with the recent server changes. [YOCTO #11898] (Bitbake rev: bbcce58e824b2793abf50efa52db158ae16e23e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Increase server startup timeoutRichard Purdie2017-08-161-1/+1
| | | | | | | | | | We're seeing the server fail to start within 8s on heavily loaded autobuilders so increase this timeout to 30s which should be more than enough time. (Bitbake rev: 8d4c120ec46d6d7a54947c64d33e18cb60b60505) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Improve client disconnectsRichard Purdie2017-08-161-15/+23
| | | | | | | | | | There have been cases where the server could loop indefinitely and incorrectly handle client disconnects. In the EOFError case, ensure a full disconnect happens in the alternative disconnect path to avoid this. (Bitbake rev: 5e267f14bb0155889615f567a920af4a37eb3c6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ignore common bitbake files for the parse cache invalidationRichard Purdie2017-08-131-0/+3
| | | | | | | | | | | | | | Writes to the cookerdaemon log and/or the lockfile were meaning the parser cache was always being invalidated and reparsed. This is unnecessary so spot accesses to these two common cases and ignore the files from a reparse perspective. This doesn't remove many sources of reparse but does improve several common cases. (Bitbake rev: 218e4b6418992588312b8ef5949b84ef43263d1a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Improve inotify overflow handlingRichard Purdie2017-08-132-1/+8
| | | | | | | | | | Add a proper function for clearing the mtime cache. Clean up the inotify event overflow case to err on the side of caution and clear any potentially now out of sync caches. (Bitbake rev: ec60459fe2ba16966544eebff43b061abb7ed3ba) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure we handle inotify before running the next commandRichard Purdie2017-08-132-7/+12
| | | | | | | | | | | | | | The inotify watch events are handled "at idle" which could in theory mean a command could run before any preceeding inotify events have been processed. This leads to a theoretical race window where those events may have a signficicant effect on the command. Add a mechanism to allow us to ensure all pending events are processed before running commands. (Bitbake rev: bf76cd7e5881adf264b8ba64e27a5b6ca9df4fde) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Track directories searched for bbappend/bb filesRichard Purdie2017-08-131-3/+21
| | | | | | | | | | | | | | | | | Some of the directories searched by BBFILES are not currently being added to the inotify watch list. This can mean that added append files are not noticed leading to misleading metadata results when using BB_SERVER_TIMEOUT != 0. We use glob to expand the BBFILES references and without writing our own implentation, figuring out the directories it searches is hard. We use some horrible hacks here to intecept the listdir calls, I'm open to better ways to handle this but this does ensure we have the right watches set. (Bitbake rev: 4d508d35a224e3a25d2d59c8415ab7985964b14f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: server/process: Ensure we don't loop on client EOFErrorRichard Purdie2017-08-131-2/+2
| | | | | | | | | | | | | | The server currently crashes if we hit an EOFError due to controllersock still being in ready and the continue meaning ready isn't re-evaluated. Setting the value to False can mean the shutdown code doesn't handle the situation cleanly. Clear ready to avoid the crash/loop instead and handle any OSError whilst we're in here. (Bitbake rev: 2bc47e887c3b41417edaa89a8708c223fd2085de) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/siggen: Reset siggen when reparsingRichard Purdie2017-08-132-0/+4
| | | | | | | | | | | | If we don't do this, we get basehash mismatch errors occurring from the reparse which would then set bitbake's error exit code. This for example would cause oe-selftest -r bbtests.BitbakeTests.test_bbappend_order to fail with a non-zero BB_SERVER_TIMEOUT. (Bitbake rev: e4c6ca9440f63761560b49bbe12654441f54687e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Handle BB_SERVER_TIMEOUT = -1 for no server timeoutRobert Yang2017-08-092-3/+6
| | | | | | | | | Make BB_SERVER_TIMEOUT = -1 mean no unload forever. (Bitbake rev: 923d5567be6fd9969ca74c166f36817ec09305e3) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Fix disconnect when BB_SERVER_TIMEOUTRobert Yang2017-08-091-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed: $ export BB_SERVER_TIMEOUT=10000 $ bitbake --server-only $ bitbake --status-only [snip] File "/buildarea/lyang1/poky/bitbake/lib/bb/server/process.py", line 472, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) OSError: [Errno 9] Bad file descriptor And: $ export BB_SERVER_TIMEOUT=10000 $ bitbake --server-only -B localhost:-1 $ bitbake --status-only # Everything is fine in first run $ bitbake --status-only [snip] File "/buildarea/lyang1/poky/bitbake/lib/bb/server/process.py", line 472, in recvfds msg, ancdata, flags, addr = sock.recvmsg(1, socket.CMSG_LEN(bytes_size)) OSError: [Errno 9] Bad file descriptor This was because self.controllersock was not set to False, so it still ran sock.recvmsg() when sock was closed. And also need set command_channel to Flase, otherwise the self.command_channel.get() will always run when EOF, and cause infinite loop. (Bitbake rev: 7b739a38601b053d9bea4df2c0b44a952ab670c4) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Remove unneeded float()Robert Yang2017-08-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | There is already a type=float, so the float() is not needed, which also makes the error clearer: $ export BB_SERVER_TIMEOUT=10000AA With float(): $ bitbake quilt-native [snip] ValueError: could not convert string to float: '10000AA' Without float(): $ bitbake quilt-native [snip] optparse.OptionValueError: option --idle-timeout: invalid floating-point value: '10000AA' The second one tells clearly where is wrong. (Bitbake rev: 7a4ea1e6a10f20d80009a78f4f0aebf8f90095fd) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Revert "tinfoil: fix duplication of log messages"Paul Eggleton2017-08-092-4/+3
| | | | | | | | | | | | | | | | | In combination with the recent server reworking, this change actually prevents messages sent from tasks from being logged properly. This will of course give us the duplicated messages back, and I really hate to do that effectively a second time, but that's better than seeing no error at all in the case of a failure - we'll have to find the proper way of avoiding the duplication that doesn't result in some messages going missing. This reverts commit 8a5bae76f91f2411187c638a42fa3c762052cf11. (Bitbake rev: 645c8dd15762516ae5ab64a1df47fadb95d072d1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize: Always print any remaning UI events at exit2.4_M2Richard Purdie2017-08-031-1/+1
| | | | | | | | | | | | | If there are events in the UI queue we want to print them regardless of whether we're handling an exception or something like SystemExit. This improves error messages for some other failure cases where currently no logging would get printed and leave the user confused as to what went wrong. (Bitbake rev: 76c4f2c20216719736766e8ae7d089ccd061b71b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker: Ensure all tasks are shown in task dependency treeRichard Purdie2017-08-031-3/+3
| | | | | | | | | | | | "bitbake -g m4-native -c do_unpack" doesn't list any dependencies in task-depends.dot. This is because no header describing the task was added unless a task has dependencies. Tweak the code to fix this. [YOCTO #10893] (Bitbake rev: e2fbe728769f09ef4178e2ef4d69a2f60ffe3ebb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize: clean up codeRichard Purdie2017-07-311-134/+14
| | | | | | | | | | This was originally based on some other code but its mostly misleading comments now. Massively clean this up and accept its now a total fork. There are no funciton changes here, just cleanup. (Bitbake rev: 3daa9001269e5b9156d0eb3299d2628d2d219fff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: daemonize: Ensure child process exits safelyRichard Purdie2017-07-311-3/+8
| | | | | | | | | | | | | When we create the child, if an exception occurred it was transfering back into the parent context. We don't want to do that us use a try/finally to ensure we exit. We need to ensure a traceback is printed and any queued UI messages which may not have made it to the client UI at this point. (Bitbake rev: dec1d2c26f6cb3ffeb44beaab0129cd531a6d08b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Add some extra server startup logsRichard Purdie2017-07-311-1/+4
| | | | | | | | | | We have cases where the server is being started but we're not seeing any messages from it. Add some earlier logging so we can try and better understand where issues may be occurring. (Bitbake rev: 10a6a48c0c5a2fe2051e90143e66075356853971) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Reorder server command processing and handle EOFErrorRichard Purdie2017-07-311-13/+18
| | | | | | | | | | | | | If the connection control socket and the command channel close together, we can race and hit EOFError exceptions before we close the channel. Reorder the code to handle this in the correct order and ignore the EOFError exceptions as they mean the client is disconnecting and shouldn't terminate the server. (Bitbake rev: 974281ed72d8366baa16ee85f7e93970689b5956) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: wget: fix FusionForge workaroundRoss Burton2017-07-311-6/+3
| | | | | | | | | | | | | | | | | My previous assertion about FusionForge appears to have been wrong, or FusionForge has changed behaviour, or both. FusionForge now mandates that downloads have the Accept header set, despite that header being optional, and returns a 406 Not Acceptable error if it isn't set. As we were pretending that 406 was actually 405 (Moved) and tried to handle it as a redirect this results in an infinite loop until Python kills the recursion. Delete the handling of 406 as 405, and pass Accept: */* in the headers. (Bitbake rev: bb70ae0c9aac5ec688026d23a64ac0cac1947187) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: move to new bitbake xmlrpc defaultDavid Reyna2017-07-313-7/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | The bitbake option "-t SERVERTYPE" was deprecated and can be removed since the desired XMLRPC listener now the default. The bitbake server port cannot be "-1" anymore and must be explicitly passed. There is a race condition for the bblock file to not only be created but to actually be populated, so a delay test loop is required (usually only one pass). The 'xmlrpcclient' is now the class that allows Toaster to for example kill builds. The events for populating the recipe parsing now either show only the final result or are skipped entiredly, so the progress calculator needs to be changed to not block on the parsing phase. [YOCTO #11851] (Bitbake rev: 2aa7ad38f760ec003fb18faa5aa0014cff191a7a) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: enable remote HTTP API for status aggregationDavid Reyna2017-07-314-4/+100
| | | | | | | | | | | | | Add support for Toaster aggregators with a set of api links that return JSON data for (a) builds in progress, (b) builds completed, (c) specific build data, and (d) an is-alive health ping link. [YOCTO #11794] (Bitbake rev: d8e79661c69671dd424dca5cc3f7f2f855b0afed) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: runqueue: Tweak debug message to make it more readable/diffableRichard Purdie2017-07-311-1/+1
| | | | | | | | | Having this as one huge long line isn't easy to manipulate, split it into multiple lines for ease of debugging issues. (Bitbake rev: 5753fe81194f75fbcf4ccdc733cc585d02794cb1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2: fix checkstatus fallback to MIRRORSRoss Burton2017-07-311-3/+2
| | | | | | | | | | The checkstatus() code was expecting checkstatus to throw exceptions if it failed, but in general it should return False. (Bitbake rev: 57be5cc6228518e60f564570a39cebbeb6cf564e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: tests/fetch: fix GitShallowTest.test_bitbakeChristopher Larson2017-07-301-1/+1
| | | | | | | | | | | | | | | `git fetch --tags` seems to interact badly with `mirror=fetch`, resulting in the regular branches not being fetched, so drop the unnecessary `--tags`. This fixes this unit test failure: `bb.fetch2.FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /tmp/tmp4ag_mgmn/gitsource` [YOCTO #11698] (Bitbake rev: 2d0203fae08c5ff8dc3e9afaa9a819abc4a1af6f) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: main: Always return 0 for bitbake --kill-serverRichard Purdie2017-07-301-2/+4
| | | | | | | | | | If the server isn't running return 0 as to do otherwise complicates scripts which do cleanup of bitbake servers which would potentially be memory resident. (Bitbake rev: 0f85c54385f325f3ebbf911b35bbac11b8863b6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Clean up server communication timeout errorsRichard Purdie2017-07-301-12/+3
| | | | | | | | | | | | | | | | | | | | | This timeout path was commonly hit due to errors starting the server. Now we have a better way to handle that, the retry logic can be improved and cleaned up. This patch: * Makes the timeout 5s rather than intervals of 1s with a message. Paul noted some commands can take around 1s to run on a server which has just been started on a loaded system. * Allows a broke connection to exit immediately rather than retrying something which will never work. * Drops the Ctrl+C masking, we shouldn't need that anymore and any issues would be better handled in other ways. This should make things clearer and less confusing for users and is much cleaner code too. (Bitbake rev: 8633b7cd03cfaba3e0359aa5da22fc76b66768c7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Don't leak open pipes upon reconnectionRichard Purdie2017-07-301-1/+18
| | | | | | | | | If we reconnect to the server, stop leaking pipes and clean up after ourselves. (Bitbake rev: f41e4e971e807157be68cf4496580494b8b60643) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process/cooker: Allow UI process to know if the cooker was started ↵Richard Purdie2017-07-302-3/+23
| | | | | | | | | | | | | | | | | | | successfully Currently if the server fails to start, the user sees no error message and the server will be repeatedly attempted to be started until some longer timeouts expire. There are error messages in the cookerdeamon log but nobody thinks to look there. Add in a pipe which can be used to tell the starting process whether the cooker did actually start or not. If it fails to start, no further attempts can be made and if present, the log file can be shown to the user. [YOCTO #11834] (Bitbake rev: 57000d44beb1aeba37dfc70782b0d6418943acc5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Move socket keep alive into BitBakeProcessServerConnectionRichard Purdie2017-07-301-4/+6
| | | | | | | | | This cleans up the socket keep alive into better class structured code and adds cleanup of the open file descriptors upon shutdown. (Bitbake rev: 77fd3a3a29a569e212374b27aea742ddbaafcdd5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Allow BBUIEventQueue to exit cleanlyRichard Purdie2017-07-301-3/+8
| | | | | | | | | Currently the monitoring thread exits with some error code or runs indefinitely. Allow closure of the pipe its monitoring to have the thread exit cleanly/silently. (Bitbake rev: 930d077637928213e13a07c78fee3bf7a8c37ebf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Ensure ConnectionReader/Writer have fileno() and close() ↵Richard Purdie2017-07-301-0/+9
| | | | | | | | | | | methods Expose the underlying close() and fileno() methods which allow connection monitoring and cleanup. (Bitbake rev: f79187f4ebfad7969be47b429995e7f7a3e33c1e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: event: Don't write duplicate logs to stdout and stderr in no UI caseRichard Purdie2017-07-301-10/+17
| | | | | | | | | | | | This code would duplicate messages to stdout and stderr when no UI connected and there were error level messages. Rework the code so it either uses stderr (for errors and above) or stdout for warnings/debug but not both for the same messages. (Bitbake rev: 45cff5734ba2ba8c8d36d17d722a5804d39b258b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: bitbake-diffsigs: fix regression after recent server changesPaul Eggleton2017-07-272-0/+19
| | | | | | | | | | | | | | | | | | We were bridging the gap between the server and UI here by calling a bb.siggen.find_siginfo, a function defined and set on that module from the metadata. This worked from the UI side before but since the recent server changes is no longer accessible. Create a new command so this can execute on the server side and return the result by way of a new event. (We're still running compare_sigfiles() on the signature generator but that isn't quite the same thing and does still work.) Fixes [YOCTO #11844]. (Bitbake rev: fdcea991baa4f83d9c98d468d7b49c8c388a4a15) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/fetch2/__init__.py: Fix unpack commentMark Hatle2017-07-271-1/+1
| | | | | | | (Bitbake rev: b8f477de204ab5d0680b2b7c42370d13395be46c) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils.py: Add missing debug levelMark Hatle2017-07-271-2/+2
| | | | | | | (Bitbake rev: a0cd748d2f830a305da086eff3462875f64f2a70) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/__init__.py: replace stray logger.warn() with logger.warning()Andre McCurdy2017-07-251-1/+1
| | | | | | | | | | | | | | Update stray usage of deprecated logger.warn(), which was introduced to fetch2/__init__.py after all other instances had been replaced by logger.warning(): http://git.openembedded.org/bitbake/commit/?id=5a53e7d7b017769a6eb0f0a6335735a1fe51a5ec http://git.openembedded.org/bitbake/commit/?id=676a5f592e8507e81b8f748d58acfea7572f8796 (Bitbake rev: 1b14f115a9e929e29e91e8ac70826a3fe7259961) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch/wget: mitigate a wget race condition when listing FTP directoriesRoss Burton2017-07-251-12/+11
| | | | | | | | | | | | | | | | | | | | | | When wget is fetching a listing for a directory over FTP it writes to a temporary file called .listing in the current directory. If there are many such operations happening in parallel - for example during 'bitbake world -c checkpkg' - then up to BB_NUMBER_THREADS instances of wget will be racing to write to, read, and delete the same file. This results in various failures such as the file disappearing before wget has processed it or the file changing contents, which causes checkpkg to randomly fail. Mitigate the race condition by creating a temporary directory to run wget in when doing directory listings. [ YOCTO #11828 ] (Bitbake rev: 91d4ca93df092cf86ab84faaa94cc66ff9f43057) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: process: Change timeout warning to a noteRichard Purdie2017-07-241-1/+1
| | | | | | | | | | | The warning message currently shown can occur more frequently than previously if a previous bitbake server is shutting down and we're reconnecting to a new server. Change it to a note message to match the higher level connection logging retry messages and so as not to interfer with selftests. (Bitbake rev: b7514340cd6a2753eb217b059229bb279c3849ec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: cooker/process: Drop server_main functionRichard Purdie2017-07-242-23/+21
| | | | | | | | | | | | | | Now that there is only one server, this abstraction is no longer needed and causes indrection/confusion. The server shutdown is also broken with the cooker post_server calls happening too late, leading to "lock held" warnings in the logs if PRServ is enabled. Remove the abstraction and put the shutdown calls in the right order with respect to the locking. (Bitbake rev: c0ddde7cf680225127d6285685652b905ed176c3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: prserv/serv: Improve process exit handlingRichard Purdie2017-07-241-2/+17
| | | | | | | | | | | | | | | | | | | The server shutdown is currenlty laggy and race prone. This patch: * adds a waitpid so that no zombie server is left around if its not running in daemon mode. * adds a quit "sentinal" using a pipe so that we're not sitting in a socket poll() until timeout in order just to quit. * use a select() call to poll the socket and the pipe for a quit signal. The net result of this change is that the prserv exits with the cooker server and it does so immediately and doesn't wait for the select/poll calls to timeout. This makes bitbake a lot more responsive for startup/shutdown and doesn't cause UI timeout errors as often when prserv is used. (Bitbake rev: 0b5a837477d18442caf97dd0fa14a806c54f5842) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: fetch2/wget.py: improve error handling during sstate checkPatrick Ohly2017-07-211-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the sstate is accessed via HTTP, the existence check can fail due to network issues, in which case bitbake silently continues without sstate. One such network issue is an HTTP server like Python's own SimpleHTTP which closes the TCP connection despite an explicit "Keep-Alive" in the HTTP request header. The server does that without a "close" in the HTTP response header, so the socket remains in the connection cache, leading to "urlopen failed: <urlopen error [Errno 9] Bad file descriptor>" (only visible in "bitbake -D -D" output) when trying to use the cached connection again. The connection might also get closed for other reasons (proxy, timeouts, etc.), so this is something that the client should be able to handle. This is achieved by checking for the error, removing the bad connection, and letting the check_status() method try again with a new connection. It is necessary to let the second attempt fail permanently, because bad proxy setups have been observed to also lead to such broken connections. In that case, we need to abort for real after trying twice, otherwise a build would just hang forever. [YOCTO #11782] (Bitbake rev: 6fa07752bbd3ac345cd8617da49a70e0b2dd565f) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: toaster: trim build target inputDavid Reyna2017-07-211-2/+2
| | | | | | | | | | | | Trim the entered built target value so that Toaster is not confused with no real targets nor a ghost second target. [YOCTO #11727] (Bitbake rev: dd83c732b945ba6f9b7cdf66f6e88cfd15a745a0) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-11-15 06:19:55 +0000