linux/poky.git/bitbake/lib/bb/process.py, branch uninative-2.1

bitbake: bitbake:process: flush stderr/stdout to log

2017-06-14T13:54:24+00:00

When a process terminates, some messages may still remain in stdout or stderr and do not make it into the log file. In addition, the messages that do make it to the log file may end up in the log file in incorrect order. This patch flushes all messages into the log file after the process terminates. Some additional log flushing is also needed to keep the various messages showing up in the log file in proper order. [YOCTO#10785] (Bitbake rev: 1f6e6aa8262369eafc3bbf9f01f8d981f90becdf) Signed-off-by: Juro Bystricky Signed-off-by: Richard Purdie

bitbake: process: stop bb.process.communicate mixing bytes and str return types

2017-03-01T11:16:07+00:00

Python3 regards b"" as False so it is not being converted to a string by d0f904d407f57998419bd9c305ce53e5eaa36b24. This confusingly causes three different potential types for each member of the returned tuple. Let's just assume that everything that's not None is a bytes object and convert it to a string. (Bitbake rev: 0cf5589b7fb3582a6caca5014c4d8152347df545) Signed-off-by: Mike Crowe Signed-off-by: Richard Purdie

bitbake: bitbake: Convert to python 3

2016-06-02T07:24:02+00:00

Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie

bitbake: Fix default function parameter assignment to a list

2015-08-19T17:05:44+00:00

With python you should not assign a list as the default value of a function parameter - because a list is mutable, the result will be that the first time a value is passed it will actually modify the default. Reference: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments (Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

bitbake: lib/bb/process: check output of select() before reading extrafiles

2015-07-16T14:09:26+00:00

We're calling select() to find the fds that have data available, so we really ought to check the return to see if the extra file is in there before trying to read from it. This is part of the fix for the performance regression that this code introduced (5-10 minutes extra in a reasonable size OE build); the rest is down to an issue in the way that pseudo currently handles FIFOs and will need to be addressed there, see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993 Solution suggested by Richard Purdie (Bitbake rev: 3e2db8d7eaa0f14813213d1c95d3ee9efd97dc9d) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

bitbake: lib/bb: set up infrastructure for shell message functions

2015-07-12T21:50:40+00:00

Create a fifo under ${T} for each running task that can be used by the task to send events back to BitBake. The purpose of this is to allow OpenEmbedded's logging.bbclass (which provides shell function equivalents for bb.warn(), bb.note() etc.) to have those functions trigger the appropriate events within BitBake so that messages are shown through the BitBake UI rather than just in the task log; to do that we just call the python functions. Part of the fix for [YOCTO #5275]. (Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38) Signed-off-by: Paul Eggleton Signed-off-by: Richard Purdie

bitbake: process: Add timeout to select call

2013-12-10T17:55:29+00:00

On some machines, a hang has been noticed where the system sits in the select call despite the task having completed. The exact reasons for this as unknown but adding a timeout unblocked the builds and resolved the hangs in question. (Bitbake rev: 5223ffb5b6a46d8b3f6ac3362bd2672e2edf2691) Signed-off-by: Richard Purdie

bitbake: lib: Use modern expcetion syntax

2013-05-09T21:28:24+00:00

(Bitbake rev: a4a37b6a83faa62f61433122c4583e93e64f7372) Signed-off-by: Richard Purdie

bitbake: lib: Clean up various file access syntax

2013-05-09T21:28:04+00:00

Python 3 is stricter about how files are accessed. Specficially: * Use open(), not file() * Use binary mode for binary files (when checksumming) * Use with statements to ensure files get closed * Add missing file close statements (Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f) Signed-off-by: Richard Purdie

bitbake: process: Improve _logged_communicate buffering

2012-06-25T13:57:16+00:00

There are two problems with the _logged_communicate that are both caused as a result of buffering I/O issues: 1) log truncation when python fails 2) While a bitbake task is running it is impossible to see what is going on if it is only writing a small incremental log that is smaller than the buffer, or you get only a partial log, up until the task exists. It is worse in the case that stderr and stdout are separate file handles, because previous code blocks on the read of stdout and then stderr, serially. The right approach is simply to use select() to determine if there is data available and also flush the log before exiting. This is based on a patch from Jason Wessel with some changes to flush upon exit, abstract the non blocking file descriptor setup and drop the buffer size parameter. (Bitbake rev: 361fb71e907aa84c28ecec79fefc6ca39c39172f) Signed-off-by: Richard Purdie