linux/poky.git/bitbake/bin/bitbake-worker, branch python3

bitbake: bitbake: Convert to python 3 megacommit This needs breaking up into smaller changes.

2016-05-16T22:32:40+00:00

(Bitbake rev: cf51f19aed208a75d38c14cd585d9b9f115e3ba3) Signed-off-by: Richard Purdie

bitbake: bin/bitbake-worker: Fix invalid bb.msg.fatal usage

2016-05-13T12:41:32+00:00

The logging domain specified to bb.msg.fatal was invalid. Replace with a logger.critical() call instead. (Bitbake rev: 1ffd8737e065a3cd634c74cd67e634d785ea93a5) Signed-off-by: Richard Purdie

bitbake: bitbake: Update logger.warn() -> logger.warning()

2016-05-11T09:34:30+00:00

python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie

bitbake: cooker, bitbake-worker: Fix spelling of "received"

2016-02-01T11:20:42+00:00

I before E, except after C... (Bitbake rev: 14c9593265f7469cb8a205a46f845ac7491246df) Signed-off-by: Phil Blundell Signed-off-by: Richard Purdie

bitbake: bitbake: Set process names to be meaninful

2016-01-30T11:43:56+00:00

This means that when you view the process tree, the processes have meaningful names, aiding debugging: $ pstree -p 30021 bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593) ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634) └─{ProcessEQueue}(115591) $ pstree -p 30021 bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335) ├─PRServ(117332)───{PRServ Handler}(117333) ├─Parser-1:2(117336) └─{ProcessEQueue}(117331) Applies to parse threads, PR Server, cooker, the workers and execution threads, working within the 16 character limit as best we can. Needed to tweak the bitbake-worker magic values to tell the workers apart. (Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Guard against multiprocessing corruption of event data

2015-10-01T06:43:38+00:00

In the forked child, we may use multiprocessing. There is only one event pipe to the worker controlling process and if we're unlucky, multiple processes can write to it at once corrupting the data by intermixing it. We don't see this often but when we do, its quite puzzling. I suspect it only happens in tasks which use multiprocessng (do_rootfs, do_package) and is much more likely to happen when we have long messages, usually many times PAGE_SIZE since PAGE_SIZE writes are atomic. This makes it much more likely within do_roofs, when for example a subprocess lists the contents of a rootfs. To fix this, we give each child a Lock() object and use this to serialise writes to the controlling worker. (Bitbake rev: 3cb55bdf06148943960e438291f9a562857340a3) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Ensure pipe closure doesn't crash before killpg()

2015-09-12T21:50:14+00:00

If the pipe is closed, we want to ensure that we kill any child processes by triggering the sigterm handler before we exit. This code does that, hopefully avoiding the remaining process left behind issues on the autobuilder. (Bitbake rev: 60f6c2818f38c4d9c2d9aaa42acf3071636f4a3b) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Handle SIGKILL of parent gracefully

2015-09-09T13:27:57+00:00

If we SIGKILL cooker (the parent process), ensure the worker notices and shuts down gracefully. To do this: * trigger the sigterm handler if the parent exits * ensure broken pipe writes don't trigger backtraces which interfer with other exit work * notice if our command pipe is broken due to EOF and sigterm if so (Bitbake rev: c43d6a8d711db8d3bd9a1976b9f8e3efdb4cb4ae) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Simple code cleanup

2015-09-09T13:27:56+00:00

start/end are unused here and we can improve the code conditional blocks. (Bitbake rev: 68f53dd77fe0bbfa044bd037a9484e0e1c9088b4) Signed-off-by: Richard Purdie

bitbake: bitbake-worker: Fix regression with unbuffered logs

2015-05-26T09:33:06+00:00

I noticed that I was seeing loss of the log files when hitting control-c while debugging a function in bitbake. In fact if you take a recipe and replace its compile function as shown below let it run for a few seconds and hit control-c, you will see first hand that log data is not there. do_compile () { while [ 1 ] ; do echo -n "Output date: " date sleep 1 done } It turns out there was a regression introduced by commit: d0f0e5d9e69 which created the bitbake worker. Since the bitbake worker is started in its own process space, it needs the exact same code added from commit: 88429f018b where the problem was fixed the first time around. (Bitbake rev: 8d1748f75763b4a66516cc46d5457ee6404b1b68) Signed-off-by: Jason Wessel Signed-off-by: Richard Purdie