linux/poky.git/bitbake, branch uninative-3.2

bitbake: server/process: Handle error in heartbeat funciton in OOM case

2021-05-18T22:53:15+00:00

We've seen cases where an OOM error causes bitbake server to hang: 9171 02:21:09.127810 Command Completed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/bin/bitbake-server", line 51, in bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 550, in execServer server.run() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 108, in run ret = self.main() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 242, in main ready = self.idle_commands(.1, fds) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 370, in idle_commands bb.event.fire(heartbeat, self.cooker.data) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 216, in fire fire_class_handlers(event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 123, in fire_class_handlers execute_handler(name, handler, event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 93, in execute_handler ret = handler(event) File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 182, in defaultrun_buildstats write_host_data(os.path.join(bsdir, "host_stats"), e, d, "interval") File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 160, in write_host_data output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child restore_signals, start_new_session, preexec_fn) OSError: [Errno 12] Cannot allocate memory We need to wrap the calls in the same high level wrapper as idle function calls and trigger an exit upon an unhandled exception. (Bitbake rev: 74042b5b89d5a170013fc1a327ce3a6530fbf7d5) Signed-off-by: Richard Purdie

bitbake: hashserver/client.py: drop unused imports

2021-05-14T06:57:57+00:00

(Bitbake rev: 9dc77266085c605b108641a9d76ac4dbdc064c34) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: hashserv/server.py: drop unused imports

2021-05-14T06:57:57+00:00

remove unused vars. (Bitbake rev: 3287d28a506f67abd192799e61ef28e74ce7002d) Signed-off-by: Armin Kuster Signed-off-by: Richard Purdie

bitbake: fetch2/wget: when checking latest versions, consider all numerical directories

2021-05-14T06:57:57+00:00

Previously the regex was maching x.y, but wasn't matching x, which is a problem e.g. here: https://download.gnome.org/sources/epiphany/ (the new gnome version scheme adds 40-series at the end). (Bitbake rev: c03101576f447263ea38e8464210d3a3a2c27226) Signed-off-by: Alexander Kanavin Signed-off-by: Richard Purdie

bitbake: bitbake: Switch to post release version number 1.51.0

2021-05-06T10:08:08+00:00

(Bitbake rev: 97a64d12f70eb02f1d35b4ffefb291b80ca8c425) Signed-off-by: Richard Purdie

bitbake: bin/bitbake-getvar: Add a new command to query a variable value (with history)

2021-05-06T10:08:08+00:00

We've talked about having this for long enough. Add a command which queries a single variable value with history. This saves "bitbake -e | grep" and avoids the various pitfalls that has. It also provides a neat example of using tinfoil to make such a query. Parameters to limit the output to just the value, to limit to a variable flag and to not expand the output are provided. [YOCTO #10748] (Bitbake rev: 4c1881b620e885f55d7772f8626b8a76c2828333) Signed-off-by: Richard Purdie

bitbake: fetch/svn: Fix parsing revision of SVN repos with redirects

2021-05-06T10:08:08+00:00

svn was printing a message when encountering HTTP redirects. This confused the revision parser. (Bitbake rev: a944a335f8f4c4fe5df55f3d7d8e757bd2835146) Signed-off-by: Harald Brinkmann Signed-off-by: Richard Purdie

bitbake: prserv: Handle requests in main thread

2021-05-06T10:08:07+00:00

The prserver process is cleanly separated from the main bitbake process so requests can be handled in the main thread. This removes the need for a request queue and a separate request handling thread. (Bitbake rev: 6b09415bed6b5e7c12aaf39b677d9ef72844e233) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

bitbake: prserv: Extract daemonization from PRServer class

2021-05-06T10:08:07+00:00

The code to start the prservice process as a daemon is extracted out of the PRServer class and simplified. This makes the PRServer class easier to modernise as it no longer needs to worry about process management. (Bitbake rev: 39c7c158c52157b18f5ccbbd673e3298e6402f52) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie

bitbake: prserv: Use multiprocessing to auto start prserver

2021-05-06T10:08:07+00:00

We can use the modern multiprocessing support in Python instead of manually using fork to start the prserver process. To do this we need to set up the signal handlers for the prserver process in the work_forever function (which is now used as the main function for this process). The old code to start the prserver process using fork is not removed in this commit as it is tightly intertwined with the daemonization code which will be refactored in a following commit. (Bitbake rev: b3da56240c0f92efab1c0b293738c35c0f1ee6ab) Signed-off-by: Paul Barker Signed-off-by: Richard Purdie