linux/poky.git/bitbake/lib/hashserv, branch 5.2_M2

bitbake: hashserv: client: Fix changing stream modes

2024-05-31T15:56:25+00:00

When switching from normal mode to stream mode, skip calling self._set_mode() again because this will cause a recursion into the _set_mode() function and causes problems. Also cleanup some of the error checking during this process This bug affected when a client would attempt to switch from one stream mode to another, and meant that the server would get an invalid message from the client. This would cause the server to disconnect the client, and the client would then reconnect in normal mode which was the mode it wanted anyway and thus it would carry on without any errors. This made the bug not visible on the client side, but resulting in a lot of backtrace JSON decoding exceptions in the server logs. (Bitbake rev: 1826bc41ab3369ac40034c5eaf698748b769b881) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: asyncrpc: Remove ClientPool

2024-05-31T15:56:25+00:00

Batching support on the client side has proven to be a much more effective way of dealing with server latency than multiple client connections and is also much nicer on the server, so drop the client pool support from asyncrpc and the hash server (Bitbake rev: 6f80560f1c7010d09fe5448fdde616aef8468102) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: bitbake: Drop older python version compatibility code

2024-05-31T15:56:25+00:00

cooker: We can call multiprocessing close() unconditionally and tweak a comment give 3.8 is now the minimum version. lib/bb: We can drop the logger addition code only needed before 3.6 asyncrpc/hashserv: Since the minimum version is 3.8, we can drop the conditional code. (Bitbake rev: 16f4386400f88ba50605307961c248bef09895c1) Signed-off-by: Richard Purdie

bitbake: hashserv: server: Add support for SO_REUSEPORT

2024-05-30T06:38:10+00:00

SO_REUSEPORT is a socket option that allows multiple servers to listen on the same TCP port, and the kernel will automatically load balance the connections between them. This is particularly helpful for the hash server since it runs in a single thread. To take advantage of a multi-core server, multiple servers can be started in parallel with this option (up to 1 per CPU) and the kernel will load balance between them. (Bitbake rev: d72d5a7decb489e2af0ebc43cfea0ca3e4353e9b) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: hashserv: client: Add batch stream API

2024-05-30T06:38:10+00:00

Changes the stream mode to do "batch" processing. This means that the sending and reciving of messages is done simultaneously so that messages can be sent as fast as possible without having to wait for each reply. This allows multiple messages to be in flight at once, reducing the effect of the round trip latency from the server. (Bitbake rev: e768d0f17bdb97f6ff013ec3a41f182fecd47a55) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: hashserv: client: Fix mode state errors

2024-04-16T06:33:19+00:00

Careful reading of the code can contrive cases where poorly timed ConnectionError's will result in the client mode being incorrectly reset to MODE_NORMAL when it should actual be a stream mode for the current command. Fix this by no longer attempting to restore the mode when the connection is setup. Instead, attempt to set the stream mode inside the send wrapper for the stream data, which means that it should always end up in the correct mode before continuing. Also, factor out the transition to normal mode into a invoke() override so it doesn't need to be specified over and over again. (Bitbake rev: 0cd276fd98eeca463518d4a42675fffb18d6b3de) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: asyncrpc: include parse_address from hashserv

2024-04-14T05:31:45+00:00

Moving the code and related definitions from hashserv/__init__.py to asyncrpc/client.py, allowing this function to be used in other asyncrpc clients. (Bitbake rev: b67bb05e431414866b8e8c6a4c88d20b9cdb44a3) Signed-off-by: Michael Opdenacker Suggested-by: Joshua Watt Cc: Tim Orling Cc: Thomas Petazzoni Signed-off-by: Richard Purdie

bitbake: hashserv: Re-enable connection pooling with psycopg 3 driver

2024-02-23T14:34:05+00:00

Re-enable connection pooling in case `postgresql+psygopg` driver is used. Async connection pooling is supported in psycopg 3 [psycopg] driver in SQLAlchemy. Allow the connection pool to grow to arbitrary size. (Bitbake rev: 4fe05513b5314c201725e3f8ad54f58d70c56258) Signed-off-by: Tobias Hagelborn Signed-off-by: Richard Purdie

bitbake: bitbake: hashserv: Postgres adaptations for ignoring duplicate inserts

2024-02-19T11:58:12+00:00

Hash Equivalence server performs unconditional insert also of duplicate hash entries. This causes excessive error log entries in Postgres. Rather ignore the duplicate inserts. The alternate behavior should be isolated to the postgres engine type. (Bitbake rev: e8d2d178d0fe96f9d6031c97328e8be17d752716) Signed-off-by: Tobias Hagelborn Signed-off-by: Richard Purdie Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie

bitbake: hashserv: Add Client Pool

2024-02-19T11:58:12+00:00

Implements a Client Pool derived from the AsyncRPC client pool that allows querying for multiple equivalent hashes in parallel (Bitbake rev: ba4c764d8061c7b88cd4985ca493d6ea6e317106) Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie