tools/git-repo.git, branch v2.49.2

upload: Return correct tuple values in _ProcessResults

2024-10-31T21:18:53+00:00

Incorrect tuple values were returned with http://go/grev/440221 - instead of returning (Project, ReviewableBranch), _ProcessResults was returning (int, ReviewableBranch). R=jojwang@google.com Bug: 376731172 Change-Id: I75205f42fd23f5ee6bd8d0c15b18066189b42bd9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/441121 Reviewed-by: Sam Saccone Commit-Queue: Josip Sokcevic Tested-by: Josip Sokcevic

worktree: Do not try to fix relative paths

2024-10-30T17:03:57+00:00

--worktree was broken with incorrect paths in the .git files whenever the local copy of git populated gitdir with relative paths instead of absoulte paths. Bug: 376251410 Change-Id: Id32dc1576315218967de2a9bfe43bf7a5a0e7aa6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440801 Commit-Queue: Allen Webb Reviewed-by: Josip Sokcevic Tested-by: Allen Webb

forall: Fix returning results early

2024-10-30T16:11:04+00:00

rc should be returned only after all results are processed. R=jojwang@google.com Bug: b/376454189 Change-Id: I8200b9954240dd3e8e9f2ab82494779a3cb38627 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440901 Tested-by: Josip Sokcevic Commit-Queue: Josip Sokcevic Reviewed-by: Joanna Wang

Use full name of the revision when checking dest-branch

2024-10-28T23:47:08+00:00

The manifest usually doesn't sepecify the revision with the full name (e.g. refs/heads/REV). However, when checking if the name of the merge branch, full name is used on the merge branch. The CL use full name of revision when comparing it with the merge branch. Bug: b/370919047 Test: repo upload on a project with `dest-branch` set Change-Id: Ib6fa2f7246beb5bae0a26a70048a7ac03b6c5a2f Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438401 Reviewed-by: Josip Sokcevic Tested-by: Joe Hsu Commit-Queue: Josip Sokcevic

Add REPO_SKIP_SELF_UPDATE check in sync

2024-10-28T17:46:25+00:00

The command _PostRepoFetch will try to self update during repo sync. That is beneficial but adds version uncertainty, fail potential and slow downs in non-interactive scenarios. Conditionally skip the update if env variable REPO_SKIP_SELF_UPDATE is defined. A call to selfupdate works as before, meaning even with the variable set, it will run the update. Change-Id: Iab0ef55dc3d3db3cbf1ba1f506c57fbb58a504c3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439967 Tested-by: Fredrik de Groot Commit-Queue: Josip Sokcevic Reviewed-by: Josip Sokcevic

manifest: add optional base check on remove and extend

2024-10-28T16:55:10+00:00

This adds an optional, built-in checker for guarding against patches hanging on wrong base revisions, which is useful if a lower layer of the manifest changes after a patch was done. When adding a patch with a new revision using extend-project or remove-project/project: C---D---E patches in project bla / A---B project bla in manifest state 1 If project bla gets updated, in a new snap ID or by a supplier or similar, to a new state: C---D---E patches in project bla / A---B---F---G project bla in manifest state 2 Parsing will fail because revision of bla is now G, giving the choice to create a new patch branch from G and updating base-rev, or keeping previous branch for some reason and only updating base-rev. Intended for use in a layered manifest with hashed revisions. Named refs like branches and tags also work fine when comparing, but will be misleading if a branch is used as base-rev. Change-Id: Ic6211550a7d3cc9656057f6a2087c505b40cad2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436777 Reviewed-by: Josip Sokcevic Tested-by: Fredrik de Groot Commit-Queue: Josip Sokcevic

[event_log] Stop leaking semaphore resources

2024-10-24T16:58:17+00:00

With the global state and fork, we are left with uncleaned resources. Isolate mulitprocessing.Value in a function so we stop the leak. Bug: 353656374 Change-Id: If50bb544bda12b72f00c02bc1d2c0d19de000b88 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440261 Commit-Queue: Josip Sokcevic Reviewed-by: Gavin Mak Tested-by: Josip Sokcevic

progress: always show done message

2024-10-24T16:21:28+00:00

The done message was omitted if the task is shorter than 0.5s. This might confuse users. Bug: b/371638995 Change-Id: I3fdd2cd8daea16d34fba88457d09397fff71af15 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440222 Tested-by: Kuang-che Wu Commit-Queue: Kuang-che Wu Reviewed-by: Josip Sokcevic

subcmds: reduce multiprocessing serialization overhead

2024-10-23T23:34:34+00:00

Follow the same approach as 39ffd9977e to reduce serialization overhead. Below benchmarks are tested with 2.7k projects on my workstation (warm cache). git tracing is disabled for benchmark. (seconds) | v2.48 | v2.48 | this CL | this CL | | -j32 | | -j32 ----------------------------------------------------------- with clean tree state: branches (none) | 5.6 | 5.9 | 1.0 | 0.9 status (clean) | 21.3 | 9.4 | 19.4 | 4.7 diff (none) | 7.6 | 7.2 | 5.7 | 2.2 prune (none) | 5.7 | 6.1 | 1.3 | 1.2 abandon (none) | 19.4 | 18.6 | 0.9 | 0.8 upload (none) | 19.7 | 18.7 | 0.9 | 0.8 forall -c true | 7.5 | 7.6 | 0.6 | 0.6 forall -c "git log -1" | 11.3 | 11.1 | 0.6 | 0.6 with branches: start BRANCH --all | 21.9 | 20.3 | 13.6 | 2.6 checkout BRANCH | 29.1 | 27.8 | 1.1 | 1.0 branches (2) | 28.0 | 28.6 | 1.5 | 1.3 abandon BRANCH | 29.2 | 27.5 | 9.7 | 2.2 Bug: b/371638995 Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221 Reviewed-by: Josip Sokcevic Tested-by: Kuang-che Wu Commit-Queue: Kuang-che Wu

sync: reduce multiprocessing serialization overhead

2024-10-23T02:58:45+00:00

Background: - Manifest object is large (for projects like Android) in terms of serialization cost and size (more than 1mb). - Lots of Project objects usually share only a few manifest objects. Before this CL, Project objects were passed to workers via function parameters. Function parameters are pickled separately (in chunk). In other words, manifests are serialized again and again. The major serialization overhead of repo sync was O(manifest_size * projects / chunksize) This CL uses following tricks to reduce serialization overhead. - All projects are pickled in one invocation. Because Project objects share manifests, pickle library remembers which objects are already seen and avoid the serialization cost. - Pass the Project objects to workers at worker intialization time. And pass project index as function parameters instead. The number of workers is much smaller than the number of projects. - Worker init state are shared on Linux (fork based). So it requires zero serialization for Project objects. On Linux (fork based), the serialization overhead is O(projects) --- one int per project On Windows (spawn based), the serialization overhead is O(manifest_size * min(workers, projects)) Moreover, use chunksize=1 to avoid the chance that some workers are idle while other workers still have more than one job in their chunk queue. Using 2.7k projects as the baseline, originally "repo sync" no-op sync takes 31s for fetch and 25s for checkout on my Linux workstation. With this CL, it takes 12s for fetch and 1s for checkout. Bug: b/371638995 Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921 Tested-by: Kuang-che Wu Reviewed-by: Josip Sokcevic Commit-Queue: Kuang-che Wu