tools/git-repo.git/project.py, branch v2.53

Sync: Fix full submodule sync while shallow specified

2025-03-13T16:12:45+00:00

Git allows to clone submodules as shallow clone [1]. On the other hand, when repo synchronize a projcet with submodules inside, it ignores the shallow parameter. When a project contains submodules, project.py parses the .gitmodules file for URL and path. This parsing does not consider the shallow option. Consequently, this parameter is not propgated to newly created Project instance for that submodule. [1] https://git-scm.com/docs/gitmodules#Documentation/gitmodules.txt-submoduleltnamegtshallow Change-Id: I54fc9c69ae1b8e3cda2801202e3f0c7693b718d2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/454261 Tested-by: Albert Akmukhametov Commit-Queue: Albert Akmukhametov Reviewed-by: Josip Sokcevic Reviewed-by: Никита Сказкоподателев (Nask)

Activate submodules

2025-02-04T16:07:49+00:00

This change moves further towards ensuring Git can understand repo's submodules. 'submodule init' is used to make the submodules active[1]. [1] https://git-scm.com/docs/gitsubmodules#_active_submodules Change-Id: I0c20ff1991101fc5be171e566d8fb644aab47200 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/446182 Tested-by: Kaushik Lingarkar Reviewed-by: Nasser Grainawi Reviewed-by: Josip Sokcevic Reviewed-by: Mike Frysinger

Use 'gitfile' in submodule checkouts

2025-02-04T16:07:49+00:00

This change takes another step towards ensuring Git can understand repo's submodules to some extent. Replace the old '.git' symlink with gitfile[1] pointing to the bare checkout of the submodule. This is required for Git's 'recurse submodules' opts to work with repo's submodules as '.git' is expected to be writable by Git when recursing over submodules. [1] https://git-scm.com/docs/gitrepository-layout#_description Change-Id: I52d15451768ee7bd6db289f4d2b3be5907370d42 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/446181 Tested-by: Kaushik Lingarkar Reviewed-by: Josip Sokcevic Reviewed-by: Mike Frysinger Reviewed-by: Nasser Grainawi

Update internal filesystem layout for submodules

2025-02-04T16:07:49+00:00

Change the bare checkout directory for submodules from 'subprojects' to 'modules'. Git expects bare submodule checkouts to be in the 'modules' directory. If old subproject directories are found, they will be migrated to the new modules directory. This change is the first step in ensuring Git can understand repo's submodules to some extent. Change-Id: I385029f1bb55d040616d970d6ffb4bb856692520 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/444881 Tested-by: Kaushik Lingarkar Reviewed-by: Josip Sokcevic

sync: Recover from errors during read-tree

2025-01-16T17:19:45+00:00

When repo is initializing a git repository, it calls `git read-tree`. During such operation, git is restoring workspace based on the current index. However, some things can go wrong: a user can run out of disk space, or, in case of partial clone, user may no longer reach the remote host. That will leave affected repository in a bad state with partially checked out workspace. The follow up repo sync won't try to fix such state. This change removes .git symlink, which will force the next `repo sync` to redo Git repository setup. Bug: b/363171216 Bug: b/390161127 Change-Id: I57db4b6cae0ef21826dc7cede4d3bf02cfc3d955 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/447801 Reviewed-by: Scott Lee Tested-by: Josip Sokcevic Commit-Queue: Josip Sokcevic

Remove gitc support from repo

2024-12-03T22:27:56+00:00

gitc is no longer available. Change-Id: I0cbfdf936832f2cdd4876104ae3cc5a6e26154e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/444841 Reviewed-by: Gavin Mak Commit-Queue: Josip Sokcevic Tested-by: Josip Sokcevic

worktree: Do not try to fix relative paths

2024-10-30T17:03:57+00:00

--worktree was broken with incorrect paths in the .git files whenever the local copy of git populated gitdir with relative paths instead of absoulte paths. Bug: 376251410 Change-Id: Id32dc1576315218967de2a9bfe43bf7a5a0e7aa6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440801 Commit-Queue: Allen Webb Reviewed-by: Josip Sokcevic Tested-by: Allen Webb

Fix incremental syncs for prjs with submodules

2024-10-18T03:55:10+00:00

When performing an incremental sync (re-running repo init with an updated manifest revision) with --fetch-submodules or sync-s=true, there is an attempt to get a list of all projects (including submodules) before projects are actually fetched. However, we can only list submodules of a project if we have already fetched its revision. Instead of throwing an error when we don't have the revision, assume there are no submodules for that project. In the sync cmd, we already update the list of projects to include submodules after fetching superprojects. Change-Id: I48bc68c48b5b10117356b18f5375d17f9a89ec05 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439761 Commit-Queue: Kaushik Lingarkar Tested-by: Kaushik Lingarkar Reviewed-by: Josip Sokcevic Reviewed-by: Nasser Grainawi

sync: Always use WORKER_BATCH_SIZE

2024-10-07T18:44:19+00:00

With 551285fa35ccd0836513e9cf64ee8d3372e5e3f4, the comment about number of workers no longer stands - dict is shared among multiprocesses and real time information is available. Using 2.7k projects as the baseline, using chunk size of 4 takes close to 5 minutes. A chunk size of 32 takes this down to 40s - a reduction of rougly 8 times which matches the increase. R=gavinmak@google.com Bug: b/371638995 Change-Id: Ida5fd8f7abc44b3b82c02aa0f7f7ae01dff5eb07 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438523 Commit-Queue: Josip Sokcevic Tested-by: Josip Sokcevic Reviewed-by: Gavin Mak

project: Handle git sso auth failures as repo exit

2024-10-03T20:47:50+00:00

If a user is not authenticated, repo continues execution and it will likely result in more of the same errors being printed. A user is also likely to SIGTERM the process resulting in more errors. This change stops repo sync if any of repositories can't be fetched to Git authentcation using sso helper. We could extend this to all Git authentication Change-Id: I9e471e063450c0a51f25a5e7f12a83064dfb170c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438522 Reviewed-by: Gavin Mak Tested-by: Josip Sokcevic Commit-Queue: Josip Sokcevic