tools/git-repo.git/git_superproject.py, branch v2.16

superproject: Don't exit if superproject tag doesn't exist in manifest.

2021-06-16T04:48:35+00:00

Don't exit if there are missing commit ids in superproject. This change implements the following suggestion from delphij@: "we should note the event (so we know that --use-superproject but there were some errors, e.g. manifest didn't specify commit id for some reason, or if there is no superproject but --use-superproject is used), print out a message telling the use that this is not support, but continue as if --no-use-superproject was specified?" Changes: superproject: + Added git_trace2_event_log as an argument to the constructor. + Sync method returns SyncResult a NamedTuple of ++ success - True if sync of superproject is successful, or False. ++ fatal - True if caller should exit, Or False. + UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of ++ manifest_path - path name of the overriding manifest file instead of None ++ fatal - True if caller should exit, Or False + _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of ++ commit_ids - a dictionary with the projects/commit ids on success, otherwise None ++ fatal - True if caller should exit, Or False + Added _SkipUpdatingProjectRevisionId a helper function to see if a project's revision id needs to be updated or not. This function is used to exclude projects from local manifest file. + Added the following error events into git_trace2_event_log ++ If superproject is missing in a manifest ++ If there are missing commit ids for projects. command.py: + Deleted unused import - platform + Added git_trace2_event_log as a member so all subcmds can log error events. main.py: + Initialized git_trace2_event_log as a member of command object. init.py: + Deleted unused import - optparse init.py: + Called sys.exit only if Sync returns exit=True sync.py: + Called sys.exit only if Superproject's UpdateProjectsRevisionId returns exit=True + Reloaded the manifest only if manifest path is returned by UpdateProjectsRevisionId. If not, fall back to the old way of doing repo sync. test_git_superproject: + Added code to verify error events are being logged. + Added a test for no superproject tag + Added test for UpdateProjectsRevisionId not updating the revision id with the commit ids. Tested the code with the following commands. + Positive test case with aosp-master. $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. Your identity is: Raman Tenneti If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .../android/aosp $ repo_dev sync -j40 --use-superproject remote: Total 12 (delta 4), reused 12 (delta 4) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... repo sync has finished successfully. + Negative test case without superproject tag. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml error: Cannot get project commit ids from manifest error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.589s repo sync has finished successfully. + Test for missing commit_id for a project. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint'] error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.364s repo sync has finished successfully. $ ./run_tests -v ... ...== 164 passed in 2.87s ==... Bug: [google internal] b/189371541 Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: add projects from local manifest to local:: group.

2021-06-10T00:16:36+00:00

With repo sync --use-superproject, don't update the commit ids of every project that comes from local manifest. Tested the code with the following commands. $ ./run_tests -v + Test with local.xml 1. repo init --use-superproject -u persistent-https://googleplex-android.git.corp.google.com/a/platform/manifest 2. cd .repo cp -r /google/src/head/depot/google3/wireless/android/build_tools/translations/pipeline/local_manifests local_manifests cd .. local$ time repo_dev sync --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../local/.repo/exp-superproject/feb2c2847da5e274f3d530d5ab438af8-superproject.git: Initial setup for superproject completed. ... Bug: [google internal] b/189360443 Bug: [google internal] b/189139268 Bug: https://crbug.com/gerrit/14499 Change-Id: Ideaf268c294e9b500b2b9726ffbd733dd8d63004 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308822 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger Reviewed-by: Jonathan Nieder

superproject: revert not updating commit ids if remote is different.

2021-06-08T22:43:32+00:00

superproject supports multiple remotes. Get all commit ids from superproject for all projects that are in the manifest. $ ./run_tests -v Bug: [google internal] b/186395810 Change-Id: I6edce3918853a7a3a65aec5528e6a43a544eff53 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308862 Tested-by: Raman Tenneti Reviewed-by: Jonathan Nieder Reviewed-by: Mike Frysinger

superproject: change the warning message to say it is beta.

2021-06-03T14:38:09+00:00

$ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest remote: Total 3 (delta 0), reused 3 (delta 0) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` ... $ ./run_tests -v Bug: [google internal] b/189946009 Change-Id: Ifb3ef266a72b67f3c4a2a3ac2033b10e03b789d4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308522 Reviewed-by: Jonathan Nieder Tested-by: Raman Tenneti

superproject: require git version 2.28.0 or higher.

2021-06-03T00:30:10+00:00

Tested the code with the following commands. $ repo init --use-superproject -u https://android.googlesource.com/platform/manifest + Tested for the wrong git version. $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest WARNING: --use-superproject is experimental and not for general use superproject requires a git version 2.28 or later error: git update of superproject failed $ ./run_tests -v Bug: https://crbug.com/gerrit/14617 Bug: [google internal] b/189846687 Change-Id: I5cd4158ea29b3b3c8c81234f4e818165de346e63 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308442 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: Use bugurl from contactinfo in the missing commits error message.

2021-05-18T15:35:54+00:00

+ In XmlManifest._Unload set 'bugurl' to Wrapper().BUG_URL. + contactinfo returns a namedtuple. + bug_url can be accessed as self._manifest.contactinfo.bugurl. Tested the code with the following commands. $ ./run_tests -v Added contactinfo tag to default.xml and verified that bugurl is used. Bug: [google internal] b/186220520. Change-Id: Iaafd6465e072b2e47a0a0b548bf6cb608a0b0a04 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/306342 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: skip updating commit ids if remote's fetchUrl don't match.

2021-05-04T15:32:23+00:00

Tested the code with the following commands. $ ./run_tests -v + Test with local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --use-superproject --partial-clone --clone-filter=blob:limit=10M && mkdir -p .repo/local_manifests && (gcertstatus -quiet=true || gcert) && ln -s /google/src/head/depot/google3/wireless/android/build_tools/aosp/manifests/mirror-aosp-master-with-vendor/local.xml .repo/local_manifests/local.xml $ repo_dev sync -c -j8 + Test without local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j8 Bug: [google internal] b/186395810 Change-Id: Id618113a91c12bcb90a30a3c23d3d6842bcb49e1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304942 Reviewed-by: Mike Frysinger Tested-by: Raman Tenneti

superproject: Don't update the commit ids of projects if remote is different.

2021-05-03T05:13:23+00:00

1) Skip setting the revision id (commit id) for the projects whose remote doesn't match superproject's remote. 2) exp-superproject/superproject_override.xml includes local_manfiest's projects. When we load this XML, don't reload projects from local.xml (otherwise we will get duplicate projects errors). Tested the code with the following commands. $ ./run_tests -v + Test with local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --use-superproject --partial-clone --clone-filter=blob:limit=10M && mkdir -p .repo/local_manifests && (gcertstatus -quiet=true || gcert) && ln -s /google/src/head/depot/google3/wireless/android/build_tools/aosp/manifests/mirror-aosp-master-with-vendor/local.xml .repo/local_manifests/local.xml $ repo_dev sync -c -j8 + Test without local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j8 Bug: [google internal] b/186395810 Change-Id: I4e9d4ac2d94a9fc0cef0ccd787b6310758009e86 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304882 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

help/version: sprinkle bug report URL around

2021-04-26T21:43:43+00:00

Make it a bit easier for people to locate bug reporting info. Change-Id: If9c8939c84ebd52eb96b353c1797afa25868bb85 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303943 Tested-by: Mike Frysinger Reviewed-by: Raman Tenneti

superproject: Added --depth=1 argument to git fetch command.

2021-03-19T21:10:39+00:00

Tested the code with the following commands. $ ./run_tests -v Without --depth $ time repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest -b master --repo-rev=main --use-superproject real 6m48.086s user 3m27.281s sys 1m1.386s With --depth=1 $ time repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest -b master --repo-rev=main --use-superproject real 2m49.637s user 2m51.458s sys 0m39.108s From dwillemsen@: "For me it's the difference between 9m28s using the current code and 16s using --depth=1 while fetching the superproject." Bug: [google internal] b/180451672 Bug: https://crbug.com/gerrit/13707 Change-Id: I1c3e4aef4414c4e9dd259fb6e4619da0421896b0 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/300922 Reviewed-by: Mike Frysinger Tested-by: Raman Tenneti