tools/git-repo.git/tests/test_git_superproject.py, branch v2.19

superproject: support a new revision attribute.

2021-09-27T06:36:05+00:00

Tested: $ ./run_tests Verified that a manifest that specified superproject revision would use the specified revision, and superproject will use the default revision. Note that this is a slight behavior change from earlier repo versions, which would always use the branch name of the manifest itself. However, the new behavior would be more consisitent with regular "project" element and would allow superproject be used even if it is not enabled for the particular manifest branch, so we have decided to make the change as it would provide more flexibility and better matches what other elements would do. Bug: [google internal] b/187868160 Change-Id: I35255ee347aff6e65179f7879d52931f168b477e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643 Tested-by: Xin Li Reviewed-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: print messages if the manifest has superproject tag.

2021-08-13T20:07:40+00:00

1) If the manifest has superproject tag (git_master, etc), then display error/warning messages (as it is doing today) 2) If the manifest doesn't have superproject tag (nest, chromeos manifests), then don't display any error/warning messages about superrproject (behave as though user has specified --no-use-superproject). 3) Print error/warning messages if --use-superproject passed as argument to repo sync. 4) No change in behavior for the repo init command. git_superproject.py: + Fixed typo in _WriteManifestFile method name + Superproject accepts print_message as an argument and it defaults to True. All messages that are printed to stderr are controlled by this flag. If it is True, then messages get printed. + Added PrintMessages function which return true if either --use-superproject is specified on the command line or if the manifest has a superproject tag. sync.py: + Displays the warning message if PrintMessgages are enabled and passes that as argument to superproject object. + Added 'hassuperprojecttag' trace2 log entry for analysis. We can find users/branches that are using superproject, but the manifest is missing the superproject tag. Tested: $ ./run_tests + Verified printing of messages with and without superproject tag, with with --use-superproject option. + aosp-master $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest $ repo_dev sync + A manifest without superproject tag. $ repo_dev init -m $(pwd)/manifest_7482982.xml $ repo_dev sync -n -c -j32 -m $(pwd)/manifest_7482982.xml Bug: [google internal] b/196411099 Change-Id: I92166dcad15a4129fab82edcf869e7c8db3efd4b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/314982 Reviewed-by: Xin Li Tested-by: Raman Tenneti

Fix an issue when syncing with --use-superproject and clone bundles.

2021-06-30T15:31:15+00:00

It is possible that a clone bundle contained the object referenced by the branch in the manifest and in the superproject, but not the branch itself (for example, the branch may be newly created from an existing branch, or is not vislble to the user downloading the clone bundle). When --use-superproject is enabled, because we are overriding revisionExpr with the SHA1 revision provided by the superproject, the verification step would succeed, but because the expected branch do not exist, it would confuse git-repo at a later time, as it is expecting the remote branch to exist in the local clone. In project.py, fix this by making SetRevisionId() to always remember the actual branch name and verify it in _CheckForImmutableRevision() so that we only skip the fetch step when both objects exists locally. Bug: [google internal] b/191974277 Change-Id: I49d3ca0667f524c8c45f416492faf95b1dd822fb Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310802 Reviewed-by: Raman Tenneti Reviewed-by: Mike Frysinger Tested-by: Xin Li

superproject - don't update the commit ids of projects that have revisionId.

2021-06-29T16:46:07+00:00

Pinned manifests and release manifest have revisionId set for all projects. For such projects don't update the commit ids. Tested the code with the following commands. $ ./run_tests -v $ repo_dev sync --use-superproject -j8 $ repo_dev sync -n -c -j32 -m $(pwd)/manifest_7482982.xml Bug: [google internal] b/191995372 Change-Id: I4681135b1d15f4a63527b6f0356d76ec842485d6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310582 Reviewed-by: Xin Li Reviewed-by: Mike Frysinger Tested-by: Raman Tenneti

superproject: Don't exit if superproject tag doesn't exist in manifest.

2021-06-16T04:48:35+00:00

Don't exit if there are missing commit ids in superproject. This change implements the following suggestion from delphij@: "we should note the event (so we know that --use-superproject but there were some errors, e.g. manifest didn't specify commit id for some reason, or if there is no superproject but --use-superproject is used), print out a message telling the use that this is not support, but continue as if --no-use-superproject was specified?" Changes: superproject: + Added git_trace2_event_log as an argument to the constructor. + Sync method returns SyncResult a NamedTuple of ++ success - True if sync of superproject is successful, or False. ++ fatal - True if caller should exit, Or False. + UpdateProjectsRevisionId returns UpdateProjectsResult a NamedTuple of ++ manifest_path - path name of the overriding manifest file instead of None ++ fatal - True if caller should exit, Or False + _GetAllProjectsCommitIds returns CommitIdsResult a NamedTuple of ++ commit_ids - a dictionary with the projects/commit ids on success, otherwise None ++ fatal - True if caller should exit, Or False + Added _SkipUpdatingProjectRevisionId a helper function to see if a project's revision id needs to be updated or not. This function is used to exclude projects from local manifest file. + Added the following error events into git_trace2_event_log ++ If superproject is missing in a manifest ++ If there are missing commit ids for projects. command.py: + Deleted unused import - platform + Added git_trace2_event_log as a member so all subcmds can log error events. main.py: + Initialized git_trace2_event_log as a member of command object. init.py: + Deleted unused import - optparse init.py: + Called sys.exit only if Sync returns exit=True sync.py: + Called sys.exit only if Superproject's UpdateProjectsRevisionId returns exit=True + Reloaded the manifest only if manifest path is returned by UpdateProjectsRevisionId. If not, fall back to the old way of doing repo sync. test_git_superproject: + Added code to verify error events are being logged. + Added a test for no superproject tag + Added test for UpdateProjectsRevisionId not updating the revision id with the commit ids. Tested the code with the following commands. + Positive test case with aosp-master. $ repo_dev init -u persistent-https://android.git.corp.google.com/platform/manifest -b master --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. Your identity is: Raman Tenneti If you want to change this, please re-run 'repo init' with --config-name repo has been initialized in .../android/aosp $ repo_dev sync -j40 --use-superproject remote: Total 12 (delta 4), reused 12 (delta 4) NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. ... repo sync has finished successfully. + Negative test case without superproject tag. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` repo error: superproject tag is not defined in manifest: .../android/aosp/.repo/manifest.xml error: Cannot get project commit ids from manifest error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.589s repo sync has finished successfully. + Test for missing commit_id for a project. $ repo_dev sync -j40 --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed. error: please file a bug using go/repo-bug to report missing commit_ids for: ['build/blueprint'] error: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option ... Checking out: 100% (1022/1022), done in 3.364s repo sync has finished successfully. $ ./run_tests -v ... ...== 164 passed in 2.87s ==... Bug: [google internal] b/189371541 Change-Id: I5ea49f87e8fa41be590fc0c914573e16c8cdfcfa Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309162 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: add projects from local manifest to local:: group.

2021-06-10T00:16:36+00:00

With repo sync --use-superproject, don't update the commit ids of every project that comes from local manifest. Tested the code with the following commands. $ ./run_tests -v + Test with local.xml 1. repo init --use-superproject -u persistent-https://googleplex-android.git.corp.google.com/a/platform/manifest 2. cd .repo cp -r /google/src/head/depot/google3/wireless/android/build_tools/translations/pipeline/local_manifests local_manifests cd .. local$ time repo_dev sync --use-superproject NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version` .../local/.repo/exp-superproject/feb2c2847da5e274f3d530d5ab438af8-superproject.git: Initial setup for superproject completed. ... Bug: [google internal] b/189360443 Bug: [google internal] b/189139268 Bug: https://crbug.com/gerrit/14499 Change-Id: Ideaf268c294e9b500b2b9726ffbd733dd8d63004 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308822 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger Reviewed-by: Jonathan Nieder

superproject: revert not updating commit ids if remote is different.

2021-06-08T22:43:32+00:00

superproject supports multiple remotes. Get all commit ids from superproject for all projects that are in the manifest. $ ./run_tests -v Bug: [google internal] b/186395810 Change-Id: I6edce3918853a7a3a65aec5528e6a43a544eff53 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/308862 Tested-by: Raman Tenneti Reviewed-by: Jonathan Nieder Reviewed-by: Mike Frysinger

tests: Make the tests pass for Python < 3.8

2021-05-26T13:36:20+00:00

Before Python 3.8, xml.dom.minidom sorted the attributes of an element when writing it to a file, while later versions output the attributes in the order they were created. Avoid these differences by sorting the attributes for each element before comparing the generated manifests with the expected ones. Bug: https://crbug.com/gerrit/14382 Change-Id: Ie2597727afcc48f9063a7261ad970e8a549f0587 Signed-off-by: Peter Kjellerstedt Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303326 Reviewed-by: Mike Frysinger

superproject: Don't update the commit ids of projects if remote is different.

2021-05-03T05:13:23+00:00

1) Skip setting the revision id (commit id) for the projects whose remote doesn't match superproject's remote. 2) exp-superproject/superproject_override.xml includes local_manfiest's projects. When we load this XML, don't reload projects from local.xml (otherwise we will get duplicate projects errors). Tested the code with the following commands. $ ./run_tests -v + Test with local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --use-superproject --partial-clone --clone-filter=blob:limit=10M && mkdir -p .repo/local_manifests && (gcertstatus -quiet=true || gcert) && ln -s /google/src/head/depot/google3/wireless/android/build_tools/aosp/manifests/mirror-aosp-master-with-vendor/local.xml .repo/local_manifests/local.xml $ repo_dev sync -c -j8 + Test without local.xml $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j8 Bug: [google internal] b/186395810 Change-Id: I4e9d4ac2d94a9fc0cef0ccd787b6310758009e86 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304882 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger

superproject: pass groups to ToXml method.

2021-03-11T01:24:52+00:00

Added the following methods to XmlManifest class. + GetDefaultGroupsStr() - return 'default,platform-' + platform.system().lower() + GetGroupsStr() - Same as gitc_utils.py's _manifest_groups func. + Replaced gitc_utils.py's_manifest_groups calls with GetGroupsStr. + Used the above methods to get groups in command.py::GetProjects and part of init.py. TODO: clean up these funcs to take structured group data more instead of passing strings around everywhere that need parsing. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL and verified prebuilts/fullsdk-linux directory has all the folders. Tested repo init and repo sync with --use-superproject and without --use-superproject argument. $ repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject $ repo_dev sync -c -j32 Bug: [google internal] b/181804931 Bug: https://crbug.com/gerrit/13707 Change-Id: Ia98585cbfa3a1449710655af55d56241794242b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/299422 Reviewed-by: Jonathan Nieder Reviewed-by: Mike Frysinger Tested-by: Raman Tenneti