tools/git-repo.git/git_config.py, branch v2.18

Fix typo for ValueError

2021-11-07T02:32:08+00:00

which will cause error log like below: NameError: name 'ValueErrorl' is not defined Change-Id: I388886b7cf6d700e224c3847b7ba4ba4fe9c041d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323015 Reviewed-by: Mike Frysinger Tested-by: 彭杨益

git_config: update error handling with no config file

2021-09-29T01:02:47+00:00

Now that _do throws an exception when `git` fails, update the logic that tries to read config files but the file doesn't exist. Bug: b/192664812 Change-Id: I6417ecd70891b8f2d5f2bdb819f91df69ac4b70c Test: `repo upload` no longer crashes when .repo/config doesn't exist Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319295 Reviewed-by: Jack Neus Tested-by: Mike Frysinger

make file removal a bit more robust

2021-09-28T16:06:50+00:00

Some of the file removal calls are subject to race conditions (if something else deletes the file), so extend our remove API to have an option to ignore ENOENT errors. Then update a bunch of random call sites to use this new functionality. Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195 Reviewed-by: Sean McAllister Tested-by: Mike Frysinger

repo: Add support for standalone manifests

2021-09-28T15:40:46+00:00

Added --standalone_manifest to repo tool. If set, the manifest is downloaded directly from the appropriate source (currently, we only support GS) and used instead of creating a manifest git checkout. The manifests.git repo is still created to keep track of various config but is marked as being for a standalone manifest so that the repo tool doesn't try to run networked git commands in it. BUG=b:192664812 TEST=existing tests (no coverage), manual runs Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942 Tested-by: Jack Neus Reviewed-by: Mike Frysinger

sync: Remove '_' from the repo.syncstate.* keys when saved to config.

2021-07-29T22:41:57+00:00

GitConfig doesn't save keys if the keys contain "_" characters. Some of the options like mp_update, use_superproject have underscores. This fixes issue with previous_sync_state missing some of the options. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the sync command multiple times and verifing previous_sync_state and current_sync_state have the same keys. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I16b52a164f9dd1633d7dad1d8cf6b151c629fcb1 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313242 Reviewed-by: Xin Li Tested-by: Raman Tenneti

sync: Added logging of repo sync state and config options for analysis.

2021-07-29T19:20:57+00:00

git_config.py: + Added SyncAnalysisState class, which saves the following data into the config object. ++ sys.argv, options, superproject's logging data. ++ repo.*, branch.* and remote.* parameters from config object. ++ current time as synctime. ++ Version number of the object. + All the keys for the above data are prepended with 'repo.syncstate.' + Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods to GitConfig object to save/get the above data. git_trace2_event_log.py: + Added LogConfigEvents method with code from DefParamRepoEvents to log events. sync.py: + superproject_logging_data is a dictionary that collects all the superproject data that is to be logged as trace2 event. + Sync at the end logs the previously saved syncstate.* parameters as previous_sync_state. Then it calls config's UpdateSyncAnalysisState to save and log all the current options, superproject logged data. docs/internal-fs-layout.md: + Added doc string explaining [repo.syncstate ...] sections of .repo/manifests.git/config file. test_git_config.py: + Added unit test for the new methods of GitConfig object. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the following command multiple times. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123 Tested-by: Raman Tenneti Reviewed-by: Mike Frysinger Reviewed-by: Xin Li

Add the ability to administratively enroll repo into using superproject.

2021-07-01T16:27:43+00:00

Repo will remember a choice and an expiration time of the choice, per user, about whether to use superproject by default. When not specified from command line and the choice is not expired, repo would use the user default value. When a user default value is not present and when the system wide enable default is provided in git's system configuration, repo would ask the user for a confirmation which will be valid for two weeks. git_config.py: Add support for system config. When reading system config, we would use --system to avoid hardcoding a path as the value may be different on some other distributions. git_superproject.py: Add a new subroutine, _UseSuperproject(), which returns whether superproject should be used and whether it is from a user configuration. The value is determined in the following order: 1. If the user specifies either --use-superproject or --no-use-superproject, then that choice is being used. 2. If neither is specified, we would then check the saved value (upon repo init) and use that choice when there was a choice. 3. We then check if there is a saved and unexpired value for user's choice in their ~/.gitconfig, and use the unexpired choice, if available. 4. Finally, if all the above didn't give us a decision, and if the git system configuration is providing a rollout hint, present a prompt to user for their decision and save it in ~/.gitconfig. subcmds/sync.py: Make use of the new UseSuperproject() provided by git_superproject.py. While there also silent stderr from git describe when determining the version of repo. Bug: [google internal] b/190688390 Change-Id: Iad3ee03026342ee500e5d65e2f0fa600d7637613 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309762 Reviewed-by: Mike Frysinger Tested-by: Xin Li

ssh: rewrite proxy management for multiprocessing usage

2021-05-10T21:16:06+00:00

We changed sync to use multiprocessing for parallel work. This broke the ssh proxy code as it's all based on threads. Rewrite the logic to be multiprocessing safe. Now instead of the module acting as a stateful object, callers have to instantiate a new ProxyManager class that holds all the state, an pass that down to any users. Bug: https://crbug.com/gerrit/12389 Change-Id: I4b1af116f7306b91e825d3c56fb4274c9b033562 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305486 Tested-by: Mike Frysinger Reviewed-by: Chris Mcdonald

ssh: move proxy usage to the sync subcommand

2021-05-10T21:10:29+00:00

The only time we really need ssh proxies is when we want to run many connections and reuse them. That only happens when running sync. Every other command makes at most two connections, and even then it's only one or none. So the effort of setting up & tearing down ssh proxies isn't worth it most of the time. The big reason we want to move this logic to sync is that it's now using multiprocessing for parallel work. The current ssh proxy code is all based on threads, which means none of the logic is working correctly. The current ssh design makes it hard to fix when all of the state lives in the global/module scope. So the first step to fixing this is top move the setup & teardown to the one place that really needs it: sync. No other commands will use proxies anymore, just direct connections. Bug: https://crbug.com/gerrit/12389 Change-Id: Ibd351acdec39a87562b3013637c5df4ea34e03c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305485 Reviewed-by: Chris Mcdonald Tested-by: Mike Frysinger

ssh: move all ssh logic to a common place

2021-05-06T19:09:16+00:00

We had ssh logic sprinkled between two git modules, and neither was quite the right home for it. This largely moves the logic as-is to its new home. We'll leave major refactoring to followup commits. Bug: https://crbug.com/gerrit/12389 Change-Id: I300a8f7dba74f2bd132232a5eb1e856a8490e0e9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305483 Reviewed-by: Chris Mcdonald Tested-by: Mike Frysinger