diff options
Diffstat (limited to 'scripts/patchtest.README')
| -rw-r--r-- | scripts/patchtest.README | 159 |
1 files changed, 0 insertions, 159 deletions
diff --git a/scripts/patchtest.README b/scripts/patchtest.README deleted file mode 100644 index 3c1ee1af1d..0000000000 --- a/scripts/patchtest.README +++ /dev/null | |||
| @@ -1,159 +0,0 @@ | |||
| 1 | # Patchtest | ||
| 2 | |||
| 3 | ## Introduction | ||
| 4 | |||
| 5 | Patchtest is a test framework for community patches based on the standard | ||
| 6 | unittest python module. As input, it needs three elements to work properly: | ||
| 7 | |||
| 8 | - a patch in mbox format (either created with `git format-patch` or fetched | ||
| 9 | from 'patchwork') | ||
| 10 | - a test suite | ||
| 11 | - a target repository | ||
| 12 | |||
| 13 | The first test suite intended to be used with patchtest is found in the | ||
| 14 | openembedded-core repository [1], targeted for patches that get into the | ||
| 15 | openembedded-core mailing list [2]. This suite is also intended as a | ||
| 16 | baseline for development of similar suites for other layers as needed. | ||
| 17 | |||
| 18 | Patchtest can either run on a host or a guest machine, depending on | ||
| 19 | which environment you prefer. If you plan to test your own patches (a | ||
| 20 | good practice before these are sent to the mailing list), the easiest | ||
| 21 | way is to install and execute on your local host; in the other hand, if | ||
| 22 | automatic testing is intended, the guest method is strongly recommended. | ||
| 23 | The guest method requires the use of the patchtest layer, in addition to | ||
| 24 | the tools available in oe-core: https://git.yoctoproject.org/patchtest/ | ||
| 25 | |||
| 26 | ## Installation | ||
| 27 | |||
| 28 | As a tool for use with the Yocto Project, the [quick start | ||
| 29 | guide](https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html) | ||
| 30 | contains the necessary prerequisites. In addition, patchtest relies on | ||
| 31 | several Python modules for parsing and analysis, which can be installed | ||
| 32 | by running `pip install -r meta/lib/patchtest/requirements.txt`. Note | ||
| 33 | that git-pw is not automatically added to the user's PATH; by default, | ||
| 34 | it is installed at ~/.local/bin/git-pw. | ||
| 35 | |||
| 36 | For git-pw (and therefore scripts such as patchtest-get--series) to work, you need | ||
| 37 | to provide a Patchwork instance in your user's .gitconfig, like so (the project | ||
| 38 | can be specified using the --project argument): | ||
| 39 | |||
| 40 | git config --global pw.server "https://patchwork.yoctoproject.org/api/1.2/" | ||
| 41 | |||
| 42 | To work with patchtest, you should have the following repositories cloned: | ||
| 43 | |||
| 44 | 1. https://git.openembedded.org/openembedded-core/ (or https://git.yoctoproject.org/poky/) | ||
| 45 | 2. https://git.openembedded.org/bitbake/ (if not using poky) | ||
| 46 | 3. https://git.yoctoproject.org/patchtest (if using guest mode) | ||
| 47 | |||
| 48 | ## Usage | ||
| 49 | |||
| 50 | ### Obtaining Patches | ||
| 51 | |||
| 52 | Patch files can be obtained directly from cloned repositories using `git | ||
| 53 | format-patch -N` (where N is the number of patches starting from HEAD to | ||
| 54 | generate). git-pw can also be used with filters for users, patch/series IDs, | ||
| 55 | and timeboxes if specific patches are desired. For more information, see the | ||
| 56 | git-pw [documentation](https://patchwork.readthedocs.io/projects/git-pw/en/latest/). | ||
| 57 | |||
| 58 | Alternatively, `scripts/patchtest-get-series` can be used to pull mbox files from | ||
| 59 | the Patchwork instance configured previously in .gitconfig. It uses a log file | ||
| 60 | called ".series_test.log" to store and compare series IDs so that the same | ||
| 61 | versions of a patch are not tested multiple times unintentionally. By default, | ||
| 62 | it will pull up to five patch series from the last 30 minutes using oe-core as | ||
| 63 | the target project, but these parameters can be configured using the `--limit`, | ||
| 64 | `--interval`, and `--project` arguments respectively. For more information, run | ||
| 65 | `patchtest-get-series -h`. | ||
| 66 | |||
| 67 | ### Host Mode | ||
| 68 | |||
| 69 | To run patchtest on the host, do the following: | ||
| 70 | |||
| 71 | 1. In openembedded-core/poky, do `source oe-init-build-env` | ||
| 72 | 2. Generate patch files from the target repository by doing `git format-patch -N`, | ||
| 73 | where N is the number of patches starting at HEAD, or by using git-pw | ||
| 74 | or patchtest-get-series | ||
| 75 | 3. Run patchtest on a patch file by doing the following: | ||
| 76 | |||
| 77 | patchtest --patch /path/to/patch/file | ||
| 78 | |||
| 79 | or, if you have stored the patch files in a directory, do: | ||
| 80 | |||
| 81 | patchtest --directory /path/to/patch/directory | ||
| 82 | |||
| 83 | For example, to test `master-gcc-Fix--fstack-protector-issue-on-aarch64.patch` against the oe-core test suite: | ||
| 84 | |||
| 85 | patchtest --patch master-gcc-Fix--fstack-protector-issue-on-aarch64.patch | ||
| 86 | |||
| 87 | If you want to use a different test suite or target repository, you can use the --testdir and --repodir flags: | ||
| 88 | |||
| 89 | patchtest --patch /path/to/patch/file --repodir /path/to/repo --testdir /path/to/test/dir | ||
| 90 | |||
| 91 | ### Guest Mode | ||
| 92 | |||
| 93 | Patchtest's guest mode has been refactored to more closely mirror the | ||
| 94 | typical Yocto Project image build workflow, but there are still some key | ||
| 95 | differences to keep in mind. The primary objective is to provide a level | ||
| 96 | of isolation from the host when testing patches pulled automatically | ||
| 97 | from the mailing lists. When executed this way, the test process is | ||
| 98 | essentially running random code from the internet and could be | ||
| 99 | catastrophic if malicious bits or even poorly-handled edge cases aren't | ||
| 100 | protected against. In order to use this mode, the | ||
| 101 | https://git.yoctoproject.org/patchtest/ repository must be cloned and | ||
| 102 | the meta-patchtest layer added to bblayers.conf. | ||
| 103 | |||
| 104 | The general flow of guest mode is: | ||
| 105 | |||
| 106 | 1. Run patchtest-setup-sharedir --directory <dirname> to create a | ||
| 107 | directory for mounting | ||
| 108 | 2. Collect patches via patchtest-get-series (or other manual step) into the | ||
| 109 | <dirname>/mboxes path | ||
| 110 | 3. Ensure that a user with ID 1200 has appropriate read/write | ||
| 111 | permissions to <dirname> and <dirname>/mboxes, so that the | ||
| 112 | "patchtest" user in the core-image-patchtest image can function | ||
| 113 | 4. Build the core-image-patchtest image | ||
| 114 | 5. Run the core-image-patchtest image with the mounted sharedir, like | ||
| 115 | so: | ||
| 116 | `runqemu kvm nographic qemuparams="-snapshot -fsdev | ||
| 117 | local,id=test_mount,path=/workspace/yocto/poky/build/patchtestdir,security_model=mapped | ||
| 118 | -device virtio-9p-pci,fsdev=test_mount,mount_tag=test_mount -smp 4 -m | ||
| 119 | 2048"` | ||
| 120 | |||
| 121 | Patchtest is run by an initscript for the core-image-patchtest image and | ||
| 122 | shuts down after completion, so there is no input required from a user | ||
| 123 | during operation. Unlike in host mode, the guest is designed to | ||
| 124 | automatically generate test result files, in the same directory as the | ||
| 125 | targeted patch files but with .testresult as an extension. These contain | ||
| 126 | the entire output of the patchtest run for each respective pass, | ||
| 127 | including the PASS, FAIL, and SKIP indicators for each test run. | ||
| 128 | |||
| 129 | ### Running Patchtest Selftests | ||
| 130 | |||
| 131 | Patchtest also includes selftests, which are currently in the form of | ||
| 132 | several contrived patch files and a runner script found in | ||
| 133 | `meta/lib/patchtest/selftest/`. In order to run these, the | ||
| 134 | `meta-selftest` layer must be added to bblayers.conf. It is also | ||
| 135 | recommended to set BB_SERVER_TIMEOUT (and thus enable memory-resident | ||
| 136 | bitbake) in local.conf to reduce runtime, as the bitbake startup process | ||
| 137 | will otherwise add to it significantly when restarted for each test | ||
| 138 | patch. | ||
| 139 | |||
| 140 | ## Contributing | ||
| 141 | |||
| 142 | The yocto mailing list (openembedded-core@lists.openembedded.org) is used for questions, | ||
| 143 | comments and patch review. It is subscriber only, so please register before | ||
| 144 | posting. | ||
| 145 | |||
| 146 | When sending single patches, please use something like: | ||
| 147 | |||
| 148 | git send-email -M -1 --to=openembedded-core@lists.openembedded.org --subject-prefix=OE-core][PATCH | ||
| 149 | |||
| 150 | ## Maintenance | ||
| 151 | ----------- | ||
| 152 | |||
| 153 | Maintainers: | ||
| 154 | Trevor Gamblin <tgamblin@baylibre.com> | ||
| 155 | |||
| 156 | ## Links | ||
| 157 | ----- | ||
| 158 | [1] https://git.openembedded.org/openembedded-core/ | ||
| 159 | [2] https://www.yoctoproject.org/community/mailing-lists/ | ||
