diff options
Diffstat (limited to 'scripts/patchtest.README')
-rw-r--r-- | scripts/patchtest.README | 153 |
1 files changed, 153 insertions, 0 deletions
diff --git a/scripts/patchtest.README b/scripts/patchtest.README new file mode 100644 index 0000000000..76b5fcdb6d --- /dev/null +++ b/scripts/patchtest.README | |||
@@ -0,0 +1,153 @@ | |||
1 | # Patchtest | ||
2 | |||
3 | ## Introduction | ||
4 | |||
5 | Patchtest is a test framework for community patches based on the standard | ||
6 | unittest python module. As input, it needs tree elements to work properly: | ||
7 | a patch in mbox format (either created with `git format-patch` or fetched | ||
8 | from 'patchwork'), a test suite and a target repository. | ||
9 | |||
10 | The first test suite intended to be used with patchtest is found in the | ||
11 | openembedded-core repository [1] targeted for patches that get into the | ||
12 | openembedded-core mailing list [2]. This suite is also intended as a | ||
13 | baseline for development of similar suites for other layers as needed. | ||
14 | |||
15 | Patchtest can either run on a host or a guest machine, depending on which | ||
16 | environment the execution needs to be done. If you plan to test your own patches | ||
17 | (a good practice before these are sent to the mailing list), the easiest way is | ||
18 | to install and execute on your local host; in the other hand, if automatic | ||
19 | testing is intended, the guest method is strongly recommended. The guest | ||
20 | method requires the use of the patchtest layer, in addition to the tools | ||
21 | available in oe-core: https://git.yoctoproject.org/patchtest/ | ||
22 | |||
23 | ## Installation | ||
24 | |||
25 | As a tool for use with the Yocto Project, the [quick start guide](https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html) | ||
26 | contains the necessary prerequisites for a basic project. In addition, | ||
27 | patchtest relies on the following Python modules: | ||
28 | |||
29 | - boto3 (for sending automated results emails only) | ||
30 | - git-pw>=2.5.0 | ||
31 | - jinja2 | ||
32 | - pylint | ||
33 | - pyparsing>=3.0.9 | ||
34 | - unidiff | ||
35 | |||
36 | These can be installed by running `pip install -r | ||
37 | meta/lib/patchtest/requirements.txt`. Note that git-pw is not | ||
38 | automatically added to the user's PATH; by default, it is installed at | ||
39 | ~/.local/bin/git-pw. | ||
40 | |||
41 | For git-pw (and therefore scripts such as patchtest-get--series) to work, you need | ||
42 | to provide a Patchwork instance in your user's .gitconfig, like so (the project | ||
43 | can be specified using the --project argument): | ||
44 | |||
45 | git config --global pw.server "https://patchwork.yoctoproject.org/api/1.2/" | ||
46 | |||
47 | To work with patchtest, you should have the following repositories cloned: | ||
48 | |||
49 | 1. https://git.openembedded.org/openembedded-core/ (or https://git.yoctoproject.org/poky/) | ||
50 | 2. https://git.openembedded.org/bitbake/ (if not using poky) | ||
51 | 3. https://git.yoctoproject.org/patchtest (if using guest mode) | ||
52 | |||
53 | ## Usage | ||
54 | |||
55 | ### Obtaining Patches | ||
56 | |||
57 | Patch files can be obtained directly from cloned repositories using `git | ||
58 | format-patch -N` (where N is the number of patches starting from HEAD to | ||
59 | generate). git-pw can also be used with filters for users, patch/series IDs, | ||
60 | and timeboxes if specific patches are desired. For more information, see the | ||
61 | git-pw [documentation](https://patchwork.readthedocs.io/projects/git-pw/en/latest/). | ||
62 | |||
63 | Alternatively, `scripts/patchtest-get-series` can be used to pull mbox files from | ||
64 | the Patchwork instance configured previously in .gitconfig. It uses a log file | ||
65 | called ".series_test.log" to store and compare series IDs so that the same | ||
66 | versions of a patch are not tested multiple times unintentionally. By default, | ||
67 | it will pull up to five patch series from the last 30 minutes using oe-core as | ||
68 | the target project, but these parameters can be configured using the `--limit`, | ||
69 | `--interval`, and `--project` arguments respectively. For more information, run | ||
70 | `patchtest-get-series -h`. | ||
71 | |||
72 | ### Host Mode | ||
73 | |||
74 | To run patchtest on the host, do the following: | ||
75 | |||
76 | 1. In openembedded-core/poky, do `source oe-init-build-env` | ||
77 | 2. Generate patch files from the target repository by doing `git-format patch -N`, | ||
78 | where N is the number of patches starting at HEAD, or by using git-pw | ||
79 | or patchtest-get-series | ||
80 | 3. Run patchtest on a patch file by doing the following: | ||
81 | |||
82 | patchtest --patch /path/to/patch/file | ||
83 | |||
84 | or, if you have stored the patch files in a directory, do: | ||
85 | |||
86 | patchtest --directory /path/to/patch/directory | ||
87 | |||
88 | For example, to test `master-gcc-Fix--fstack-protector-issue-on-aarch64.patch` against the oe-core test suite: | ||
89 | |||
90 | patchtest --patch master-gcc-Fix--fstack-protector-issue-on-aarch64.patch | ||
91 | |||
92 | If you want to use a different test suite or target repository, you can use the --testdir and --repodir flags: | ||
93 | |||
94 | patchtest --patch /path/to/patch/file --repodir /path/to/repo --testdir /path/to/test/dir | ||
95 | |||
96 | ### Guest Mode | ||
97 | |||
98 | Patchtest's guest mode has been refactored to more closely mirror the | ||
99 | typical Yocto Project image build workflow, but there are still some key | ||
100 | differences to keep in mind. The primary objective is to provide a level | ||
101 | of isolation from the host when testing patches pulled automatically | ||
102 | from the mailing lists. When executed this way, the test process is | ||
103 | essentially running random code from the internet and could be | ||
104 | catastrophic if malicious bits or even poorly-handled edge cases aren't | ||
105 | protected against. In order to use this mode, the | ||
106 | https://git.yoctoproject.org/patchtest/ repository must be cloned and | ||
107 | the meta-patchtest layer added to bblayers.conf. | ||
108 | |||
109 | The general flow of guest mode is: | ||
110 | |||
111 | 1. Run patchtest-setup-sharedir --directory <dirname> to create a | ||
112 | directory for mounting | ||
113 | 2. Collect patches via patchtest-get-series (or other manual step) into the | ||
114 | <dirname>/mboxes path | ||
115 | 3. Ensure that a user with ID 1200 has appropriate read/write | ||
116 | permissions to <dirname> and <dirname>/mboxes, so that the | ||
117 | "patchtest" user in the core-image-patchtest image can function | ||
118 | 4. Build the core-image-patchtest image | ||
119 | 5. Run the core-image-patchtest image with the mounted sharedir, like | ||
120 | so: | ||
121 | `runqemu kvm nographic qemuparams="-snapshot -fsdev | ||
122 | local,id=test_mount,path=/workspace/yocto/poky/build/patchtestdir,security_model=mapped | ||
123 | -device virtio-9p-pci,fsdev=test_mount,mount_tag=test_mount -smp 4 -m | ||
124 | 2048"` | ||
125 | |||
126 | Patchtest runs as an initscript for the core-image-patchtest image and | ||
127 | shuts down after completion, so there is no input required from a user | ||
128 | during operation. Unlike in host mode, the guest is designed to | ||
129 | automatically generate test result files, in the same directory as the | ||
130 | targeted patch files but with .testresult as an extension. These contain | ||
131 | the entire output of the patchtest run for each respective pass, | ||
132 | including the PASS, FAIL, and SKIP indicators for each test run. | ||
133 | |||
134 | ## Contributing | ||
135 | |||
136 | The yocto mailing list (openembedded-core@lists.openembedded.org) is used for questions, | ||
137 | comments and patch review. It is subscriber only, so please register before | ||
138 | posting. | ||
139 | |||
140 | When sending single patches, please use something like: | ||
141 | |||
142 | git send-email -M -1 --to=openembedded-core@lists.openembedded.org --subject-prefix=OE-core][PATCH | ||
143 | |||
144 | ## Maintenance | ||
145 | ----------- | ||
146 | |||
147 | Maintainers: | ||
148 | Trevor Gamblin <tgamblin@baylibre.com> | ||
149 | |||
150 | ## Links | ||
151 | ----- | ||
152 | [1] https://git.openembedded.org/openembedded-core/ | ||
153 | [2] https://www.yoctoproject.org/community/mailing-lists/ | ||