diff options
Diffstat (limited to 'documentation/dev-manual/upgrading-recipes.rst')
-rw-r--r-- | documentation/dev-manual/upgrading-recipes.rst | 397 |
1 files changed, 397 insertions, 0 deletions
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst new file mode 100644 index 0000000000..4fac78bdfb --- /dev/null +++ b/documentation/dev-manual/upgrading-recipes.rst | |||
@@ -0,0 +1,397 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | Upgrading Recipes | ||
4 | ***************** | ||
5 | |||
6 | Over time, upstream developers publish new versions for software built | ||
7 | by layer recipes. It is recommended to keep recipes up-to-date with | ||
8 | upstream version releases. | ||
9 | |||
10 | While there are several methods to upgrade a recipe, you might | ||
11 | consider checking on the upgrade status of a recipe first. You can do so | ||
12 | using the ``devtool check-upgrade-status`` command. See the | ||
13 | ":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" | ||
14 | section in the Yocto Project Reference Manual for more information. | ||
15 | |||
16 | The remainder of this section describes three ways you can upgrade a | ||
17 | recipe. You can use the Automated Upgrade Helper (AUH) to set up | ||
18 | automatic version upgrades. Alternatively, you can use | ||
19 | ``devtool upgrade`` to set up semi-automatic version upgrades. Finally, | ||
20 | you can manually upgrade a recipe by editing the recipe itself. | ||
21 | |||
22 | Using the Auto Upgrade Helper (AUH) | ||
23 | =================================== | ||
24 | |||
25 | The AUH utility works in conjunction with the OpenEmbedded build system | ||
26 | in order to automatically generate upgrades for recipes based on new | ||
27 | versions being published upstream. Use AUH when you want to create a | ||
28 | service that performs the upgrades automatically and optionally sends | ||
29 | you an email with the results. | ||
30 | |||
31 | AUH allows you to update several recipes with a single use. You can also | ||
32 | optionally perform build and integration tests using images with the | ||
33 | results saved to your hard drive and emails of results optionally sent | ||
34 | to recipe maintainers. Finally, AUH creates Git commits with appropriate | ||
35 | commit messages in the layer's tree for the changes made to recipes. | ||
36 | |||
37 | .. note:: | ||
38 | |||
39 | In some conditions, you should not use AUH to upgrade recipes | ||
40 | and should instead use either ``devtool upgrade`` or upgrade your | ||
41 | recipes manually: | ||
42 | |||
43 | - When AUH cannot complete the upgrade sequence. This situation | ||
44 | usually results because custom patches carried by the recipe | ||
45 | cannot be automatically rebased to the new version. In this case, | ||
46 | ``devtool upgrade`` allows you to manually resolve conflicts. | ||
47 | |||
48 | - When for any reason you want fuller control over the upgrade | ||
49 | process. For example, when you want special arrangements for | ||
50 | testing. | ||
51 | |||
52 | The following steps describe how to set up the AUH utility: | ||
53 | |||
54 | #. *Be Sure the Development Host is Set Up:* You need to be sure that | ||
55 | your development host is set up to use the Yocto Project. For | ||
56 | information on how to set up your host, see the | ||
57 | ":ref:`dev-manual/start:Preparing the Build Host`" section. | ||
58 | |||
59 | #. *Make Sure Git is Configured:* The AUH utility requires Git to be | ||
60 | configured because AUH uses Git to save upgrades. Thus, you must have | ||
61 | Git user and email configured. The following command shows your | ||
62 | configurations:: | ||
63 | |||
64 | $ git config --list | ||
65 | |||
66 | If you do not have the user and | ||
67 | email configured, you can use the following commands to do so:: | ||
68 | |||
69 | $ git config --global user.name some_name | ||
70 | $ git config --global user.email username@domain.com | ||
71 | |||
72 | #. *Clone the AUH Repository:* To use AUH, you must clone the repository | ||
73 | onto your development host. The following command uses Git to create | ||
74 | a local copy of the repository on your system:: | ||
75 | |||
76 | $ git clone git://git.yoctoproject.org/auto-upgrade-helper | ||
77 | Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. | ||
78 | remote: Compressing objects: 100% (300/300), done. | ||
79 | remote: Total 768 (delta 499), reused 703 (delta 434) | ||
80 | Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. | ||
81 | Resolving deltas: 100% (499/499), done. | ||
82 | Checking connectivity... done. | ||
83 | |||
84 | AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or | ||
85 | :term:`Poky` repositories. | ||
86 | |||
87 | #. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script` | ||
88 | script to create a fresh :term:`Build Directory` that you use exclusively | ||
89 | for running the AUH utility:: | ||
90 | |||
91 | $ cd poky | ||
92 | $ source oe-init-build-env your_AUH_build_directory | ||
93 | |||
94 | Re-using an existing :term:`Build Directory` and its configurations is not | ||
95 | recommended as existing settings could cause AUH to fail or behave | ||
96 | undesirably. | ||
97 | |||
98 | #. *Make Configurations in Your Local Configuration File:* Several | ||
99 | settings are needed in the ``local.conf`` file in the build | ||
100 | directory you just created for AUH. Make these following | ||
101 | configurations: | ||
102 | |||
103 | - If you want to enable :ref:`Build | ||
104 | History <dev-manual/build-quality:maintaining build output quality>`, | ||
105 | which is optional, you need the following lines in the | ||
106 | ``conf/local.conf`` file:: | ||
107 | |||
108 | INHERIT =+ "buildhistory" | ||
109 | BUILDHISTORY_COMMIT = "1" | ||
110 | |||
111 | With this configuration and a successful | ||
112 | upgrade, a build history "diff" file appears in the | ||
113 | ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in | ||
114 | your :term:`Build Directory`. | ||
115 | |||
116 | - If you want to enable testing through the :ref:`ref-classes-testimage` | ||
117 | class, which is optional, you need to have the following set in | ||
118 | your ``conf/local.conf`` file:: | ||
119 | |||
120 | IMAGE_CLASSES += "testimage" | ||
121 | |||
122 | .. note:: | ||
123 | |||
124 | If your distro does not enable by default ptest, which Poky | ||
125 | does, you need the following in your ``local.conf`` file:: | ||
126 | |||
127 | DISTRO_FEATURES:append = " ptest" | ||
128 | |||
129 | |||
130 | #. *Optionally Start a vncserver:* If you are running in a server | ||
131 | without an X11 session, you need to start a vncserver:: | ||
132 | |||
133 | $ vncserver :1 | ||
134 | $ export DISPLAY=:1 | ||
135 | |||
136 | #. *Create and Edit an AUH Configuration File:* You need to have the | ||
137 | ``upgrade-helper/upgrade-helper.conf`` configuration file in your | ||
138 | :term:`Build Directory`. You can find a sample configuration file in the | ||
139 | :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`. | ||
140 | |||
141 | Read through the sample file and make configurations as needed. For | ||
142 | example, if you enabled build history in your ``local.conf`` as | ||
143 | described earlier, you must enable it in ``upgrade-helper.conf``. | ||
144 | |||
145 | Also, if you are using the default ``maintainers.inc`` file supplied | ||
146 | with Poky and located in ``meta-yocto`` and you do not set a | ||
147 | "maintainers_whitelist" or "global_maintainer_override" in the | ||
148 | ``upgrade-helper.conf`` configuration, and you specify "-e all" on | ||
149 | the AUH command-line, the utility automatically sends out emails to | ||
150 | all the default maintainers. Please avoid this. | ||
151 | |||
152 | This next set of examples describes how to use the AUH: | ||
153 | |||
154 | - *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the | ||
155 | following form:: | ||
156 | |||
157 | $ upgrade-helper.py recipe_name | ||
158 | |||
159 | For example, this command upgrades the ``xmodmap`` recipe:: | ||
160 | |||
161 | $ upgrade-helper.py xmodmap | ||
162 | |||
163 | - *Upgrading a Specific Recipe to a Particular Version:* To upgrade a | ||
164 | specific recipe to a particular version, use the following form:: | ||
165 | |||
166 | $ upgrade-helper.py recipe_name -t version | ||
167 | |||
168 | For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3:: | ||
169 | |||
170 | $ upgrade-helper.py xmodmap -t 1.2.3 | ||
171 | |||
172 | - *Upgrading all Recipes to the Latest Versions and Suppressing Email | ||
173 | Notifications:* To upgrade all recipes to their most recent versions | ||
174 | and suppress the email notifications, use the following command:: | ||
175 | |||
176 | $ upgrade-helper.py all | ||
177 | |||
178 | - *Upgrading all Recipes to the Latest Versions and Send Email | ||
179 | Notifications:* To upgrade all recipes to their most recent versions | ||
180 | and send email messages to maintainers for each attempted recipe as | ||
181 | well as a status email, use the following command:: | ||
182 | |||
183 | $ upgrade-helper.py -e all | ||
184 | |||
185 | Once you have run the AUH utility, you can find the results in the AUH | ||
186 | :term:`Build Directory`:: | ||
187 | |||
188 | ${BUILDDIR}/upgrade-helper/timestamp | ||
189 | |||
190 | The AUH utility | ||
191 | also creates recipe update commits from successful upgrade attempts in | ||
192 | the layer tree. | ||
193 | |||
194 | You can easily set up to run the AUH utility on a regular basis by using | ||
195 | a cron job. See the | ||
196 | :yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>` | ||
197 | file distributed with the utility for an example. | ||
198 | |||
199 | Using ``devtool upgrade`` | ||
200 | ========================= | ||
201 | |||
202 | As mentioned earlier, an alternative method for upgrading recipes to | ||
203 | newer versions is to use | ||
204 | :doc:`devtool upgrade </ref-manual/devtool-reference>`. | ||
205 | You can read about ``devtool upgrade`` in general in the | ||
206 | ":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" | ||
207 | section in the Yocto Project Application Development and the Extensible | ||
208 | Software Development Kit (eSDK) Manual. | ||
209 | |||
210 | To see all the command-line options available with ``devtool upgrade``, | ||
211 | use the following help command:: | ||
212 | |||
213 | $ devtool upgrade -h | ||
214 | |||
215 | If you want to find out what version a recipe is currently at upstream | ||
216 | without any attempt to upgrade your local version of the recipe, you can | ||
217 | use the following command:: | ||
218 | |||
219 | $ devtool latest-version recipe_name | ||
220 | |||
221 | As mentioned in the previous section describing AUH, ``devtool upgrade`` | ||
222 | works in a less-automated manner than AUH. Specifically, | ||
223 | ``devtool upgrade`` only works on a single recipe that you name on the | ||
224 | command line, cannot perform build and integration testing using images, | ||
225 | and does not automatically generate commits for changes in the source | ||
226 | tree. Despite all these "limitations", ``devtool upgrade`` updates the | ||
227 | recipe file to the new upstream version and attempts to rebase custom | ||
228 | patches contained by the recipe as needed. | ||
229 | |||
230 | .. note:: | ||
231 | |||
232 | AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat | ||
233 | of a "wrapper" application for ``devtool upgrade``. | ||
234 | |||
235 | A typical scenario involves having used Git to clone an upstream | ||
236 | repository that you use during build operations. Because you have built the | ||
237 | recipe in the past, the layer is likely added to your | ||
238 | configuration already. If for some reason, the layer is not added, you | ||
239 | could add it easily using the | ||
240 | ":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" | ||
241 | script. For example, suppose you use the ``nano.bb`` recipe from the | ||
242 | ``meta-oe`` layer in the ``meta-openembedded`` repository. For this | ||
243 | example, assume that the layer has been cloned into following area:: | ||
244 | |||
245 | /home/scottrif/meta-openembedded | ||
246 | |||
247 | The following command from your :term:`Build Directory` adds the layer to | ||
248 | your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``):: | ||
249 | |||
250 | $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe | ||
251 | NOTE: Starting bitbake server... | ||
252 | Parsing recipes: 100% |##########################################| Time: 0:00:55 | ||
253 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
254 | Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 | ||
255 | Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 | ||
256 | Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 | ||
257 | Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 | ||
258 | |||
259 | For this example, assume that the ``nano.bb`` recipe that | ||
260 | is upstream has a 2.9.3 version number. However, the version in the | ||
261 | local repository is 2.7.4. The following command from your build | ||
262 | directory automatically upgrades the recipe for you:: | ||
263 | |||
264 | $ devtool upgrade nano -V 2.9.3 | ||
265 | NOTE: Starting bitbake server... | ||
266 | NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace | ||
267 | Parsing recipes: 100% |##########################################| Time: 0:00:46 | ||
268 | Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. | ||
269 | NOTE: Extracting current version source... | ||
270 | NOTE: Resolving any missing task queue dependencies | ||
271 | . | ||
272 | . | ||
273 | . | ||
274 | NOTE: Executing SetScene Tasks | ||
275 | NOTE: Executing RunQueue Tasks | ||
276 | NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. | ||
277 | Adding changed files: 100% |#####################################| Time: 0:00:00 | ||
278 | NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano | ||
279 | NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb | ||
280 | |||
281 | .. note:: | ||
282 | |||
283 | Using the ``-V`` option is not necessary. Omitting the version number causes | ||
284 | ``devtool upgrade`` to upgrade the recipe to the most recent version. | ||
285 | |||
286 | Continuing with this example, you can use ``devtool build`` to build the | ||
287 | newly upgraded recipe:: | ||
288 | |||
289 | $ devtool build nano | ||
290 | NOTE: Starting bitbake server... | ||
291 | Loading cache: 100% |################################################################################################| Time: 0:00:01 | ||
292 | Loaded 2040 entries from dependency cache. | ||
293 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 | ||
294 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
295 | NOTE: Resolving any missing task queue dependencies | ||
296 | . | ||
297 | . | ||
298 | . | ||
299 | NOTE: Executing SetScene Tasks | ||
300 | NOTE: Executing RunQueue Tasks | ||
301 | NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano | ||
302 | NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. | ||
303 | |||
304 | Within the ``devtool upgrade`` workflow, you can | ||
305 | deploy and test your rebuilt software. For this example, | ||
306 | however, running ``devtool finish`` cleans up the workspace once the | ||
307 | source in your workspace is clean. This usually means using Git to stage | ||
308 | and submit commits for the changes generated by the upgrade process. | ||
309 | |||
310 | Once the tree is clean, you can clean things up in this example with the | ||
311 | following command from the ``${BUILDDIR}/workspace/sources/nano`` | ||
312 | directory:: | ||
313 | |||
314 | $ devtool finish nano meta-oe | ||
315 | NOTE: Starting bitbake server... | ||
316 | Loading cache: 100% |################################################################################################| Time: 0:00:00 | ||
317 | Loaded 2040 entries from dependency cache. | ||
318 | Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 | ||
319 | Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. | ||
320 | NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch | ||
321 | NOTE: Updating recipe nano_2.9.3.bb | ||
322 | NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb | ||
323 | NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano | ||
324 | NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually | ||
325 | |||
326 | |||
327 | Using the ``devtool finish`` command cleans up the workspace and creates a patch | ||
328 | file based on your commits. The tool puts all patch files back into the | ||
329 | source directory in a sub-directory named ``nano`` in this case. | ||
330 | |||
331 | Manually Upgrading a Recipe | ||
332 | =========================== | ||
333 | |||
334 | If for some reason you choose not to upgrade recipes using | ||
335 | :ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or | ||
336 | by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, | ||
337 | you can manually edit the recipe files to upgrade the versions. | ||
338 | |||
339 | .. note:: | ||
340 | |||
341 | Manually updating multiple recipes scales poorly and involves many | ||
342 | steps. The recommendation to upgrade recipe versions is through AUH | ||
343 | or ``devtool upgrade``, both of which automate some steps and provide | ||
344 | guidance for others needed for the manual process. | ||
345 | |||
346 | To manually upgrade recipe versions, follow these general steps: | ||
347 | |||
348 | #. *Change the Version:* Rename the recipe such that the version (i.e. | ||
349 | the :term:`PV` part of the recipe name) | ||
350 | changes appropriately. If the version is not part of the recipe name, | ||
351 | change the value as it is set for :term:`PV` within the recipe itself. | ||
352 | |||
353 | #. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds | ||
354 | is fetched from Git or some other version control system, update | ||
355 | :term:`SRCREV` to point to the | ||
356 | commit hash that matches the new version. | ||
357 | |||
358 | #. *Build the Software:* Try to build the recipe using BitBake. Typical | ||
359 | build failures include the following: | ||
360 | |||
361 | - License statements were updated for the new version. For this | ||
362 | case, you need to review any changes to the license and update the | ||
363 | values of :term:`LICENSE` and | ||
364 | :term:`LIC_FILES_CHKSUM` | ||
365 | as needed. | ||
366 | |||
367 | .. note:: | ||
368 | |||
369 | License changes are often inconsequential. For example, the | ||
370 | license text's copyright year might have changed. | ||
371 | |||
372 | - Custom patches carried by the older version of the recipe might | ||
373 | fail to apply to the new version. For these cases, you need to | ||
374 | review the failures. Patches might not be necessary for the new | ||
375 | version of the software if the upgraded version has fixed those | ||
376 | issues. If a patch is necessary and failing, you need to rebase it | ||
377 | into the new version. | ||
378 | |||
379 | #. *Optionally Attempt to Build for Several Architectures:* Once you | ||
380 | successfully build the new software for a given architecture, you | ||
381 | could test the build for other architectures by changing the | ||
382 | :term:`MACHINE` variable and | ||
383 | rebuilding the software. This optional step is especially important | ||
384 | if the recipe is to be released publicly. | ||
385 | |||
386 | #. *Check the Upstream Change Log or Release Notes:* Checking both these | ||
387 | reveals if there are new features that could break | ||
388 | backwards-compatibility. If so, you need to take steps to mitigate or | ||
389 | eliminate that situation. | ||
390 | |||
391 | #. *Optionally Create a Bootable Image and Test:* If you want, you can | ||
392 | test the new software by booting it onto actual hardware. | ||
393 | |||
394 | #. *Create a Commit with the Change in the Layer Repository:* After all | ||
395 | builds work and any testing is successful, you can create commits for | ||
396 | any changes in the layer holding your upgraded recipe. | ||
397 | |||