diff options
Diffstat (limited to 'documentation/overview-manual/overview-manual-yp-intro.rst')
-rw-r--r-- | documentation/overview-manual/overview-manual-yp-intro.rst | 941 |
1 files changed, 941 insertions, 0 deletions
diff --git a/documentation/overview-manual/overview-manual-yp-intro.rst b/documentation/overview-manual/overview-manual-yp-intro.rst new file mode 100644 index 0000000000..5cdab7ca4a --- /dev/null +++ b/documentation/overview-manual/overview-manual-yp-intro.rst | |||
@@ -0,0 +1,941 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-2.0-UK | ||
2 | |||
3 | ***************************** | ||
4 | Introducing the Yocto Project | ||
5 | ***************************** | ||
6 | |||
7 | What is the Yocto Project? | ||
8 | ========================== | ||
9 | |||
10 | The Yocto Project is an open source collaboration project that helps | ||
11 | developers create custom Linux-based systems that are designed for | ||
12 | embedded products regardless of the product's hardware architecture. | ||
13 | Yocto Project provides a flexible toolset and a development environment | ||
14 | that allows embedded device developers across the world to collaborate | ||
15 | through shared technologies, software stacks, configurations, and best | ||
16 | practices used to create these tailored Linux images. | ||
17 | |||
18 | Thousands of developers worldwide have discovered that Yocto Project | ||
19 | provides advantages in both systems and applications development, | ||
20 | archival and management benefits, and customizations used for speed, | ||
21 | footprint, and memory utilization. The project is a standard when it | ||
22 | comes to delivering embedded software stacks. The project allows | ||
23 | software customizations and build interchange for multiple hardware | ||
24 | platforms as well as software stacks that can be maintained and scaled. | ||
25 | |||
26 | .. image:: figures/key-dev-elements.png | ||
27 | :align: center | ||
28 | |||
29 | For further introductory information on the Yocto Project, you might be | ||
30 | interested in this | ||
31 | `article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__ | ||
32 | by Drew Moseley and in this short introductory | ||
33 | `video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__. | ||
34 | |||
35 | The remainder of this section overviews advantages and challenges tied | ||
36 | to the Yocto Project. | ||
37 | |||
38 | .. _gs-features: | ||
39 | |||
40 | Features | ||
41 | -------- | ||
42 | |||
43 | The following list describes features and advantages of the Yocto | ||
44 | Project: | ||
45 | |||
46 | - *Widely Adopted Across the Industry:* Semiconductor, operating | ||
47 | system, software, and service vendors exist whose products and | ||
48 | services adopt and support the Yocto Project. For a look at the Yocto | ||
49 | Project community and the companies involved with the Yocto Project, | ||
50 | see the "COMMUNITY" and "ECOSYSTEM" tabs on the | ||
51 | :yocto_home:`Yocto Project <>` home page. | ||
52 | |||
53 | - *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS, | ||
54 | AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors | ||
55 | create and supply BSPs that support their hardware. If you have | ||
56 | custom silicon, you can create a BSP that supports that architecture. | ||
57 | |||
58 | Aside from lots of architecture support, the Yocto Project fully | ||
59 | supports a wide range of device emulation through the Quick EMUlator | ||
60 | (QEMU). | ||
61 | |||
62 | - *Images and Code Transfer Easily:* Yocto Project output can easily | ||
63 | move between architectures without moving to new development | ||
64 | environments. Additionally, if you have used the Yocto Project to | ||
65 | create an image or application and you find yourself not able to | ||
66 | support it, commercial Linux vendors such as Wind River, Mentor | ||
67 | Graphics, Timesys, and ENEA could take it and provide ongoing | ||
68 | support. These vendors have offerings that are built using the Yocto | ||
69 | Project. | ||
70 | |||
71 | - *Flexibility:* Corporations use the Yocto Project many different | ||
72 | ways. One example is to create an internal Linux distribution as a | ||
73 | code base the corporation can use across multiple product groups. | ||
74 | Through customization and layering, a project group can leverage the | ||
75 | base Linux distribution to create a distribution that works for their | ||
76 | product needs. | ||
77 | |||
78 | - *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux | ||
79 | distribution, you can use the Yocto Project to create exactly what | ||
80 | you need for embedded devices. You only add the feature support or | ||
81 | packages that you absolutely need for the device. For devices that | ||
82 | have display hardware, you can use available system components such | ||
83 | as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich | ||
84 | user experience. For devices that do not have a display or where you | ||
85 | want to use alternative UI frameworks, you can choose to not install | ||
86 | these components. | ||
87 | |||
88 | - *Comprehensive Toolchain Capabilities:* Toolchains for supported | ||
89 | architectures satisfy most use cases. However, if your hardware | ||
90 | supports features that are not part of a standard toolchain, you can | ||
91 | easily customize that toolchain through specification of | ||
92 | platform-specific tuning parameters. And, should you need to use a | ||
93 | third-party toolchain, mechanisms built into the Yocto Project allow | ||
94 | for that. | ||
95 | |||
96 | - *Mechanism Rules Over Policy:* Focusing on mechanism rather than | ||
97 | policy ensures that you are free to set policies based on the needs | ||
98 | of your design instead of adopting decisions enforced by some system | ||
99 | software provider. | ||
100 | |||
101 | - *Uses a Layer Model:* The Yocto Project `layer | ||
102 | infrastructure <#the-yocto-project-layer-model>`__ groups related | ||
103 | functionality into separate bundles. You can incrementally add these | ||
104 | grouped functionalities to your project as needed. Using layers to | ||
105 | isolate and group functionality reduces project complexity and | ||
106 | redundancy, allows you to easily extend the system, make | ||
107 | customizations, and keep functionality organized. | ||
108 | |||
109 | - *Supports Partial Builds:* You can build and rebuild individual | ||
110 | packages as needed. Yocto Project accomplishes this through its | ||
111 | `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being | ||
112 | able to build and debug components individually eases project | ||
113 | development. | ||
114 | |||
115 | - *Releases According to a Strict Schedule:* Major releases occur on a | ||
116 | :doc:`six-month cycle <../ref-manual/ref-release-process>` | ||
117 | predictably in October and April. The most recent two releases | ||
118 | support point releases to address common vulnerabilities and | ||
119 | exposures. This predictability is crucial for projects based on the | ||
120 | Yocto Project and allows development teams to plan activities. | ||
121 | |||
122 | - *Rich Ecosystem of Individuals and Organizations:* For open source | ||
123 | projects, the value of community is very important. Support forums, | ||
124 | expertise, and active developers who continue to push the Yocto | ||
125 | Project forward are readily available. | ||
126 | |||
127 | - *Binary Reproducibility:* The Yocto Project allows you to be very | ||
128 | specific about dependencies and achieves very high percentages of | ||
129 | binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When | ||
130 | distributions are not specific about which packages are pulled in and | ||
131 | in what order to support dependencies, other build systems can | ||
132 | arbitrarily include packages. | ||
133 | |||
134 | - *License Manifest:* The Yocto Project provides a :ref:`license | ||
135 | manifest <dev-manual/dev-manual-common-tasks:maintaining open source license compliance during your product's lifecycle>` | ||
136 | for review by people who need to track the use of open source | ||
137 | licenses (e.g. legal teams). | ||
138 | |||
139 | .. _gs-challenges: | ||
140 | |||
141 | Challenges | ||
142 | ---------- | ||
143 | |||
144 | The following list presents challenges you might encounter when | ||
145 | developing using the Yocto Project: | ||
146 | |||
147 | - *Steep Learning Curve:* The Yocto Project has a steep learning curve | ||
148 | and has many different ways to accomplish similar tasks. It can be | ||
149 | difficult to choose how to proceed when varying methods exist by | ||
150 | which to accomplish a given task. | ||
151 | |||
152 | - *Understanding What Changes You Need to Make For Your Design Requires | ||
153 | Some Research:* Beyond the simple tutorial stage, understanding what | ||
154 | changes need to be made for your particular design can require a | ||
155 | significant amount of research and investigation. For information | ||
156 | that helps you transition from trying out the Yocto Project to using | ||
157 | it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and | ||
158 | ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`" | ||
159 | documents on the Yocto Project website. | ||
160 | |||
161 | - *Project Workflow Could Be Confusing:* The `Yocto Project | ||
162 | workflow <#overview-development-environment>`__ could be confusing if | ||
163 | you are used to traditional desktop and server software development. | ||
164 | In a desktop development environment, mechanisms exist to easily pull | ||
165 | and install new packages, which are typically pre-compiled binaries | ||
166 | from servers accessible over the Internet. Using the Yocto Project, | ||
167 | you must modify your configuration and rebuild to add additional | ||
168 | packages. | ||
169 | |||
170 | - *Working in a Cross-Build Environment Can Feel Unfamiliar:* When | ||
171 | developing code to run on a target, compilation, execution, and | ||
172 | testing done on the actual target can be faster than running a | ||
173 | BitBake build on a development host and then deploying binaries to | ||
174 | the target for test. While the Yocto Project does support development | ||
175 | tools on the target, the additional step of integrating your changes | ||
176 | back into the Yocto Project build environment would be required. | ||
177 | Yocto Project supports an intermediate approach that involves making | ||
178 | changes on the development system within the BitBake environment and | ||
179 | then deploying only the updated packages to the target. | ||
180 | |||
181 | The Yocto Project :term:`OpenEmbedded Build System` | ||
182 | produces packages | ||
183 | in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy | ||
184 | these packages into the running system on the target by using | ||
185 | utilities on the target such as ``rpm`` or ``ipk``. | ||
186 | |||
187 | - *Initial Build Times Can be Significant:* Long initial build times | ||
188 | are unfortunately unavoidable due to the large number of packages | ||
189 | initially built from scratch for a fully functioning Linux system. | ||
190 | Once that initial build is completed, however, the shared-state | ||
191 | (sstate) cache mechanism Yocto Project uses keeps the system from | ||
192 | rebuilding packages that have not been "touched" since the last | ||
193 | build. The sstate mechanism significantly reduces times for | ||
194 | successive builds. | ||
195 | |||
196 | The Yocto Project Layer Model | ||
197 | ============================= | ||
198 | |||
199 | The Yocto Project's "Layer Model" is a development model for embedded | ||
200 | and IoT Linux creation that distinguishes the Yocto Project from other | ||
201 | simple build systems. The Layer Model simultaneously supports | ||
202 | collaboration and customization. Layers are repositories that contain | ||
203 | related sets of instructions that tell the :term:`OpenEmbedded Build System` | ||
204 | what to do. You can | ||
205 | collaborate, share, and reuse layers. | ||
206 | |||
207 | Layers can contain changes to previous instructions or settings at any | ||
208 | time. This powerful override capability is what allows you to customize | ||
209 | previously supplied collaborative or community layers to suit your | ||
210 | product requirements. | ||
211 | |||
212 | You use different layers to logically separate information in your | ||
213 | build. As an example, you could have BSP, GUI, distro configuration, | ||
214 | middleware, or application layers. Putting your entire build into one | ||
215 | layer limits and complicates future customization and reuse. Isolating | ||
216 | information into layers, on the other hand, helps simplify future | ||
217 | customizations and reuse. You might find it tempting to keep everything | ||
218 | in one layer when working on a single project. However, the more modular | ||
219 | your Metadata, the easier it is to cope with future changes. | ||
220 | |||
221 | .. note:: | ||
222 | |||
223 | - Use Board Support Package (BSP) layers from silicon vendors when | ||
224 | possible. | ||
225 | |||
226 | - Familiarize yourself with the `Yocto Project curated layer | ||
227 | index <https://www.yoctoproject.org/software-overview/layers/>`__ | ||
228 | or the `OpenEmbedded layer | ||
229 | index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__. | ||
230 | The latter contains more layers but they are less universally | ||
231 | validated. | ||
232 | |||
233 | - Layers support the inclusion of technologies, hardware components, | ||
234 | and software components. The :ref:`Yocto Project | ||
235 | Compatible <dev-manual/dev-manual-common-tasks:making sure your layer is compatible with yocto project>` | ||
236 | designation provides a minimum level of standardization that | ||
237 | contributes to a strong ecosystem. "YP Compatible" is applied to | ||
238 | appropriate products and software components such as BSPs, other | ||
239 | OE-compatible layers, and related open-source projects, allowing | ||
240 | the producer to use Yocto Project badges and branding assets. | ||
241 | |||
242 | To illustrate how layers are used to keep things modular, consider | ||
243 | machine customizations. These types of customizations typically reside | ||
244 | in a special layer, rather than a general layer, called a BSP Layer. | ||
245 | Furthermore, the machine customizations should be isolated from recipes | ||
246 | and Metadata that support a new GUI environment, for example. This | ||
247 | situation gives you a couple of layers: one for the machine | ||
248 | configurations, and one for the GUI environment. It is important to | ||
249 | understand, however, that the BSP layer can still make machine-specific | ||
250 | additions to recipes within the GUI environment layer without polluting | ||
251 | the GUI layer itself with those machine-specific changes. You can | ||
252 | accomplish this through a recipe that is a BitBake append | ||
253 | (``.bbappend``) file, which is described later in this section. | ||
254 | |||
255 | .. note:: | ||
256 | |||
257 | For general information on BSP layer structure, see the | ||
258 | :doc:`../bsp-guide/bsp-guide` | ||
259 | . | ||
260 | |||
261 | The :term:`Source Directory` | ||
262 | contains both general layers and BSP layers right out of the box. You | ||
263 | can easily identify layers that ship with a Yocto Project release in the | ||
264 | Source Directory by their names. Layers typically have names that begin | ||
265 | with the string ``meta-``. | ||
266 | |||
267 | .. note:: | ||
268 | |||
269 | It is not a requirement that a layer name begin with the prefix | ||
270 | meta- | ||
271 | , but it is a commonly accepted standard in the Yocto Project | ||
272 | community. | ||
273 | |||
274 | For example, if you were to examine the `tree | ||
275 | view <https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/>`__ of the | ||
276 | ``poky`` repository, you will see several layers: ``meta``, | ||
277 | ``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and | ||
278 | ``meta-yocto-bsp``. Each of these repositories represents a distinct | ||
279 | layer. | ||
280 | |||
281 | For procedures on how to create layers, see the | ||
282 | ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`" | ||
283 | section in the Yocto Project Development Tasks Manual. | ||
284 | |||
285 | Components and Tools | ||
286 | ==================== | ||
287 | |||
288 | The Yocto Project employs a collection of components and tools used by | ||
289 | the project itself, by project developers, and by those using the Yocto | ||
290 | Project. These components and tools are open source projects and | ||
291 | metadata that are separate from the reference distribution | ||
292 | (:term:`Poky`) and the | ||
293 | :term:`OpenEmbedded Build System`. Most of the | ||
294 | components and tools are downloaded separately. | ||
295 | |||
296 | This section provides brief overviews of the components and tools | ||
297 | associated with the Yocto Project. | ||
298 | |||
299 | .. _gs-development-tools: | ||
300 | |||
301 | Development Tools | ||
302 | ----------------- | ||
303 | |||
304 | The following list consists of tools that help you develop images and | ||
305 | applications using the Yocto Project: | ||
306 | |||
307 | - *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an | ||
308 | open source, cross-platform development framework that leverages | ||
309 | `Docker Containers <https://www.docker.com/>`__. CROPS provides an | ||
310 | easily managed, extensible environment that allows you to build | ||
311 | binaries for a variety of architectures on Windows, Linux and Mac OS | ||
312 | X hosts. | ||
313 | |||
314 | - *devtool:* This command-line tool is available as part of the | ||
315 | extensible SDK (eSDK) and is its cornerstone. You can use ``devtool`` | ||
316 | to help build, test, and package software within the eSDK. You can | ||
317 | use the tool to optionally integrate what you build into an image | ||
318 | built by the OpenEmbedded build system. | ||
319 | |||
320 | The ``devtool`` command employs a number of sub-commands that allow | ||
321 | you to add, modify, and upgrade recipes. As with the OpenEmbedded | ||
322 | build system, "recipes" represent software packages within | ||
323 | ``devtool``. When you use ``devtool add``, a recipe is automatically | ||
324 | created. When you use ``devtool modify``, the specified existing | ||
325 | recipe is used in order to determine where to get the source code and | ||
326 | how to patch it. In both cases, an environment is set up so that when | ||
327 | you build the recipe a source tree that is under your control is used | ||
328 | in order to allow you to make changes to the source as desired. By | ||
329 | default, both new recipes and the source go into a "workspace" | ||
330 | directory under the eSDK. The ``devtool upgrade`` command updates an | ||
331 | existing recipe so that you can build it for an updated set of source | ||
332 | files. | ||
333 | |||
334 | You can read about the ``devtool`` workflow in the Yocto Project | ||
335 | Application Development and Extensible Software Development Kit | ||
336 | (eSDK) Manual in the | ||
337 | ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
338 | section. | ||
339 | |||
340 | - *Extensible Software Development Kit (eSDK):* The eSDK provides a | ||
341 | cross-development toolchain and libraries tailored to the contents of | ||
342 | a specific image. The eSDK makes it easy to add new applications and | ||
343 | libraries to an image, modify the source for an existing component, | ||
344 | test changes on the target hardware, and integrate into the rest of | ||
345 | the OpenEmbedded build system. The eSDK gives you a toolchain | ||
346 | experience supplemented with the powerful set of ``devtool`` commands | ||
347 | tailored for the Yocto Project environment. | ||
348 | |||
349 | For information on the eSDK, see the :doc:`../sdk-manual/sdk-manual` Manual. | ||
350 | |||
351 | - *Toaster:* Toaster is a web interface to the Yocto Project | ||
352 | OpenEmbedded build system. Toaster allows you to configure, run, and | ||
353 | view information about builds. For information on Toaster, see the | ||
354 | :doc:`../toaster-manual/toaster-manual`. | ||
355 | |||
356 | .. _gs-production-tools: | ||
357 | |||
358 | Production Tools | ||
359 | ---------------- | ||
360 | |||
361 | The following list consists of tools that help production related | ||
362 | activities using the Yocto Project: | ||
363 | |||
364 | - *Auto Upgrade Helper:* This utility when used in conjunction with the | ||
365 | :term:`OpenEmbedded Build System` | ||
366 | (BitBake and | ||
367 | OE-Core) automatically generates upgrades for recipes that are based | ||
368 | on new versions of the recipes published upstream. See | ||
369 | :ref:`dev-manual/dev-manual-common-tasks:using the auto upgrade helper (auh)` | ||
370 | for how to set it up. | ||
371 | |||
372 | - *Recipe Reporting System:* The Recipe Reporting System tracks recipe | ||
373 | versions available for Yocto Project. The main purpose of the system | ||
374 | is to help you manage the recipes you maintain and to offer a dynamic | ||
375 | overview of the project. The Recipe Reporting System is built on top | ||
376 | of the `OpenEmbedded Layer | ||
377 | Index <http://layers.openembedded.org/layerindex/layers/>`__, which | ||
378 | is a website that indexes OpenEmbedded-Core layers. | ||
379 | |||
380 | - *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__ | ||
381 | is a fork of a project originally started by | ||
382 | `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking | ||
383 | system designed to streamline the process of bringing contributions | ||
384 | into a project. The Yocto Project uses Patchwork as an organizational | ||
385 | tool to handle patches, which number in the thousands for every | ||
386 | release. | ||
387 | |||
388 | - *AutoBuilder:* AutoBuilder is a project that automates build tests | ||
389 | and quality assurance (QA). By using the public AutoBuilder, anyone | ||
390 | can determine the status of the current "master" branch of Poky. | ||
391 | |||
392 | .. note:: | ||
393 | |||
394 | AutoBuilder is based on buildbot. | ||
395 | |||
396 | A goal of the Yocto Project is to lead the open source industry with | ||
397 | a project that automates testing and QA procedures. In doing so, the | ||
398 | project encourages a development community that publishes QA and test | ||
399 | plans, publicly demonstrates QA and test plans, and encourages | ||
400 | development of tools that automate and test and QA procedures for the | ||
401 | benefit of the development community. | ||
402 | |||
403 | You can learn more about the AutoBuilder used by the Yocto Project | ||
404 | Autobuilder :doc:`here <../test-manual/test-manual-understand-autobuilder>`. | ||
405 | |||
406 | - *Cross-Prelink:* Prelinking is the process of pre-computing the load | ||
407 | addresses and link tables generated by the dynamic linker as compared | ||
408 | to doing this at runtime. Doing this ahead of time results in | ||
409 | performance improvements when the application is launched and reduced | ||
410 | memory usage for libraries shared by many applications. | ||
411 | |||
412 | Historically, cross-prelink is a variant of prelink, which was | ||
413 | conceived by `Jakub | ||
414 | JelÃnek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of | ||
415 | years ago. Both prelink and cross-prelink are maintained in the same | ||
416 | repository albeit on separate branches. By providing an emulated | ||
417 | runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation), | ||
418 | the cross-prelink project extends the prelink software's ability to | ||
419 | prelink a sysroot environment. Additionally, the cross-prelink | ||
420 | software enables the ability to work in sysroot style environments. | ||
421 | |||
422 | The dynamic linker determines standard load address calculations | ||
423 | based on a variety of factors such as mapping addresses, library | ||
424 | usage, and library function conflicts. The prelink tool uses this | ||
425 | information, from the dynamic linker, to determine unique load | ||
426 | addresses for executable and linkable format (ELF) binaries that are | ||
427 | shared libraries and dynamically linked. The prelink tool modifies | ||
428 | these ELF binaries with the pre-computed information. The result is | ||
429 | faster loading and often lower memory consumption because more of the | ||
430 | library code can be re-used from shared Copy-On-Write (COW) pages. | ||
431 | |||
432 | The original upstream prelink project only supports running prelink | ||
433 | on the end target device due to the reliance on the target device's | ||
434 | dynamic linker. This restriction causes issues when developing a | ||
435 | cross-compiled system. The cross-prelink adds a synthesized dynamic | ||
436 | loader that runs on the host, thus permitting cross-prelinking | ||
437 | without ever having to run on a read-write target filesystem. | ||
438 | |||
439 | - *Pseudo:* Pseudo is the Yocto Project implementation of | ||
440 | `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run | ||
441 | commands in an environment that seemingly has root privileges. | ||
442 | |||
443 | During a build, it can be necessary to perform operations that | ||
444 | require system administrator privileges. For example, file ownership | ||
445 | or permissions might need definition. Pseudo is a tool that you can | ||
446 | either use directly or through the environment variable | ||
447 | ``LD_PRELOAD``. Either method allows these operations to succeed as | ||
448 | if system administrator privileges exist even when they do not. | ||
449 | |||
450 | You can read more about Pseudo in the "`Fakeroot and | ||
451 | Pseudo <#fakeroot-and-pseudo>`__" section. | ||
452 | |||
453 | .. _gs-openembedded-build-system: | ||
454 | |||
455 | Open-Embedded Build System Components | ||
456 | ------------------------------------- | ||
457 | |||
458 | The following list consists of components associated with the | ||
459 | :term:`OpenEmbedded Build System`: | ||
460 | |||
461 | - *BitBake:* BitBake is a core component of the Yocto Project and is | ||
462 | used by the OpenEmbedded build system to build images. While BitBake | ||
463 | is key to the build system, BitBake is maintained separately from the | ||
464 | Yocto Project. | ||
465 | |||
466 | BitBake is a generic task execution engine that allows shell and | ||
467 | Python tasks to be run efficiently and in parallel while working | ||
468 | within complex inter-task dependency constraints. In short, BitBake | ||
469 | is a build engine that works through recipes written in a specific | ||
470 | format in order to perform sets of tasks. | ||
471 | |||
472 | You can learn more about BitBake in the :doc:`BitBake User | ||
473 | Manual <bitbake:index>`. | ||
474 | |||
475 | - *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of | ||
476 | metadata (i.e. recipes, classes, and associated files) used by | ||
477 | OpenEmbedded-derived systems, which includes the Yocto Project. The | ||
478 | Yocto Project and the OpenEmbedded Project both maintain the | ||
479 | OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto | ||
480 | Project :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/meta>`. | ||
481 | |||
482 | Historically, the Yocto Project integrated the OE-Core metadata | ||
483 | throughout the Yocto Project source repository reference system | ||
484 | (Poky). After Yocto Project Version 1.0, the Yocto Project and | ||
485 | OpenEmbedded agreed to work together and share a common core set of | ||
486 | metadata (OE-Core), which contained much of the functionality | ||
487 | previously found in Poky. This collaboration achieved a long-standing | ||
488 | OpenEmbedded objective for having a more tightly controlled and | ||
489 | quality-assured core. The results also fit well with the Yocto | ||
490 | Project objective of achieving a smaller number of fully featured | ||
491 | tools as compared to many different ones. | ||
492 | |||
493 | Sharing a core set of metadata results in Poky as an integration | ||
494 | layer on top of OE-Core. You can see that in this | ||
495 | `figure <#yp-key-dev-elements>`__. The Yocto Project combines various | ||
496 | components such as BitBake, OE-Core, script "glue", and documentation | ||
497 | for its build system. | ||
498 | |||
499 | .. _gs-reference-distribution-poky: | ||
500 | |||
501 | Reference Distribution (Poky) | ||
502 | ----------------------------- | ||
503 | |||
504 | Poky is the Yocto Project reference distribution. It contains the | ||
505 | :term:`OpenEmbedded Build System` | ||
506 | (BitBake and OE-Core) as well as a set of metadata to get you started | ||
507 | building your own distribution. See the | ||
508 | `figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?" | ||
509 | section for an illustration that shows Poky and its relationship with | ||
510 | other parts of the Yocto Project. | ||
511 | |||
512 | To use the Yocto Project tools and components, you can download | ||
513 | (``clone``) Poky and use it to bootstrap your own distribution. | ||
514 | |||
515 | .. note:: | ||
516 | |||
517 | Poky does not contain binary files. It is a working example of how to | ||
518 | build your own custom Linux distribution from source. | ||
519 | |||
520 | You can read more about Poky in the "`Reference Embedded Distribution | ||
521 | (Poky) <#reference-embedded-distribution>`__" section. | ||
522 | |||
523 | .. _gs-packages-for-finished-targets: | ||
524 | |||
525 | Packages for Finished Targets | ||
526 | ----------------------------- | ||
527 | |||
528 | The following lists components associated with packages for finished | ||
529 | targets: | ||
530 | |||
531 | - *Matchbox:* Matchbox is an Open Source, base environment for the X | ||
532 | Window System running on non-desktop, embedded platforms such as | ||
533 | handhelds, set-top boxes, kiosks, and anything else for which screen | ||
534 | space, input mechanisms, or system resources are limited. | ||
535 | |||
536 | Matchbox consists of a number of interchangeable and optional | ||
537 | applications that you can tailor to a specific, non-desktop platform | ||
538 | to enhance usability in constrained environments. | ||
539 | |||
540 | You can find the Matchbox source in the Yocto Project | ||
541 | :yocto_git:`Source Repositories <>`. | ||
542 | |||
543 | - *Opkg:* Open PacKaGe management (opkg) is a lightweight package | ||
544 | management system based on the itsy package (ipkg) management system. | ||
545 | Opkg is written in C and resembles Advanced Package Tool (APT) and | ||
546 | Debian Package (dpkg) in operation. | ||
547 | |||
548 | Opkg is intended for use on embedded Linux devices and is used in | ||
549 | this capacity in the | ||
550 | `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and | ||
551 | `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto | ||
552 | Project. | ||
553 | |||
554 | .. note:: | ||
555 | |||
556 | As best it can, opkg maintains backwards compatibility with ipkg | ||
557 | and conforms to a subset of Debian's policy manual regarding | ||
558 | control files. | ||
559 | |||
560 | You can find the opkg source in the Yocto Project | ||
561 | :yocto_git:`Source Repositories <>`. | ||
562 | |||
563 | .. _gs-archived-components: | ||
564 | |||
565 | Archived Components | ||
566 | ------------------- | ||
567 | |||
568 | The Build Appliance is a virtual machine image that enables you to build | ||
569 | and boot a custom embedded Linux image with the Yocto Project using a | ||
570 | non-Linux development system. | ||
571 | |||
572 | Historically, the Build Appliance was the second of three methods by | ||
573 | which you could use the Yocto Project on a system that was not native to | ||
574 | Linux. | ||
575 | |||
576 | 1. *Hob:* Hob, which is now deprecated and is no longer available since | ||
577 | the 2.1 release of the Yocto Project provided a rudimentary, | ||
578 | GUI-based interface to the Yocto Project. Toaster has fully replaced | ||
579 | Hob. | ||
580 | |||
581 | 2. *Build Appliance:* Post Hob, the Build Appliance became available. It | ||
582 | was never recommended that you use the Build Appliance as a | ||
583 | day-to-day production development environment with the Yocto Project. | ||
584 | Build Appliance was useful as a way to try out development in the | ||
585 | Yocto Project environment. | ||
586 | |||
587 | 3. *CROPS:* The final and best solution available now for developing | ||
588 | using the Yocto Project on a system not native to Linux is with | ||
589 | `CROPS <#gs-crops-overview>`__. | ||
590 | |||
591 | .. _gs-development-methods: | ||
592 | |||
593 | Development Methods | ||
594 | =================== | ||
595 | |||
596 | The Yocto Project development environment usually involves a | ||
597 | :term:`Build Host` and target | ||
598 | hardware. You use the Build Host to build images and develop | ||
599 | applications, while you use the target hardware to test deployed | ||
600 | software. | ||
601 | |||
602 | This section provides an introduction to the choices or development | ||
603 | methods you have when setting up your Build Host. Depending on the your | ||
604 | particular workflow preference and the type of operating system your | ||
605 | Build Host runs, several choices exist that allow you to use the Yocto | ||
606 | Project. | ||
607 | |||
608 | .. note:: | ||
609 | |||
610 | For additional detail about the Yocto Project development | ||
611 | environment, see the ":doc:`overview-manual-development-environment`" | ||
612 | chapter. | ||
613 | |||
614 | - *Native Linux Host:* By far the best option for a Build Host. A | ||
615 | system running Linux as its native operating system allows you to | ||
616 | develop software by directly using the | ||
617 | :term:`BitBake` tool. You can | ||
618 | accomplish all aspects of development from a familiar shell of a | ||
619 | supported Linux distribution. | ||
620 | |||
621 | For information on how to set up a Build Host on a system running | ||
622 | Linux as its native operating system, see the | ||
623 | ":ref:`dev-manual/dev-manual-start:setting up a native linux host`" | ||
624 | section in the Yocto Project Development Tasks Manual. | ||
625 | |||
626 | - *CROss PlatformS (CROPS):* Typically, you use | ||
627 | `CROPS <https://github.com/crops/poky-container/>`__, which leverages | ||
628 | `Docker Containers <https://www.docker.com/>`__, to set up a Build | ||
629 | Host that is not running Linux (e.g. Microsoft Windows or macOS). | ||
630 | |||
631 | .. note:: | ||
632 | |||
633 | You can, however, use CROPS on a Linux-based system. | ||
634 | |||
635 | CROPS is an open source, cross-platform development framework that | ||
636 | provides an easily managed, extensible environment for building | ||
637 | binaries targeted for a variety of architectures on Windows, macOS, | ||
638 | or Linux hosts. Once the Build Host is set up using CROPS, you can | ||
639 | prepare a shell environment to mimic that of a shell being used on a | ||
640 | system natively running Linux. | ||
641 | |||
642 | For information on how to set up a Build Host with CROPS, see the | ||
643 | ":ref:`dev-manual/dev-manual-start:setting up to use cross platforms (crops)`" | ||
644 | section in the Yocto Project Development Tasks Manual. | ||
645 | |||
646 | - *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem | ||
647 | For Linux v2 to set up a build host using Windows 10. | ||
648 | |||
649 | .. note:: | ||
650 | |||
651 | The Yocto Project is not compatible with WSLv1, it is compatible | ||
652 | but not officially supported nor validated with WSLv2, if you | ||
653 | still decide to use WSL please upgrade to WSLv2. | ||
654 | |||
655 | The Windows Subsystem For Linux allows Windows 10 to run a real Linux | ||
656 | kernel inside of a lightweight utility virtual machine (VM) using | ||
657 | virtualization technology. | ||
658 | |||
659 | For information on how to set up a Build Host with WSLv2, see the | ||
660 | ":ref:`dev-manual/dev-manual-start:setting up to use windows subsystem for linux (wslv2)`" | ||
661 | section in the Yocto Project Development Tasks Manual. | ||
662 | |||
663 | - *Toaster:* Regardless of what your Build Host is running, you can use | ||
664 | Toaster to develop software using the Yocto Project. Toaster is a web | ||
665 | interface to the Yocto Project's :term:`OpenEmbedded Build System`. | ||
666 | The interface | ||
667 | enables you to configure and run your builds. Information about | ||
668 | builds is collected and stored in a database. You can use Toaster to | ||
669 | configure and start builds on multiple remote build servers. | ||
670 | |||
671 | For information about and how to use Toaster, see the | ||
672 | :doc:`../toaster-manual/toaster-manual`. | ||
673 | |||
674 | .. _reference-embedded-distribution: | ||
675 | |||
676 | Reference Embedded Distribution (Poky) | ||
677 | ====================================== | ||
678 | |||
679 | "Poky", which is pronounced *Pock*-ee, is the name of the Yocto | ||
680 | Project's reference distribution or Reference OS Kit. Poky contains the | ||
681 | :term:`OpenEmbedded Build System` | ||
682 | (:term:`BitBake` and | ||
683 | :term:`OpenEmbedded-Core (OE-Core)`) as well as a set | ||
684 | of :term:`Metadata` to get you started | ||
685 | building your own distro. In other words, Poky is a base specification | ||
686 | of the functionality needed for a typical embedded system as well as the | ||
687 | components from the Yocto Project that allow you to build a distribution | ||
688 | into a usable binary image. | ||
689 | |||
690 | Poky is a combined repository of BitBake, OpenEmbedded-Core (which is | ||
691 | found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation | ||
692 | provided all together and known to work well together. You can view | ||
693 | these items that make up the Poky repository in the | ||
694 | :yocto_git:`Source Repositories </cgit/cgit.cgi/poky/tree/>`. | ||
695 | |||
696 | .. note:: | ||
697 | |||
698 | If you are interested in all the contents of the | ||
699 | poky | ||
700 | Git repository, see the ":ref:`ref-manual/ref-structure:top-level core components`" | ||
701 | section in the Yocto Project Reference Manual. | ||
702 | |||
703 | The following figure illustrates what generally comprises Poky: | ||
704 | |||
705 | .. image:: figures/poky-reference-distribution.png | ||
706 | :align: center | ||
707 | |||
708 | - BitBake is a task executor and scheduler that is the heart of the | ||
709 | OpenEmbedded build system. | ||
710 | |||
711 | - ``meta-poky``, which is Poky-specific metadata. | ||
712 | |||
713 | - ``meta-yocto-bsp``, which are Yocto Project-specific Board Support | ||
714 | Packages (BSPs). | ||
715 | |||
716 | - OpenEmbedded-Core (OE-Core) metadata, which includes shared | ||
717 | configurations, global variable definitions, shared classes, | ||
718 | packaging, and recipes. Classes define the encapsulation and | ||
719 | inheritance of build logic. Recipes are the logical units of software | ||
720 | and images to be built. | ||
721 | |||
722 | - Documentation, which contains the Yocto Project source files used to | ||
723 | make the set of user manuals. | ||
724 | |||
725 | .. note:: | ||
726 | |||
727 | While Poky is a "complete" distribution specification and is tested | ||
728 | and put through QA, you cannot use it as a product "out of the box" | ||
729 | in its current form. | ||
730 | |||
731 | To use the Yocto Project tools, you can use Git to clone (download) the | ||
732 | Poky repository then use your local copy of the reference distribution | ||
733 | to bootstrap your own distribution. | ||
734 | |||
735 | .. note:: | ||
736 | |||
737 | Poky does not contain binary files. It is a working example of how to | ||
738 | build your own custom Linux distribution from source. | ||
739 | |||
740 | Poky has a regular, well established, six-month release cycle under its | ||
741 | own version. Major releases occur at the same time major releases (point | ||
742 | releases) occur for the Yocto Project, which are typically in the Spring | ||
743 | and Fall. For more information on the Yocto Project release schedule and | ||
744 | cadence, see the ":doc:`../ref-manual/ref-release-process`" chapter in the | ||
745 | Yocto Project Reference Manual. | ||
746 | |||
747 | Much has been said about Poky being a "default configuration". A default | ||
748 | configuration provides a starting image footprint. You can use Poky out | ||
749 | of the box to create an image ranging from a shell-accessible minimal | ||
750 | image all the way up to a Linux Standard Base-compliant image that uses | ||
751 | a GNOME Mobile and Embedded (GMAE) based reference user interface called | ||
752 | Sato. | ||
753 | |||
754 | One of the most powerful properties of Poky is that every aspect of a | ||
755 | build is controlled by the metadata. You can use metadata to augment | ||
756 | these base image types by adding metadata | ||
757 | `layers <#the-yocto-project-layer-model>`__ that extend functionality. | ||
758 | These layers can provide, for example, an additional software stack for | ||
759 | an image type, add a board support package (BSP) for additional | ||
760 | hardware, or even create a new image type. | ||
761 | |||
762 | Metadata is loosely grouped into configuration files or package recipes. | ||
763 | A recipe is a collection of non-executable metadata used by BitBake to | ||
764 | set variables or define additional build-time tasks. A recipe contains | ||
765 | fields such as the recipe description, the recipe version, the license | ||
766 | of the package and the upstream source repository. A recipe might also | ||
767 | indicate that the build process uses autotools, make, distutils or any | ||
768 | other build process, in which case the basic functionality can be | ||
769 | defined by the classes it inherits from the OE-Core layer's class | ||
770 | definitions in ``./meta/classes``. Within a recipe you can also define | ||
771 | additional tasks as well as task prerequisites. Recipe syntax through | ||
772 | BitBake also supports both ``_prepend`` and ``_append`` operators as a | ||
773 | method of extending task functionality. These operators inject code into | ||
774 | the beginning or end of a task. For information on these BitBake | ||
775 | operators, see the | ||
776 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`" | ||
777 | section in the BitBake User's Manual. | ||
778 | |||
779 | .. _openembedded-build-system-workflow: | ||
780 | |||
781 | The OpenEmbedded Build System Workflow | ||
782 | ====================================== | ||
783 | |||
784 | The :term:`OpenEmbedded Build System` uses a "workflow" to | ||
785 | accomplish image and SDK generation. The following figure overviews that | ||
786 | workflow: | ||
787 | |||
788 | .. image:: figures/YP-flow-diagram.png | ||
789 | :align: center | ||
790 | |||
791 | Following is a brief summary of the "workflow": | ||
792 | |||
793 | 1. Developers specify architecture, policies, patches and configuration | ||
794 | details. | ||
795 | |||
796 | 2. The build system fetches and downloads the source code from the | ||
797 | specified location. The build system supports standard methods such | ||
798 | as tarballs or source code repositories systems such as Git. | ||
799 | |||
800 | 3. Once source code is downloaded, the build system extracts the sources | ||
801 | into a local work area where patches are applied and common steps for | ||
802 | configuring and compiling the software are run. | ||
803 | |||
804 | 4. The build system then installs the software into a temporary staging | ||
805 | area where the binary package format you select (DEB, RPM, or IPK) is | ||
806 | used to roll up the software. | ||
807 | |||
808 | 5. Different QA and sanity checks run throughout entire build process. | ||
809 | |||
810 | 6. After the binaries are created, the build system generates a binary | ||
811 | package feed that is used to create the final root file image. | ||
812 | |||
813 | 7. The build system generates the file system image and a customized | ||
814 | Extensible SDK (eSDK) for application development in parallel. | ||
815 | |||
816 | For a very detailed look at this workflow, see the "`OpenEmbedded Build | ||
817 | System Concepts <#openembedded-build-system-build-concepts>`__" section. | ||
818 | |||
819 | Some Basic Terms | ||
820 | ================ | ||
821 | |||
822 | It helps to understand some basic fundamental terms when learning the | ||
823 | Yocto Project. Although a list of terms exists in the ":doc:`Yocto Project | ||
824 | Terms <../ref-manual/ref-terms>`" section of the Yocto Project | ||
825 | Reference Manual, this section provides the definitions of some terms | ||
826 | helpful for getting started: | ||
827 | |||
828 | - *Configuration Files:* Files that hold global definitions of | ||
829 | variables, user-defined variables, and hardware configuration | ||
830 | information. These files tell the :term:`OpenEmbedded Build System` | ||
831 | what to build and | ||
832 | what to put into the image to support a particular platform. | ||
833 | |||
834 | - *Extensible Software Development Kit (eSDK):* A custom SDK for | ||
835 | application developers. This eSDK allows developers to incorporate | ||
836 | their library and programming changes back into the image to make | ||
837 | their code available to other application developers. For information | ||
838 | on the eSDK, see the :doc:`../sdk-manual/sdk-manual` manual. | ||
839 | |||
840 | - *Layer:* A collection of related recipes. Layers allow you to | ||
841 | consolidate related metadata to customize your build. Layers also | ||
842 | isolate information used when building for multiple architectures. | ||
843 | Layers are hierarchical in their ability to override previous | ||
844 | specifications. You can include any number of available layers from | ||
845 | the Yocto Project and customize the build by adding your layers after | ||
846 | them. You can search the Layer Index for layers used within Yocto | ||
847 | Project. | ||
848 | |||
849 | For more detailed information on layers, see the | ||
850 | ":ref:`dev-manual/dev-manual-common-tasks:understanding and creating layers`" | ||
851 | section in the Yocto Project Development Tasks Manual. For a | ||
852 | discussion specifically on BSP Layers, see the | ||
853 | ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto | ||
854 | Project Board Support Packages (BSP) Developer's Guide. | ||
855 | |||
856 | - *Metadata:* A key element of the Yocto Project is the Metadata that | ||
857 | is used to construct a Linux distribution and is contained in the | ||
858 | files that the OpenEmbedded build system parses when building an | ||
859 | image. In general, Metadata includes recipes, configuration files, | ||
860 | and other information that refers to the build instructions | ||
861 | themselves, as well as the data used to control what things get built | ||
862 | and the effects of the build. Metadata also includes commands and | ||
863 | data used to indicate what versions of software are used, from where | ||
864 | they are obtained, and changes or additions to the software itself | ||
865 | (patches or auxiliary files) that are used to fix bugs or customize | ||
866 | the software for use in a particular situation. OpenEmbedded-Core is | ||
867 | an important set of validated metadata. | ||
868 | |||
869 | - *OpenEmbedded Build System:* The terms "BitBake" and "build system" | ||
870 | are sometimes used for the OpenEmbedded Build System. | ||
871 | |||
872 | BitBake is a task scheduler and execution engine that parses | ||
873 | instructions (i.e. recipes) and configuration data. After a parsing | ||
874 | phase, BitBake creates a dependency tree to order the compilation, | ||
875 | schedules the compilation of the included code, and finally executes | ||
876 | the building of the specified custom Linux image (distribution). | ||
877 | BitBake is similar to the ``make`` tool. | ||
878 | |||
879 | During a build process, the build system tracks dependencies and | ||
880 | performs a native or cross-compilation of the package. As a first | ||
881 | step in a cross-build setup, the framework attempts to create a | ||
882 | cross-compiler toolchain (i.e. Extensible SDK) suited for the target | ||
883 | platform. | ||
884 | |||
885 | - *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of | ||
886 | foundation recipes, classes, and associated files that are meant to | ||
887 | be common among many different OpenEmbedded-derived systems, | ||
888 | including the Yocto Project. OE-Core is a curated subset of an | ||
889 | original repository developed by the OpenEmbedded community that has | ||
890 | been pared down into a smaller, core set of continuously validated | ||
891 | recipes. The result is a tightly controlled and quality-assured core | ||
892 | set of recipes. | ||
893 | |||
894 | You can see the Metadata in the ``meta`` directory of the Yocto | ||
895 | Project `Source | ||
896 | Repositories <http://git.yoctoproject.org/cgit/cgit.cgi>`__. | ||
897 | |||
898 | - *Packages:* In the context of the Yocto Project, this term refers to | ||
899 | a recipe's packaged output produced by BitBake (i.e. a "baked | ||
900 | recipe"). A package is generally the compiled binaries produced from | ||
901 | the recipe's sources. You "bake" something by running it through | ||
902 | BitBake. | ||
903 | |||
904 | It is worth noting that the term "package" can, in general, have | ||
905 | subtle meanings. For example, the packages referred to in the | ||
906 | ":ref:`ref-manual/ref-system-requirements:required packages for the build host`" | ||
907 | section in the Yocto Project Reference Manual are compiled binaries | ||
908 | that, when installed, add functionality to your Linux distribution. | ||
909 | |||
910 | Another point worth noting is that historically within the Yocto | ||
911 | Project, recipes were referred to as packages - thus, the existence | ||
912 | of several BitBake variables that are seemingly mis-named, (e.g. | ||
913 | :term:`PR`, | ||
914 | :term:`PV`, and | ||
915 | :term:`PE`). | ||
916 | |||
917 | - *Poky:* Poky is a reference embedded distribution and a reference | ||
918 | test configuration. Poky provides the following: | ||
919 | |||
920 | - A base-level functional distro used to illustrate how to customize | ||
921 | a distribution. | ||
922 | |||
923 | - A means by which to test the Yocto Project components (i.e. Poky | ||
924 | is used to validate the Yocto Project). | ||
925 | |||
926 | - A vehicle through which you can download the Yocto Project. | ||
927 | |||
928 | Poky is not a product level distro. Rather, it is a good starting | ||
929 | point for customization. | ||
930 | |||
931 | .. note:: | ||
932 | |||
933 | Poky is an integration layer on top of OE-Core. | ||
934 | |||
935 | - *Recipe:* The most common form of metadata. A recipe contains a list | ||
936 | of settings and tasks (i.e. instructions) for building packages that | ||
937 | are then used to build the binary image. A recipe describes where you | ||
938 | get source code and which patches to apply. Recipes describe | ||
939 | dependencies for libraries or for other recipes as well as | ||
940 | configuration and compilation options. Related recipes are | ||
941 | consolidated into a layer. | ||