diff options
Diffstat (limited to 'documentation/overview-manual/yp-intro.rst')
-rw-r--r-- | documentation/overview-manual/yp-intro.rst | 917 |
1 files changed, 917 insertions, 0 deletions
diff --git a/documentation/overview-manual/yp-intro.rst b/documentation/overview-manual/yp-intro.rst new file mode 100644 index 0000000000..3afbb9378f --- /dev/null +++ b/documentation/overview-manual/yp-intro.rst | |||
@@ -0,0 +1,917 @@ | |||
1 | .. SPDX-License-Identifier: CC-BY-SA-2.0-UK | ||
2 | |||
3 | ***************************** | ||
4 | Introducing the Yocto Project | ||
5 | ***************************** | ||
6 | |||
7 | What is the Yocto Project? | ||
8 | ========================== | ||
9 | |||
10 | The Yocto Project is an open source collaboration project that helps | ||
11 | developers create custom Linux-based systems that are designed for | ||
12 | embedded products regardless of the product's hardware architecture. | ||
13 | Yocto Project provides a flexible toolset and a development environment | ||
14 | that allows embedded device developers across the world to collaborate | ||
15 | through shared technologies, software stacks, configurations, and best | ||
16 | practices used to create these tailored Linux images. | ||
17 | |||
18 | Thousands of developers worldwide have discovered that Yocto Project | ||
19 | provides advantages in both systems and applications development, | ||
20 | archival and management benefits, and customizations used for speed, | ||
21 | footprint, and memory utilization. The project is a standard when it | ||
22 | comes to delivering embedded software stacks. The project allows | ||
23 | software customizations and build interchange for multiple hardware | ||
24 | platforms as well as software stacks that can be maintained and scaled. | ||
25 | |||
26 | .. image:: figures/key-dev-elements.png | ||
27 | :align: center | ||
28 | |||
29 | For further introductory information on the Yocto Project, you might be | ||
30 | interested in this | ||
31 | `article <https://www.embedded.com/electronics-blogs/say-what-/4458600/Why-the-Yocto-Project-for-my-IoT-Project->`__ | ||
32 | by Drew Moseley and in this short introductory | ||
33 | `video <https://www.youtube.com/watch?v=utZpKM7i5Z4>`__. | ||
34 | |||
35 | The remainder of this section overviews advantages and challenges tied | ||
36 | to the Yocto Project. | ||
37 | |||
38 | Features | ||
39 | -------- | ||
40 | |||
41 | The following list describes features and advantages of the Yocto | ||
42 | Project: | ||
43 | |||
44 | - *Widely Adopted Across the Industry:* Semiconductor, operating | ||
45 | system, software, and service vendors exist whose products and | ||
46 | services adopt and support the Yocto Project. For a look at the Yocto | ||
47 | Project community and the companies involved with the Yocto Project, | ||
48 | see the "COMMUNITY" and "ECOSYSTEM" tabs on the | ||
49 | :yocto_home:`Yocto Project <>` home page. | ||
50 | |||
51 | - *Architecture Agnostic:* Yocto Project supports Intel, ARM, MIPS, | ||
52 | AMD, PPC and other architectures. Most ODMs, OSVs, and chip vendors | ||
53 | create and supply BSPs that support their hardware. If you have | ||
54 | custom silicon, you can create a BSP that supports that architecture. | ||
55 | |||
56 | Aside from lots of architecture support, the Yocto Project fully | ||
57 | supports a wide range of device emulation through the Quick EMUlator | ||
58 | (QEMU). | ||
59 | |||
60 | - *Images and Code Transfer Easily:* Yocto Project output can easily | ||
61 | move between architectures without moving to new development | ||
62 | environments. Additionally, if you have used the Yocto Project to | ||
63 | create an image or application and you find yourself not able to | ||
64 | support it, commercial Linux vendors such as Wind River, Mentor | ||
65 | Graphics, Timesys, and ENEA could take it and provide ongoing | ||
66 | support. These vendors have offerings that are built using the Yocto | ||
67 | Project. | ||
68 | |||
69 | - *Flexibility:* Corporations use the Yocto Project many different | ||
70 | ways. One example is to create an internal Linux distribution as a | ||
71 | code base the corporation can use across multiple product groups. | ||
72 | Through customization and layering, a project group can leverage the | ||
73 | base Linux distribution to create a distribution that works for their | ||
74 | product needs. | ||
75 | |||
76 | - *Ideal for Constrained Embedded and IoT devices:* Unlike a full Linux | ||
77 | distribution, you can use the Yocto Project to create exactly what | ||
78 | you need for embedded devices. You only add the feature support or | ||
79 | packages that you absolutely need for the device. For devices that | ||
80 | have display hardware, you can use available system components such | ||
81 | as X11, GTK+, Qt, Clutter, and SDL (among others) to create a rich | ||
82 | user experience. For devices that do not have a display or where you | ||
83 | want to use alternative UI frameworks, you can choose to not install | ||
84 | these components. | ||
85 | |||
86 | - *Comprehensive Toolchain Capabilities:* Toolchains for supported | ||
87 | architectures satisfy most use cases. However, if your hardware | ||
88 | supports features that are not part of a standard toolchain, you can | ||
89 | easily customize that toolchain through specification of | ||
90 | platform-specific tuning parameters. And, should you need to use a | ||
91 | third-party toolchain, mechanisms built into the Yocto Project allow | ||
92 | for that. | ||
93 | |||
94 | - *Mechanism Rules Over Policy:* Focusing on mechanism rather than | ||
95 | policy ensures that you are free to set policies based on the needs | ||
96 | of your design instead of adopting decisions enforced by some system | ||
97 | software provider. | ||
98 | |||
99 | - *Uses a Layer Model:* The Yocto Project `layer | ||
100 | infrastructure <#the-yocto-project-layer-model>`__ groups related | ||
101 | functionality into separate bundles. You can incrementally add these | ||
102 | grouped functionalities to your project as needed. Using layers to | ||
103 | isolate and group functionality reduces project complexity and | ||
104 | redundancy, allows you to easily extend the system, make | ||
105 | customizations, and keep functionality organized. | ||
106 | |||
107 | - *Supports Partial Builds:* You can build and rebuild individual | ||
108 | packages as needed. Yocto Project accomplishes this through its | ||
109 | `shared-state cache <#shared-state-cache>`__ (sstate) scheme. Being | ||
110 | able to build and debug components individually eases project | ||
111 | development. | ||
112 | |||
113 | - *Releases According to a Strict Schedule:* Major releases occur on a | ||
114 | :doc:`six-month cycle </ref-manual/ref-release-process>` | ||
115 | predictably in October and April. The most recent two releases | ||
116 | support point releases to address common vulnerabilities and | ||
117 | exposures. This predictability is crucial for projects based on the | ||
118 | Yocto Project and allows development teams to plan activities. | ||
119 | |||
120 | - *Rich Ecosystem of Individuals and Organizations:* For open source | ||
121 | projects, the value of community is very important. Support forums, | ||
122 | expertise, and active developers who continue to push the Yocto | ||
123 | Project forward are readily available. | ||
124 | |||
125 | - *Binary Reproducibility:* The Yocto Project allows you to be very | ||
126 | specific about dependencies and achieves very high percentages of | ||
127 | binary reproducibility (e.g. 99.8% for ``core-image-minimal``). When | ||
128 | distributions are not specific about which packages are pulled in and | ||
129 | in what order to support dependencies, other build systems can | ||
130 | arbitrarily include packages. | ||
131 | |||
132 | - *License Manifest:* The Yocto Project provides a :ref:`license | ||
133 | manifest <dev-manual/common-tasks:maintaining open source license compliance during your product's lifecycle>` | ||
134 | for review by people who need to track the use of open source | ||
135 | licenses (e.g. legal teams). | ||
136 | |||
137 | Challenges | ||
138 | ---------- | ||
139 | |||
140 | The following list presents challenges you might encounter when | ||
141 | developing using the Yocto Project: | ||
142 | |||
143 | - *Steep Learning Curve:* The Yocto Project has a steep learning curve | ||
144 | and has many different ways to accomplish similar tasks. It can be | ||
145 | difficult to choose how to proceed when varying methods exist by | ||
146 | which to accomplish a given task. | ||
147 | |||
148 | - *Understanding What Changes You Need to Make For Your Design Requires | ||
149 | Some Research:* Beyond the simple tutorial stage, understanding what | ||
150 | changes need to be made for your particular design can require a | ||
151 | significant amount of research and investigation. For information | ||
152 | that helps you transition from trying out the Yocto Project to using | ||
153 | it for your project, see the ":ref:`what-i-wish-id-known:what i wish i'd known about yocto project`" and | ||
154 | ":ref:`transitioning-to-a-custom-environment:transitioning to a custom environment for systems development`" | ||
155 | documents on the Yocto Project website. | ||
156 | |||
157 | - *Project Workflow Could Be Confusing:* The `Yocto Project | ||
158 | workflow <#overview-development-environment>`__ could be confusing if | ||
159 | you are used to traditional desktop and server software development. | ||
160 | In a desktop development environment, mechanisms exist to easily pull | ||
161 | and install new packages, which are typically pre-compiled binaries | ||
162 | from servers accessible over the Internet. Using the Yocto Project, | ||
163 | you must modify your configuration and rebuild to add additional | ||
164 | packages. | ||
165 | |||
166 | - *Working in a Cross-Build Environment Can Feel Unfamiliar:* When | ||
167 | developing code to run on a target, compilation, execution, and | ||
168 | testing done on the actual target can be faster than running a | ||
169 | BitBake build on a development host and then deploying binaries to | ||
170 | the target for test. While the Yocto Project does support development | ||
171 | tools on the target, the additional step of integrating your changes | ||
172 | back into the Yocto Project build environment would be required. | ||
173 | Yocto Project supports an intermediate approach that involves making | ||
174 | changes on the development system within the BitBake environment and | ||
175 | then deploying only the updated packages to the target. | ||
176 | |||
177 | The Yocto Project :term:`OpenEmbedded Build System` | ||
178 | produces packages | ||
179 | in standard formats (i.e. RPM, DEB, IPK, and TAR). You can deploy | ||
180 | these packages into the running system on the target by using | ||
181 | utilities on the target such as ``rpm`` or ``ipk``. | ||
182 | |||
183 | - *Initial Build Times Can be Significant:* Long initial build times | ||
184 | are unfortunately unavoidable due to the large number of packages | ||
185 | initially built from scratch for a fully functioning Linux system. | ||
186 | Once that initial build is completed, however, the shared-state | ||
187 | (sstate) cache mechanism Yocto Project uses keeps the system from | ||
188 | rebuilding packages that have not been "touched" since the last | ||
189 | build. The sstate mechanism significantly reduces times for | ||
190 | successive builds. | ||
191 | |||
192 | The Yocto Project Layer Model | ||
193 | ============================= | ||
194 | |||
195 | The Yocto Project's "Layer Model" is a development model for embedded | ||
196 | and IoT Linux creation that distinguishes the Yocto Project from other | ||
197 | simple build systems. The Layer Model simultaneously supports | ||
198 | collaboration and customization. Layers are repositories that contain | ||
199 | related sets of instructions that tell the :term:`OpenEmbedded Build System` | ||
200 | what to do. You can | ||
201 | collaborate, share, and reuse layers. | ||
202 | |||
203 | Layers can contain changes to previous instructions or settings at any | ||
204 | time. This powerful override capability is what allows you to customize | ||
205 | previously supplied collaborative or community layers to suit your | ||
206 | product requirements. | ||
207 | |||
208 | You use different layers to logically separate information in your | ||
209 | build. As an example, you could have BSP, GUI, distro configuration, | ||
210 | middleware, or application layers. Putting your entire build into one | ||
211 | layer limits and complicates future customization and reuse. Isolating | ||
212 | information into layers, on the other hand, helps simplify future | ||
213 | customizations and reuse. You might find it tempting to keep everything | ||
214 | in one layer when working on a single project. However, the more modular | ||
215 | your Metadata, the easier it is to cope with future changes. | ||
216 | |||
217 | .. note:: | ||
218 | |||
219 | - Use Board Support Package (BSP) layers from silicon vendors when | ||
220 | possible. | ||
221 | |||
222 | - Familiarize yourself with the `Yocto Project curated layer | ||
223 | index <https://www.yoctoproject.org/software-overview/layers/>`__ | ||
224 | or the `OpenEmbedded layer | ||
225 | index <http://layers.openembedded.org/layerindex/branch/master/layers/>`__. | ||
226 | The latter contains more layers but they are less universally | ||
227 | validated. | ||
228 | |||
229 | - Layers support the inclusion of technologies, hardware components, | ||
230 | and software components. The :ref:`Yocto Project | ||
231 | Compatible <dev-manual/common-tasks:making sure your layer is compatible with yocto project>` | ||
232 | designation provides a minimum level of standardization that | ||
233 | contributes to a strong ecosystem. "YP Compatible" is applied to | ||
234 | appropriate products and software components such as BSPs, other | ||
235 | OE-compatible layers, and related open-source projects, allowing | ||
236 | the producer to use Yocto Project badges and branding assets. | ||
237 | |||
238 | To illustrate how layers are used to keep things modular, consider | ||
239 | machine customizations. These types of customizations typically reside | ||
240 | in a special layer, rather than a general layer, called a BSP Layer. | ||
241 | Furthermore, the machine customizations should be isolated from recipes | ||
242 | and Metadata that support a new GUI environment, for example. This | ||
243 | situation gives you a couple of layers: one for the machine | ||
244 | configurations, and one for the GUI environment. It is important to | ||
245 | understand, however, that the BSP layer can still make machine-specific | ||
246 | additions to recipes within the GUI environment layer without polluting | ||
247 | the GUI layer itself with those machine-specific changes. You can | ||
248 | accomplish this through a recipe that is a BitBake append | ||
249 | (``.bbappend``) file, which is described later in this section. | ||
250 | |||
251 | .. note:: | ||
252 | |||
253 | For general information on BSP layer structure, see the | ||
254 | :doc:`/bsp-guide/index` | ||
255 | . | ||
256 | |||
257 | The :term:`Source Directory` | ||
258 | contains both general layers and BSP layers right out of the box. You | ||
259 | can easily identify layers that ship with a Yocto Project release in the | ||
260 | Source Directory by their names. Layers typically have names that begin | ||
261 | with the string ``meta-``. | ||
262 | |||
263 | .. note:: | ||
264 | |||
265 | It is not a requirement that a layer name begin with the prefix | ||
266 | meta- | ||
267 | , but it is a commonly accepted standard in the Yocto Project | ||
268 | community. | ||
269 | |||
270 | For example, if you were to examine the :yocto_git:`tree view </poky/tree/>` | ||
271 | of the ``poky`` repository, you will see several layers: ``meta``, | ||
272 | ``meta-skeleton``, ``meta-selftest``, ``meta-poky``, and | ||
273 | ``meta-yocto-bsp``. Each of these repositories represents a distinct | ||
274 | layer. | ||
275 | |||
276 | For procedures on how to create layers, see the | ||
277 | ":ref:`dev-manual/common-tasks:understanding and creating layers`" | ||
278 | section in the Yocto Project Development Tasks Manual. | ||
279 | |||
280 | Components and Tools | ||
281 | ==================== | ||
282 | |||
283 | The Yocto Project employs a collection of components and tools used by | ||
284 | the project itself, by project developers, and by those using the Yocto | ||
285 | Project. These components and tools are open source projects and | ||
286 | metadata that are separate from the reference distribution | ||
287 | (:term:`Poky`) and the | ||
288 | :term:`OpenEmbedded Build System`. Most of the | ||
289 | components and tools are downloaded separately. | ||
290 | |||
291 | This section provides brief overviews of the components and tools | ||
292 | associated with the Yocto Project. | ||
293 | |||
294 | Development Tools | ||
295 | ----------------- | ||
296 | |||
297 | The following list consists of tools that help you develop images and | ||
298 | applications using the Yocto Project: | ||
299 | |||
300 | - *CROPS:* `CROPS <https://github.com/crops/poky-container/>`__ is an | ||
301 | open source, cross-platform development framework that leverages | ||
302 | `Docker Containers <https://www.docker.com/>`__. CROPS provides an | ||
303 | easily managed, extensible environment that allows you to build | ||
304 | binaries for a variety of architectures on Windows, Linux and Mac OS | ||
305 | X hosts. | ||
306 | |||
307 | - *devtool:* This command-line tool is available as part of the | ||
308 | extensible SDK (eSDK) and is its cornerstone. You can use ``devtool`` | ||
309 | to help build, test, and package software within the eSDK. You can | ||
310 | use the tool to optionally integrate what you build into an image | ||
311 | built by the OpenEmbedded build system. | ||
312 | |||
313 | The ``devtool`` command employs a number of sub-commands that allow | ||
314 | you to add, modify, and upgrade recipes. As with the OpenEmbedded | ||
315 | build system, "recipes" represent software packages within | ||
316 | ``devtool``. When you use ``devtool add``, a recipe is automatically | ||
317 | created. When you use ``devtool modify``, the specified existing | ||
318 | recipe is used in order to determine where to get the source code and | ||
319 | how to patch it. In both cases, an environment is set up so that when | ||
320 | you build the recipe a source tree that is under your control is used | ||
321 | in order to allow you to make changes to the source as desired. By | ||
322 | default, both new recipes and the source go into a "workspace" | ||
323 | directory under the eSDK. The ``devtool upgrade`` command updates an | ||
324 | existing recipe so that you can build it for an updated set of source | ||
325 | files. | ||
326 | |||
327 | You can read about the ``devtool`` workflow in the Yocto Project | ||
328 | Application Development and Extensible Software Development Kit | ||
329 | (eSDK) Manual in the | ||
330 | ":ref:`sdk-manual/sdk-extensible:using \`\`devtool\`\` in your sdk workflow`" | ||
331 | section. | ||
332 | |||
333 | - *Extensible Software Development Kit (eSDK):* The eSDK provides a | ||
334 | cross-development toolchain and libraries tailored to the contents of | ||
335 | a specific image. The eSDK makes it easy to add new applications and | ||
336 | libraries to an image, modify the source for an existing component, | ||
337 | test changes on the target hardware, and integrate into the rest of | ||
338 | the OpenEmbedded build system. The eSDK gives you a toolchain | ||
339 | experience supplemented with the powerful set of ``devtool`` commands | ||
340 | tailored for the Yocto Project environment. | ||
341 | |||
342 | For information on the eSDK, see the :doc:`/sdk-manual/index` Manual. | ||
343 | |||
344 | - *Toaster:* Toaster is a web interface to the Yocto Project | ||
345 | OpenEmbedded build system. Toaster allows you to configure, run, and | ||
346 | view information about builds. For information on Toaster, see the | ||
347 | :doc:`/toaster-manual/index`. | ||
348 | |||
349 | Production Tools | ||
350 | ---------------- | ||
351 | |||
352 | The following list consists of tools that help production related | ||
353 | activities using the Yocto Project: | ||
354 | |||
355 | - *Auto Upgrade Helper:* This utility when used in conjunction with the | ||
356 | :term:`OpenEmbedded Build System` | ||
357 | (BitBake and | ||
358 | OE-Core) automatically generates upgrades for recipes that are based | ||
359 | on new versions of the recipes published upstream. See | ||
360 | :ref:`dev-manual/common-tasks:using the auto upgrade helper (auh)` | ||
361 | for how to set it up. | ||
362 | |||
363 | - *Recipe Reporting System:* The Recipe Reporting System tracks recipe | ||
364 | versions available for Yocto Project. The main purpose of the system | ||
365 | is to help you manage the recipes you maintain and to offer a dynamic | ||
366 | overview of the project. The Recipe Reporting System is built on top | ||
367 | of the `OpenEmbedded Layer | ||
368 | Index <http://layers.openembedded.org/layerindex/layers/>`__, which | ||
369 | is a website that indexes OpenEmbedded-Core layers. | ||
370 | |||
371 | - *Patchwork:* `Patchwork <http://jk.ozlabs.org/projects/patchwork/>`__ | ||
372 | is a fork of a project originally started by | ||
373 | `OzLabs <http://ozlabs.org/>`__. The project is a web-based tracking | ||
374 | system designed to streamline the process of bringing contributions | ||
375 | into a project. The Yocto Project uses Patchwork as an organizational | ||
376 | tool to handle patches, which number in the thousands for every | ||
377 | release. | ||
378 | |||
379 | - *AutoBuilder:* AutoBuilder is a project that automates build tests | ||
380 | and quality assurance (QA). By using the public AutoBuilder, anyone | ||
381 | can determine the status of the current "master" branch of Poky. | ||
382 | |||
383 | .. note:: | ||
384 | |||
385 | AutoBuilder is based on buildbot. | ||
386 | |||
387 | A goal of the Yocto Project is to lead the open source industry with | ||
388 | a project that automates testing and QA procedures. In doing so, the | ||
389 | project encourages a development community that publishes QA and test | ||
390 | plans, publicly demonstrates QA and test plans, and encourages | ||
391 | development of tools that automate and test and QA procedures for the | ||
392 | benefit of the development community. | ||
393 | |||
394 | You can learn more about the AutoBuilder used by the Yocto Project | ||
395 | Autobuilder :doc:`here </test-manual/understand-autobuilder>`. | ||
396 | |||
397 | - *Cross-Prelink:* Prelinking is the process of pre-computing the load | ||
398 | addresses and link tables generated by the dynamic linker as compared | ||
399 | to doing this at runtime. Doing this ahead of time results in | ||
400 | performance improvements when the application is launched and reduced | ||
401 | memory usage for libraries shared by many applications. | ||
402 | |||
403 | Historically, cross-prelink is a variant of prelink, which was | ||
404 | conceived by `Jakub | ||
405 | JelÃnek <http://people.redhat.com/jakub/prelink.pdf>`__ a number of | ||
406 | years ago. Both prelink and cross-prelink are maintained in the same | ||
407 | repository albeit on separate branches. By providing an emulated | ||
408 | runtime dynamic linker (i.e. ``glibc``-derived ``ld.so`` emulation), | ||
409 | the cross-prelink project extends the prelink software's ability to | ||
410 | prelink a sysroot environment. Additionally, the cross-prelink | ||
411 | software enables the ability to work in sysroot style environments. | ||
412 | |||
413 | The dynamic linker determines standard load address calculations | ||
414 | based on a variety of factors such as mapping addresses, library | ||
415 | usage, and library function conflicts. The prelink tool uses this | ||
416 | information, from the dynamic linker, to determine unique load | ||
417 | addresses for executable and linkable format (ELF) binaries that are | ||
418 | shared libraries and dynamically linked. The prelink tool modifies | ||
419 | these ELF binaries with the pre-computed information. The result is | ||
420 | faster loading and often lower memory consumption because more of the | ||
421 | library code can be re-used from shared Copy-On-Write (COW) pages. | ||
422 | |||
423 | The original upstream prelink project only supports running prelink | ||
424 | on the end target device due to the reliance on the target device's | ||
425 | dynamic linker. This restriction causes issues when developing a | ||
426 | cross-compiled system. The cross-prelink adds a synthesized dynamic | ||
427 | loader that runs on the host, thus permitting cross-prelinking | ||
428 | without ever having to run on a read-write target filesystem. | ||
429 | |||
430 | - *Pseudo:* Pseudo is the Yocto Project implementation of | ||
431 | `fakeroot <http://man.he.net/man1/fakeroot>`__, which is used to run | ||
432 | commands in an environment that seemingly has root privileges. | ||
433 | |||
434 | During a build, it can be necessary to perform operations that | ||
435 | require system administrator privileges. For example, file ownership | ||
436 | or permissions might need definition. Pseudo is a tool that you can | ||
437 | either use directly or through the environment variable | ||
438 | ``LD_PRELOAD``. Either method allows these operations to succeed as | ||
439 | if system administrator privileges exist even when they do not. | ||
440 | |||
441 | You can read more about Pseudo in the "`Fakeroot and | ||
442 | Pseudo <#fakeroot-and-pseudo>`__" section. | ||
443 | |||
444 | Open-Embedded Build System Components | ||
445 | ------------------------------------- | ||
446 | |||
447 | The following list consists of components associated with the | ||
448 | :term:`OpenEmbedded Build System`: | ||
449 | |||
450 | - *BitBake:* BitBake is a core component of the Yocto Project and is | ||
451 | used by the OpenEmbedded build system to build images. While BitBake | ||
452 | is key to the build system, BitBake is maintained separately from the | ||
453 | Yocto Project. | ||
454 | |||
455 | BitBake is a generic task execution engine that allows shell and | ||
456 | Python tasks to be run efficiently and in parallel while working | ||
457 | within complex inter-task dependency constraints. In short, BitBake | ||
458 | is a build engine that works through recipes written in a specific | ||
459 | format in order to perform sets of tasks. | ||
460 | |||
461 | You can learn more about BitBake in the :doc:`BitBake User | ||
462 | Manual <bitbake:index>`. | ||
463 | |||
464 | - *OpenEmbedded-Core:* OpenEmbedded-Core (OE-Core) is a common layer of | ||
465 | metadata (i.e. recipes, classes, and associated files) used by | ||
466 | OpenEmbedded-derived systems, which includes the Yocto Project. The | ||
467 | Yocto Project and the OpenEmbedded Project both maintain the | ||
468 | OpenEmbedded-Core. You can find the OE-Core metadata in the Yocto | ||
469 | Project :yocto_git:`Source Repositories </poky/tree/meta>`. | ||
470 | |||
471 | Historically, the Yocto Project integrated the OE-Core metadata | ||
472 | throughout the Yocto Project source repository reference system | ||
473 | (Poky). After Yocto Project Version 1.0, the Yocto Project and | ||
474 | OpenEmbedded agreed to work together and share a common core set of | ||
475 | metadata (OE-Core), which contained much of the functionality | ||
476 | previously found in Poky. This collaboration achieved a long-standing | ||
477 | OpenEmbedded objective for having a more tightly controlled and | ||
478 | quality-assured core. The results also fit well with the Yocto | ||
479 | Project objective of achieving a smaller number of fully featured | ||
480 | tools as compared to many different ones. | ||
481 | |||
482 | Sharing a core set of metadata results in Poky as an integration | ||
483 | layer on top of OE-Core. You can see that in this | ||
484 | `figure <#yp-key-dev-elements>`__. The Yocto Project combines various | ||
485 | components such as BitBake, OE-Core, script "glue", and documentation | ||
486 | for its build system. | ||
487 | |||
488 | Reference Distribution (Poky) | ||
489 | ----------------------------- | ||
490 | |||
491 | Poky is the Yocto Project reference distribution. It contains the | ||
492 | :term:`OpenEmbedded Build System` | ||
493 | (BitBake and OE-Core) as well as a set of metadata to get you started | ||
494 | building your own distribution. See the | ||
495 | `figure <#what-is-the-yocto-project>`__ in "What is the Yocto Project?" | ||
496 | section for an illustration that shows Poky and its relationship with | ||
497 | other parts of the Yocto Project. | ||
498 | |||
499 | To use the Yocto Project tools and components, you can download | ||
500 | (``clone``) Poky and use it to bootstrap your own distribution. | ||
501 | |||
502 | .. note:: | ||
503 | |||
504 | Poky does not contain binary files. It is a working example of how to | ||
505 | build your own custom Linux distribution from source. | ||
506 | |||
507 | You can read more about Poky in the "`Reference Embedded Distribution | ||
508 | (Poky) <#reference-embedded-distribution>`__" section. | ||
509 | |||
510 | Packages for Finished Targets | ||
511 | ----------------------------- | ||
512 | |||
513 | The following lists components associated with packages for finished | ||
514 | targets: | ||
515 | |||
516 | - *Matchbox:* Matchbox is an Open Source, base environment for the X | ||
517 | Window System running on non-desktop, embedded platforms such as | ||
518 | handhelds, set-top boxes, kiosks, and anything else for which screen | ||
519 | space, input mechanisms, or system resources are limited. | ||
520 | |||
521 | Matchbox consists of a number of interchangeable and optional | ||
522 | applications that you can tailor to a specific, non-desktop platform | ||
523 | to enhance usability in constrained environments. | ||
524 | |||
525 | You can find the Matchbox source in the Yocto Project | ||
526 | :yocto_git:`Source Repositories <>`. | ||
527 | |||
528 | - *Opkg:* Open PacKaGe management (opkg) is a lightweight package | ||
529 | management system based on the itsy package (ipkg) management system. | ||
530 | Opkg is written in C and resembles Advanced Package Tool (APT) and | ||
531 | Debian Package (dpkg) in operation. | ||
532 | |||
533 | Opkg is intended for use on embedded Linux devices and is used in | ||
534 | this capacity in the | ||
535 | `OpenEmbedded <http://www.openembedded.org/wiki/Main_Page>`__ and | ||
536 | `OpenWrt <https://openwrt.org/>`__ projects, as well as the Yocto | ||
537 | Project. | ||
538 | |||
539 | .. note:: | ||
540 | |||
541 | As best it can, opkg maintains backwards compatibility with ipkg | ||
542 | and conforms to a subset of Debian's policy manual regarding | ||
543 | control files. | ||
544 | |||
545 | You can find the opkg source in the Yocto Project | ||
546 | :yocto_git:`Source Repositories <>`. | ||
547 | |||
548 | Archived Components | ||
549 | ------------------- | ||
550 | |||
551 | The Build Appliance is a virtual machine image that enables you to build | ||
552 | and boot a custom embedded Linux image with the Yocto Project using a | ||
553 | non-Linux development system. | ||
554 | |||
555 | Historically, the Build Appliance was the second of three methods by | ||
556 | which you could use the Yocto Project on a system that was not native to | ||
557 | Linux. | ||
558 | |||
559 | 1. *Hob:* Hob, which is now deprecated and is no longer available since | ||
560 | the 2.1 release of the Yocto Project provided a rudimentary, | ||
561 | GUI-based interface to the Yocto Project. Toaster has fully replaced | ||
562 | Hob. | ||
563 | |||
564 | 2. *Build Appliance:* Post Hob, the Build Appliance became available. It | ||
565 | was never recommended that you use the Build Appliance as a | ||
566 | day-to-day production development environment with the Yocto Project. | ||
567 | Build Appliance was useful as a way to try out development in the | ||
568 | Yocto Project environment. | ||
569 | |||
570 | 3. *CROPS:* The final and best solution available now for developing | ||
571 | using the Yocto Project on a system not native to Linux is with | ||
572 | `CROPS <#gs-crops-overview>`__. | ||
573 | |||
574 | Development Methods | ||
575 | =================== | ||
576 | |||
577 | The Yocto Project development environment usually involves a | ||
578 | :term:`Build Host` and target | ||
579 | hardware. You use the Build Host to build images and develop | ||
580 | applications, while you use the target hardware to test deployed | ||
581 | software. | ||
582 | |||
583 | This section provides an introduction to the choices or development | ||
584 | methods you have when setting up your Build Host. Depending on the your | ||
585 | particular workflow preference and the type of operating system your | ||
586 | Build Host runs, several choices exist that allow you to use the Yocto | ||
587 | Project. | ||
588 | |||
589 | .. note:: | ||
590 | |||
591 | For additional detail about the Yocto Project development | ||
592 | environment, see the ":doc:`/overview-manual/development-environment`" | ||
593 | chapter. | ||
594 | |||
595 | - *Native Linux Host:* By far the best option for a Build Host. A | ||
596 | system running Linux as its native operating system allows you to | ||
597 | develop software by directly using the | ||
598 | :term:`BitBake` tool. You can | ||
599 | accomplish all aspects of development from a familiar shell of a | ||
600 | supported Linux distribution. | ||
601 | |||
602 | For information on how to set up a Build Host on a system running | ||
603 | Linux as its native operating system, see the | ||
604 | ":ref:`dev-manual/start:setting up a native linux host`" | ||
605 | section in the Yocto Project Development Tasks Manual. | ||
606 | |||
607 | - *CROss PlatformS (CROPS):* Typically, you use | ||
608 | `CROPS <https://github.com/crops/poky-container/>`__, which leverages | ||
609 | `Docker Containers <https://www.docker.com/>`__, to set up a Build | ||
610 | Host that is not running Linux (e.g. Microsoft Windows or macOS). | ||
611 | |||
612 | .. note:: | ||
613 | |||
614 | You can, however, use CROPS on a Linux-based system. | ||
615 | |||
616 | CROPS is an open source, cross-platform development framework that | ||
617 | provides an easily managed, extensible environment for building | ||
618 | binaries targeted for a variety of architectures on Windows, macOS, | ||
619 | or Linux hosts. Once the Build Host is set up using CROPS, you can | ||
620 | prepare a shell environment to mimic that of a shell being used on a | ||
621 | system natively running Linux. | ||
622 | |||
623 | For information on how to set up a Build Host with CROPS, see the | ||
624 | ":ref:`dev-manual/start:setting up to use cross platforms (crops)`" | ||
625 | section in the Yocto Project Development Tasks Manual. | ||
626 | |||
627 | - *Windows Subsystem For Linux (WSLv2):* You may use Windows Subsystem | ||
628 | For Linux v2 to set up a build host using Windows 10. | ||
629 | |||
630 | .. note:: | ||
631 | |||
632 | The Yocto Project is not compatible with WSLv1, it is compatible | ||
633 | but not officially supported nor validated with WSLv2, if you | ||
634 | still decide to use WSL please upgrade to WSLv2. | ||
635 | |||
636 | The Windows Subsystem For Linux allows Windows 10 to run a real Linux | ||
637 | kernel inside of a lightweight utility virtual machine (VM) using | ||
638 | virtualization technology. | ||
639 | |||
640 | For information on how to set up a Build Host with WSLv2, see the | ||
641 | ":ref:`dev-manual/start:setting up to use windows subsystem for linux (wslv2)`" | ||
642 | section in the Yocto Project Development Tasks Manual. | ||
643 | |||
644 | - *Toaster:* Regardless of what your Build Host is running, you can use | ||
645 | Toaster to develop software using the Yocto Project. Toaster is a web | ||
646 | interface to the Yocto Project's :term:`OpenEmbedded Build System`. | ||
647 | The interface | ||
648 | enables you to configure and run your builds. Information about | ||
649 | builds is collected and stored in a database. You can use Toaster to | ||
650 | configure and start builds on multiple remote build servers. | ||
651 | |||
652 | For information about and how to use Toaster, see the | ||
653 | :doc:`/toaster-manual/index`. | ||
654 | |||
655 | Reference Embedded Distribution (Poky) | ||
656 | ====================================== | ||
657 | |||
658 | "Poky", which is pronounced *Pock*-ee, is the name of the Yocto | ||
659 | Project's reference distribution or Reference OS Kit. Poky contains the | ||
660 | :term:`OpenEmbedded Build System` | ||
661 | (:term:`BitBake` and | ||
662 | :term:`OpenEmbedded-Core (OE-Core)`) as well as a set | ||
663 | of :term:`Metadata` to get you started | ||
664 | building your own distro. In other words, Poky is a base specification | ||
665 | of the functionality needed for a typical embedded system as well as the | ||
666 | components from the Yocto Project that allow you to build a distribution | ||
667 | into a usable binary image. | ||
668 | |||
669 | Poky is a combined repository of BitBake, OpenEmbedded-Core (which is | ||
670 | found in ``meta``), ``meta-poky``, ``meta-yocto-bsp``, and documentation | ||
671 | provided all together and known to work well together. You can view | ||
672 | these items that make up the Poky repository in the | ||
673 | :yocto_git:`Source Repositories </poky/tree/>`. | ||
674 | |||
675 | .. note:: | ||
676 | |||
677 | If you are interested in all the contents of the | ||
678 | poky | ||
679 | Git repository, see the ":ref:`ref-manual/ref-structure:top-level core components`" | ||
680 | section in the Yocto Project Reference Manual. | ||
681 | |||
682 | The following figure illustrates what generally comprises Poky: | ||
683 | |||
684 | .. image:: figures/poky-reference-distribution.png | ||
685 | :align: center | ||
686 | |||
687 | - BitBake is a task executor and scheduler that is the heart of the | ||
688 | OpenEmbedded build system. | ||
689 | |||
690 | - ``meta-poky``, which is Poky-specific metadata. | ||
691 | |||
692 | - ``meta-yocto-bsp``, which are Yocto Project-specific Board Support | ||
693 | Packages (BSPs). | ||
694 | |||
695 | - OpenEmbedded-Core (OE-Core) metadata, which includes shared | ||
696 | configurations, global variable definitions, shared classes, | ||
697 | packaging, and recipes. Classes define the encapsulation and | ||
698 | inheritance of build logic. Recipes are the logical units of software | ||
699 | and images to be built. | ||
700 | |||
701 | - Documentation, which contains the Yocto Project source files used to | ||
702 | make the set of user manuals. | ||
703 | |||
704 | .. note:: | ||
705 | |||
706 | While Poky is a "complete" distribution specification and is tested | ||
707 | and put through QA, you cannot use it as a product "out of the box" | ||
708 | in its current form. | ||
709 | |||
710 | To use the Yocto Project tools, you can use Git to clone (download) the | ||
711 | Poky repository then use your local copy of the reference distribution | ||
712 | to bootstrap your own distribution. | ||
713 | |||
714 | .. note:: | ||
715 | |||
716 | Poky does not contain binary files. It is a working example of how to | ||
717 | build your own custom Linux distribution from source. | ||
718 | |||
719 | Poky has a regular, well established, six-month release cycle under its | ||
720 | own version. Major releases occur at the same time major releases (point | ||
721 | releases) occur for the Yocto Project, which are typically in the Spring | ||
722 | and Fall. For more information on the Yocto Project release schedule and | ||
723 | cadence, see the ":doc:`/ref-manual/ref-release-process`" chapter in the | ||
724 | Yocto Project Reference Manual. | ||
725 | |||
726 | Much has been said about Poky being a "default configuration". A default | ||
727 | configuration provides a starting image footprint. You can use Poky out | ||
728 | of the box to create an image ranging from a shell-accessible minimal | ||
729 | image all the way up to a Linux Standard Base-compliant image that uses | ||
730 | a GNOME Mobile and Embedded (GMAE) based reference user interface called | ||
731 | Sato. | ||
732 | |||
733 | One of the most powerful properties of Poky is that every aspect of a | ||
734 | build is controlled by the metadata. You can use metadata to augment | ||
735 | these base image types by adding metadata | ||
736 | `layers <#the-yocto-project-layer-model>`__ that extend functionality. | ||
737 | These layers can provide, for example, an additional software stack for | ||
738 | an image type, add a board support package (BSP) for additional | ||
739 | hardware, or even create a new image type. | ||
740 | |||
741 | Metadata is loosely grouped into configuration files or package recipes. | ||
742 | A recipe is a collection of non-executable metadata used by BitBake to | ||
743 | set variables or define additional build-time tasks. A recipe contains | ||
744 | fields such as the recipe description, the recipe version, the license | ||
745 | of the package and the upstream source repository. A recipe might also | ||
746 | indicate that the build process uses autotools, make, distutils or any | ||
747 | other build process, in which case the basic functionality can be | ||
748 | defined by the classes it inherits from the OE-Core layer's class | ||
749 | definitions in ``./meta/classes``. Within a recipe you can also define | ||
750 | additional tasks as well as task prerequisites. Recipe syntax through | ||
751 | BitBake also supports both ``_prepend`` and ``_append`` operators as a | ||
752 | method of extending task functionality. These operators inject code into | ||
753 | the beginning or end of a task. For information on these BitBake | ||
754 | operators, see the | ||
755 | ":ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`" | ||
756 | section in the BitBake User's Manual. | ||
757 | |||
758 | The OpenEmbedded Build System Workflow | ||
759 | ====================================== | ||
760 | |||
761 | The :term:`OpenEmbedded Build System` uses a "workflow" to | ||
762 | accomplish image and SDK generation. The following figure overviews that | ||
763 | workflow: | ||
764 | |||
765 | .. image:: figures/YP-flow-diagram.png | ||
766 | :align: center | ||
767 | |||
768 | Following is a brief summary of the "workflow": | ||
769 | |||
770 | 1. Developers specify architecture, policies, patches and configuration | ||
771 | details. | ||
772 | |||
773 | 2. The build system fetches and downloads the source code from the | ||
774 | specified location. The build system supports standard methods such | ||
775 | as tarballs or source code repositories systems such as Git. | ||
776 | |||
777 | 3. Once source code is downloaded, the build system extracts the sources | ||
778 | into a local work area where patches are applied and common steps for | ||
779 | configuring and compiling the software are run. | ||
780 | |||
781 | 4. The build system then installs the software into a temporary staging | ||
782 | area where the binary package format you select (DEB, RPM, or IPK) is | ||
783 | used to roll up the software. | ||
784 | |||
785 | 5. Different QA and sanity checks run throughout entire build process. | ||
786 | |||
787 | 6. After the binaries are created, the build system generates a binary | ||
788 | package feed that is used to create the final root file image. | ||
789 | |||
790 | 7. The build system generates the file system image and a customized | ||
791 | Extensible SDK (eSDK) for application development in parallel. | ||
792 | |||
793 | For a very detailed look at this workflow, see the "`OpenEmbedded Build | ||
794 | System Concepts <#openembedded-build-system-build-concepts>`__" section. | ||
795 | |||
796 | Some Basic Terms | ||
797 | ================ | ||
798 | |||
799 | It helps to understand some basic fundamental terms when learning the | ||
800 | Yocto Project. Although a list of terms exists in the ":doc:`Yocto Project | ||
801 | Terms </ref-manual/ref-terms>`" section of the Yocto Project | ||
802 | Reference Manual, this section provides the definitions of some terms | ||
803 | helpful for getting started: | ||
804 | |||
805 | - *Configuration Files:* Files that hold global definitions of | ||
806 | variables, user-defined variables, and hardware configuration | ||
807 | information. These files tell the :term:`OpenEmbedded Build System` | ||
808 | what to build and | ||
809 | what to put into the image to support a particular platform. | ||
810 | |||
811 | - *Extensible Software Development Kit (eSDK):* A custom SDK for | ||
812 | application developers. This eSDK allows developers to incorporate | ||
813 | their library and programming changes back into the image to make | ||
814 | their code available to other application developers. For information | ||
815 | on the eSDK, see the :doc:`/sdk-manual/index` manual. | ||
816 | |||
817 | - *Layer:* A collection of related recipes. Layers allow you to | ||
818 | consolidate related metadata to customize your build. Layers also | ||
819 | isolate information used when building for multiple architectures. | ||
820 | Layers are hierarchical in their ability to override previous | ||
821 | specifications. You can include any number of available layers from | ||
822 | the Yocto Project and customize the build by adding your layers after | ||
823 | them. You can search the Layer Index for layers used within Yocto | ||
824 | Project. | ||
825 | |||
826 | For more detailed information on layers, see the | ||
827 | ":ref:`dev-manual/common-tasks:understanding and creating layers`" | ||
828 | section in the Yocto Project Development Tasks Manual. For a | ||
829 | discussion specifically on BSP Layers, see the | ||
830 | ":ref:`bsp-guide/bsp:bsp layers`" section in the Yocto | ||
831 | Project Board Support Packages (BSP) Developer's Guide. | ||
832 | |||
833 | - *Metadata:* A key element of the Yocto Project is the Metadata that | ||
834 | is used to construct a Linux distribution and is contained in the | ||
835 | files that the OpenEmbedded build system parses when building an | ||
836 | image. In general, Metadata includes recipes, configuration files, | ||
837 | and other information that refers to the build instructions | ||
838 | themselves, as well as the data used to control what things get built | ||
839 | and the effects of the build. Metadata also includes commands and | ||
840 | data used to indicate what versions of software are used, from where | ||
841 | they are obtained, and changes or additions to the software itself | ||
842 | (patches or auxiliary files) that are used to fix bugs or customize | ||
843 | the software for use in a particular situation. OpenEmbedded-Core is | ||
844 | an important set of validated metadata. | ||
845 | |||
846 | - *OpenEmbedded Build System:* The terms "BitBake" and "build system" | ||
847 | are sometimes used for the OpenEmbedded Build System. | ||
848 | |||
849 | BitBake is a task scheduler and execution engine that parses | ||
850 | instructions (i.e. recipes) and configuration data. After a parsing | ||
851 | phase, BitBake creates a dependency tree to order the compilation, | ||
852 | schedules the compilation of the included code, and finally executes | ||
853 | the building of the specified custom Linux image (distribution). | ||
854 | BitBake is similar to the ``make`` tool. | ||
855 | |||
856 | During a build process, the build system tracks dependencies and | ||
857 | performs a native or cross-compilation of the package. As a first | ||
858 | step in a cross-build setup, the framework attempts to create a | ||
859 | cross-compiler toolchain (i.e. Extensible SDK) suited for the target | ||
860 | platform. | ||
861 | |||
862 | - *OpenEmbedded-Core (OE-Core):* OE-Core is metadata comprised of | ||
863 | foundation recipes, classes, and associated files that are meant to | ||
864 | be common among many different OpenEmbedded-derived systems, | ||
865 | including the Yocto Project. OE-Core is a curated subset of an | ||
866 | original repository developed by the OpenEmbedded community that has | ||
867 | been pared down into a smaller, core set of continuously validated | ||
868 | recipes. The result is a tightly controlled and quality-assured core | ||
869 | set of recipes. | ||
870 | |||
871 | You can see the Metadata in the ``meta`` directory of the Yocto | ||
872 | Project :yocto_git:`Source Repositories <>`. | ||
873 | |||
874 | - *Packages:* In the context of the Yocto Project, this term refers to | ||
875 | a recipe's packaged output produced by BitBake (i.e. a "baked | ||
876 | recipe"). A package is generally the compiled binaries produced from | ||
877 | the recipe's sources. You "bake" something by running it through | ||
878 | BitBake. | ||
879 | |||
880 | It is worth noting that the term "package" can, in general, have | ||
881 | subtle meanings. For example, the packages referred to in the | ||
882 | ":ref:`ref-manual/ref-system-requirements:required packages for the build host`" | ||
883 | section in the Yocto Project Reference Manual are compiled binaries | ||
884 | that, when installed, add functionality to your Linux distribution. | ||
885 | |||
886 | Another point worth noting is that historically within the Yocto | ||
887 | Project, recipes were referred to as packages - thus, the existence | ||
888 | of several BitBake variables that are seemingly mis-named, (e.g. | ||
889 | :term:`PR`, | ||
890 | :term:`PV`, and | ||
891 | :term:`PE`). | ||
892 | |||
893 | - *Poky:* Poky is a reference embedded distribution and a reference | ||
894 | test configuration. Poky provides the following: | ||
895 | |||
896 | - A base-level functional distro used to illustrate how to customize | ||
897 | a distribution. | ||
898 | |||
899 | - A means by which to test the Yocto Project components (i.e. Poky | ||
900 | is used to validate the Yocto Project). | ||
901 | |||
902 | - A vehicle through which you can download the Yocto Project. | ||
903 | |||
904 | Poky is not a product level distro. Rather, it is a good starting | ||
905 | point for customization. | ||
906 | |||
907 | .. note:: | ||
908 | |||
909 | Poky is an integration layer on top of OE-Core. | ||
910 | |||
911 | - *Recipe:* The most common form of metadata. A recipe contains a list | ||
912 | of settings and tasks (i.e. instructions) for building packages that | ||
913 | are then used to build the binary image. A recipe describes where you | ||
914 | get source code and which patches to apply. Recipes describe | ||
915 | dependencies for libraries or for other recipes as well as | ||
916 | configuration and compilation options. Related recipes are | ||
917 | consolidated into a layer. | ||