summaryrefslogtreecommitdiffstats
path: root/documentation/ref-manual/technical-details.xml
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2013-09-09 14:34:57 -0700
committerRichard Purdie <richard.purdie@linuxfoundation.org>2013-09-12 16:50:09 +0100
commitcc265bf535f5dd77bf9818bebd24930c999cff85 (patch)
tree2b0f04e89ac20f9b407fe7d26d18dab047ba67cd /documentation/ref-manual/technical-details.xml
parent7b70da93bc86b5aa6a9907c7e9f370932ff709ec (diff)
downloadpoky-cc265bf535f5dd77bf9818bebd24930c999cff85.tar.gz
ref-manual: Created new "Closer Look" chapter
Fixes [YOCTO #2808] I extracted the section that takes a closer look at the YP development process into its own chapter. Feedback during the review indicated that this information should not be buried as it was in a section but rather pulled higher out for visibility. So, The changes create a new chapter three that is dedicated to this topic. (From yocto-docs rev: 32c66976b6b84787d14d6174dab843862a0d184b) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'documentation/ref-manual/technical-details.xml')
-rw-r--r--documentation/ref-manual/technical-details.xml1039
1 files changed, 0 insertions, 1039 deletions
diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
index bfab8a6c68..40733fae6a 100644
--- a/documentation/ref-manual/technical-details.xml
+++ b/documentation/ref-manual/technical-details.xml
@@ -153,1045 +153,6 @@
153 </section> 153 </section>
154</section> 154</section>
155 155
156<section id="a-closer-look-at-the-yocto-project-development-environment">
157 <title>A Closer Look at the Yocto Project Development Environment</title>
158
159 <para>
160 This section takes a more detailed look at the Yocto Project
161 development environment.
162 The following diagram represents the development environment at a
163 high level.
164 The remainder of this section expands on the fundamental input, output,
165 process, and
166 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>) blocks
167 in the Yocto Project development environment.
168 </para>
169
170 <para>
171 <imagedata fileref="figures/yocto-environment-ref.png" align="center" width="8in" depth="4.25in" />
172 </para>
173
174 <para>
175 The generalized Yocto Project Development Environment consists of
176 several functional areas:
177 <itemizedlist>
178 <listitem><para><emphasis>User Configuration:</emphasis>
179 Metadata you can use to control the build process.
180 </para></listitem>
181 <listitem><para><emphasis>Metadata Layers:</emphasis>
182 Various layers that provide software, machine, and
183 distro Metadata.</para></listitem>
184 <listitem><para><emphasis>Source Files:</emphasis>
185 Upstream releases, local projects, and SCMs.</para></listitem>
186 <listitem><para><emphasis>Build System:</emphasis>
187 Processes under the control of BitBake.
188 This block expands on how BitBake fetches source, applies
189 patches, completes compilation, analyzes output for package
190 generation, creates and tests packages, generates images, and
191 generates cross-development tools.</para></listitem>
192 <listitem><para><emphasis>Package Feeds:</emphasis>
193 Directories containing output packages (rpm, deb or ipk),
194 which are subsequently used in the construction of an image or
195 SDK, produced by the build system.
196 These feeds can also be copied and shared using a web server or
197 other means to facilitate extending or updating existing
198 images on devices at runtime if runtime package management is
199 enabled.</para></listitem>
200 <listitem><para><emphasis>Images:</emphasis>
201 Images produced by the development process.
202 Where do they go?
203 Can you mess with them (i.e. freely delete them or move them?).
204 </para></listitem>
205 <listitem><para><emphasis>Application Development SDK:</emphasis>
206 Cross-development tools that are produced along with an image
207 or separately with BitBake.</para></listitem>
208 </itemizedlist>
209 </para>
210
211 <section id="user-configuration">
212 <title>User Configuration</title>
213
214 <para>
215 User configuration helps define the build.
216 Through user configuration, you can tell BitBake the
217 target architecture for which you are building the image,
218 where to store downloaded source, and other build properties.
219 The following figure shows an expanded representation of the
220 user configuration box of the Yocto Project development
221 environment:
222 </para>
223
224 <para>
225 <imagedata fileref="figures/user-configuration.png" align="center" width="5.5in" depth="3.5in" />
226 </para>
227
228 <para>
229 BitBake needs some basic configuration files in order to complete
230 a build.
231 These files are <filename>*.conf</filename> files.
232 The minimally necessary ones reside as example files in the
233 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
234 For simplicity, this section refers to the Source Directory as
235 the "Poky Directory."
236 </para>
237
238 <para>
239 When you clone the <filename>poky</filename> Git repository or you
240 download and unpack a Yocto Project release, you can set up the
241 Source Directory to be named anything you want.
242 For this discussion, the cloned repository uses the default
243 name <filename>poky</filename>.
244 <note>
245 The Poky repository is primarily an aggregation of existing
246 repositories.
247 It is not a canonical upstream source.
248 </note>
249 </para>
250
251 <para>
252 The <filename>meta-yocto</filename> layer inside Poky contains
253 a <filename>conf</filename> directory that has example
254 configuration files.
255 These example files are used as a basis for creating actual
256 configuration files when you source the build environment
257 script <filename>oe-init-build-env</filename>.
258 </para>
259
260 <para>
261 Sourcing the build environment script creates a
262 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
263 if one does not already exist.
264 BitBake uses the Build Directory for all its work during builds.
265 The Build Directory has a <filename>conf</filename> directory that
266 contains default versions of your <filename>local.conf</filename>
267 and <filename>bblayers.conf</filename> configuration files.
268 These default configuration files are created only if versions
269 do not already exist in the Build Directory at the time you
270 source the <filename>oe-init-build-env</filename> script.
271 </para>
272
273 <para>
274 Because the Poky repository is fundamentally an aggregation of
275 existing repositories, some users might be familiar with running
276 the <filename>oe-init-build-env</filename> script in the context of
277 separate OpenEmbedded-Core and BitBake repositories rather than a
278 single Poky repository.
279 This discussion assumes the script is executed from within a cloned
280 or unpacked version of Poky.
281 </para>
282
283 <para>
284 Depending on where the script is sourced, different sub-scripts
285 are called to set up the Build Directory (Yocto or OpenEmbedded).
286 Specifically, the script
287 <filename>scripts/oe-setup-builddir</filename> inside the
288 poky directory sets up the Build Directory and seeds the directory
289 (if necessary) with configuration files appropriate for the
290 Yocto Project development environment.
291 <note>
292 The <filename>scripts/oe-setup-builddir</filename> script
293 uses the <filename>$TEMPLATECONF</filename> variable to
294 determine which sample configuration files to locate.
295 </note>
296 </para>
297
298 <para>
299 The <filename>local.conf</filename> file provides many
300 basic variables that define a build environment.
301 Here is a list of a few.
302 To see the default configurations in a <filename>local.conf</filename>
303 file created by the build environment script, see the
304 <filename>local.conf.sample</filename> in the
305 <filename>meta-yocto</filename> layer:
306 <itemizedlist>
307 <listitem><para><emphasis>Parallelism Options:</emphasis>
308 Controlled by the
309 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
310 and
311 <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
312 variables.</para></listitem>
313 <listitem><para><emphasis>Target Machine Selection:</emphasis>
314 Controlled by the
315 <link linkend='var-MACHINE'><filename>MACHINE</filename></link>
316 variable.</para></listitem>
317 <listitem><para><emphasis>Download Directory:</emphasis>
318 Controlled by the
319 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
320 variable.</para></listitem>
321 <listitem><para><emphasis>Shared State Directory:</emphasis>
322 Controlled by the
323 <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
324 variable.</para></listitem>
325 <listitem><para><emphasis>Build Output:</emphasis>
326 Controlled by the
327 <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
328 variable.</para></listitem>
329 </itemizedlist>
330 <note>
331 Configurations set in the <filename>conf/local.conf</filename>
332 file can also be set in the
333 <filename>conf/site.conf</filename> and
334 <filename>conf/auto.conf</filename> configuration files.
335 </note>
336 </para>
337
338 <para>
339 The <filename>bblayers.conf</filename> file tells BitBake what
340 layers you want considered during the build.
341 By default, the layers listed in this file include layers
342 minimally needed by the build system.
343 However, you must manually add any custom layers you have created.
344 You can find more information on working with the
345 <filename>bblayers.conf</filename> file in the
346 "<ulink url='&YOCTO_DOCS_DEV_URL;#enabling-your-layer'>Enabling Your Layer</ulink>"
347 section in the Yocto Project Development Manual.
348 </para>
349
350 <para>
351 The files <filename>site.conf</filename> and
352 <filename>auto.conf</filename> are not created by the environment
353 initialization script.
354 If you want these configuration files, you must create them
355 yourself:
356 <itemizedlist>
357 <listitem><para><emphasis><filename>site.conf</filename>:</emphasis>
358 You can use the <filename>conf/site.conf</filename>
359 configuration file to configure multiple build directories.
360 For example, suppose you had several build environments and
361 they shared some common features.
362 You can set these default build properties here.
363 A good example is perhaps the level of parallelism you want
364 to use through the
365 <link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link>
366 and
367 <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>
368 variables.</para>
369 <para>One useful scenario for using the
370 <filename>conf/site.conf</filename> file is to extend your
371 <link linkend='var-BBPATH'><filename>BBPATH</filename></link>
372 variable to include the path to a
373 <filename>conf/site.conf</filename>.
374 Then, when BitBake looks for Metadata using
375 <filename>BBPATH</filename>, it finds the
376 <filename>conf/site.conf</filename> file and applies your
377 common configurations found in the file.
378 To override configurations in a particular build directory,
379 alter the similar configurations within that build
380 directory's <filename>conf/local.conf</filename> file.
381 </para></listitem>
382 <listitem><para><emphasis><filename>auto.conf</filename>:</emphasis>
383 This file is not hand-created.
384 Rather, the file is usually created and written to by
385 an autobuilder.
386 The settings put into the file are typically the same as
387 you would find in the <filename>conf/local.conf</filename>
388 or the <filename>conf/site.conf</filename> files.
389 </para></listitem>
390 </itemizedlist>
391 </para>
392
393 <para>
394 You can edit all configuration files to further define
395 any particular build environment.
396 This process is represented by the "User Configuration Edits"
397 box in the figure.
398 </para>
399
400 <para>
401 When you launch your build with the
402 <filename>bitbake &lt;target&gt;</filename> command, BitBake
403 sorts out the configurations to ultimately define your build
404 environment.
405 </para>
406 </section>
407
408 <section id="metadata-machine-configuration-and-policy-configuration">
409 <title>Metadata, Machine Configuration, and Policy Configuration</title>
410
411 <para>
412 The previous section described the user configurations that
413 define the BitBake's global behavior.
414 This section takes a closer look at the layers the build system
415 uses to further control the build.
416 These layers provide Metadata for the software, machine, and
417 policy.
418 </para>
419
420 <para>
421 In general, three types of layer input exist:
422 <itemizedlist>
423 <listitem><para><emphasis>Policy Configuration:</emphasis>
424 Distribution Layers provide top-level or general
425 policies for the image or SDK being built.
426 For example, this layer would dictate whether BitBake
427 produces RPM or IPK packages.</para></listitem>
428 <listitem><para><emphasis>Machine Configuration:</emphasis>
429 Board Support Package (BSP) layers provide machine
430 configurations.
431 This type of information is specific to a particular
432 target architecture.</para></listitem>
433 <listitem><para><emphasis>Metadata:</emphasis>
434 Software layers contain user-supplied recipe files,
435 patches, and append files.
436 </para></listitem>
437 </itemizedlist>
438 </para>
439
440 <para>
441 The following figure shows an expanded representation of the
442 Metadata, Machine Configuration, and Policy Configuration input
443 (layers) boxes of the Yocto Project development environment:
444 </para>
445
446 <para>
447 <imagedata fileref="figures/layer-input.png" align="center" width="8in" depth="7.5in" />
448 </para>
449
450 <para>
451 In general, all layers have a similar structure.
452 They all contain a licensing file
453 (e.g. <filename>COPYING</filename>) if the layer is to be
454 distributed, a <filename>README</filename> file as good practice
455 and especially if the layer is to be distributed, a
456 configuration directory, and recipe directories.
457 </para>
458
459 <para>
460 The Yocto Project has many layers that can be used.
461 You can see a web-interface listing of them on the
462 <ulink url="http://git.yoctoproject.org/">Source Repositories</ulink>
463 page.
464 The layers are shown at the bottom categorized under
465 "Yocto Metadata Layers."
466 These layers are fundamentally a subset of the
467 <ulink url="http://layers.openembedded.org/layerindex/layers/">OpenEmbedded Metadata Index</ulink>,
468 which lists all layers provided by the OpenEmbedded community.
469 <note>
470 Layers exist in the Yocto Project Source Repositories that
471 cannot be found in the OpenEmbedded Metadata Index.
472 These layers are either deprecated or experimental in nature.
473 </note>
474 </para>
475
476 <para>
477 BitBake uses the <filename>conf/bblayers.conf</filename> file,
478 which is part of the user configuration, to find what layers it
479 should be using as part of the build.
480 </para>
481
482 <para>
483 For more information on layers, see the
484 "<ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>Understanding and Creating Layers</ulink>"
485 section in the Yocto Project Development Manual.
486 </para>
487
488 <section id="distro-layer">
489 <title>Distro Layer</title>
490
491 <para>
492 The distribution layer provides policy configurations for your
493 distribution.
494 Best practices dictate that you isolate these types of
495 configurations into their own layer.
496 Settings you provide in
497 <filename>conf/&lt;distro&gt;.conf</filename> override similar
498 settings that BitBake finds in your
499 <filename>conf/local.conf</filename> file in the Build
500 Directory.
501 </para>
502
503 <para>
504 The following list provides some explanation and references
505 for what you typically find in the distribution layer:
506 <itemizedlist>
507 <listitem><para><emphasis>classes:</emphasis>
508 Class files (<filename>.bbclass</filename>) holds
509 common functionality that can be shared among
510 recipes in the distribution.
511 When your recipes inherit a class, they take on the
512 settings and functions for that class.
513 You can read more about class files in the
514 "<link linkend='ref-classes'>Classes</link>" section.
515 </para></listitem>
516 <listitem><para><emphasis>conf:</emphasis>
517 This area holds configuration files for the
518 layer (<filename>conf/layer.conf</filename>),
519 the distribution
520 (<filename>conf/distro/&lt;distro&gt;.conf</filename>),
521 and any distribution-wide include files.
522 </para></listitem>
523 <listitem><para><emphasis>recipes-*:</emphasis>
524 Recipes and append files that affect common
525 functionality across the distribution.
526 This area could include recipes and append files to
527 to add distribution-specific configuration,
528 initialization scripts, custom image recipes,
529 and so forth.</para></listitem>
530 </itemizedlist>
531 </para>
532 </section>
533
534 <section id="bsp-layer">
535 <title>BSP Layer</title>
536
537 <para>
538 The BSP Layer provides machine configurations.
539 Everything in this layer is specific to the machine for which
540 you are building the image or the SDK.
541 A common structure or form is defined for BSP layers.
542 You can learn more about this structure in the
543 <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>.
544 <note>
545 In order for a BSP layer to be considered compliant with the
546 Yocto Project, it must meet some structural requirements.
547 </note>
548 </para>
549
550 <para>
551 The BSP Layer's configuration directory contains
552 configuration files for the machine
553 (<filename>conf/machine/&lt;machine&gt;.conf</filename>) and,
554 of course, the layer (<filename>conf/layer.conf</filename>).
555 </para>
556
557 <para>
558 The remainder of the layer is dedicated to specific recipes
559 by function: <filename>recipes-bsp</filename>,
560 <filename>recipes-core</filename>,
561 <filename>recipes-graphics</filename>, and
562 <filename>recipes-kernel</filename>.
563 Metadata can exist for multiple formfactors, graphics
564 support systems, and so forth.
565 <note>
566 While the figure shows several <filename>recipe-*</filename>
567 directories, not all these directories appear in all
568 BSP layers.
569 </note>
570 </para>
571 </section>
572
573 <section id="software-layer">
574 <title>Software Layer</title>
575
576 <para>
577 The software layer provides the Metadata for additional
578 software packages used during the build.
579 This layer does not include Metadata that is specific to the
580 distribution or the machine, which are found in their
581 respective layers.
582 </para>
583
584 <para>
585 This layer contains any new recipes that your project needs
586 in the form of recipe files.
587 </para>
588 </section>
589 </section>
590
591 <section id="sources-dev-environment">
592 <title>Sources</title>
593
594 <para>
595 In order for the OpenEmbedded build system to create an image or
596 any target, it must be able to access source files.
597 The main
598 <link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment figure</link>
599 represents source files using the "Upstream Project Releases",
600 "Local Projects", and "SCMs (optional)" boxes.
601 The figure represents mirrors, which also play a role in locating
602 source files, with the "Source Mirror(s)" box.
603 </para>
604
605 <para>
606 The method by which source files are ultimately organized is
607 a function of the project.
608 For example, for released software, projects tend to use tarballs
609 or other archived files that can capture the state of a release
610 guaranteeing that it is statically represented.
611 On the other hand, for a project that is more dynamic or
612 experimental in nature, a project might keep source files in a
613 repository controlled by a Source Control Manager (SCM) such as
614 Git.
615 Pulling source from a repository allows you to control
616 the point in the repository (the revision) from which you want to
617 build software.
618 Finally, a combination of the two might exist, which would give the
619 consumer a choice when deciding where to get source files.
620 </para>
621
622 <para>
623 BitBake uses the
624 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
625 variable to point to source files regardless of their location.
626 Each recipe must have a <filename>SRC_URI</filename> variable
627 that points to the source.
628 </para>
629
630 <para>
631 Another area that plays a significant role in where source files
632 comes from is pointed to by the
633 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
634 variable.
635 This area is a cache that can hold previously downloaded source.
636 Judicious use of a <filename>DL_DIR</filename> directory can
637 save the build system a trip across the Internet when looking
638 for files.
639 A good method for using a download directory is to have
640 <filename>DL_DIR</filename> point to an area outside of your
641 Build Directory.
642 Doing so allows you to safely delete the Build Directory
643 if needed without fear of removing any downloaded source file.
644 </para>
645
646 <para>
647 The remainder of this section provides a deeper look into the
648 source files and the mirrors.
649 Here is a more detailed look at the source file area of the
650 base figure:
651 <imagedata fileref="figures/source-input.png" align="center" width="7in" depth="7.5in" />
652 </para>
653
654 <section id='upstream-project-releases'>
655 <title>Upstream Project Releases</title>
656
657 <para>
658 Upstream project releases exist anywhere in the form of an
659 archived file (e.g. tarball or zip file).
660 These files correspond to individual recipes.
661 For example, the figure uses specific releases each for
662 BusyBox, Qt, and Dbus.
663 An archive file can be for any released product that can be
664 built using a recipe.
665 </para>
666 </section>
667
668 <section id='local-projects'>
669 <title>Local Projects</title>
670
671 <para>
672 Local projects are custom bits of software the user provides.
673 These bits reside somewhere local to a project - perhaps
674 a directory into which the user checks in items (e.g.
675 a local directory containing a development source tree
676 used by the group).
677 </para>
678
679 <para>
680 The canonical method through which to include a local project
681 is to use the
682 <link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>
683 class to include local project.
684 You use either the <filename>local.conf</filename> or a
685 recipe's append file to override or set the
686 recipe to point to the local directory on your disk to pull
687 in the whole source tree.
688 </para>
689
690 <para>
691 For information on how to use the
692 <filename>externalsrc.bbclass</filename>, see the
693 "<link linkend='ref-classes-externalsrc'><filename>externalsrc.bbclass</filename></link>"
694 section.
695 </para>
696 </section>
697
698 <section id='scms'>
699 <title>Source Control Managers (Optional)</title>
700
701 <para>
702 Another place the build system can get source files from is
703 through an SCM such as Git or Subversion.
704 In this case, a repository is cloned or checked out.
705 The <filename>do_fetch</filename> task inside BitBake uses
706 the <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
707 variable and the argument's prefix to determine the correct
708 fetcher module.
709 </para>
710
711 <para>
712 When fetching a repository, BitBake uses the
713 <link linkend='var-SRCREV'><filename>SRCREV</filename></link>
714 variable to determine the specific revision from which to
715 build.
716 </para>
717 </section>
718
719 <section id='source-mirrors'>
720 <title>Source Mirror(s)</title>
721
722 <para>
723 Two kinds of mirrors exist: pre-mirrors and regular mirrors.
724 The <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link>
725 and
726 <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link>
727 variables point to these, respectively.
728 BitBake checks pre-mirrors before looking upstream for any
729 source files.
730 Pre-mirrors are appropriate when you have a shared directory
731 that is not a directory defined by the
732 <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
733 variable.
734 A Pre-mirror typically points to a shared directory that is
735 local to your organization.
736 </para>
737
738 <para>
739 Regular mirrors can be any site across the Internet that is
740 used as an alternative location for source code should the
741 primary site not be functioning for some reason or another.
742 </para>
743 </section>
744 </section>
745
746 <section id='bitbake-dev-environment'>
747 <title>BitBake</title>
748
749 <para>
750 The OpenEmbedded build system uses BitBake to produce images.
751 You can see from the
752 <link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
753 figure, the BitBake area consists of several functional areas.
754 This section takes a closer look at each of those areas.
755 </para>
756
757 <section id='source-fetching-dev-environment'>
758 <title>Source Fetching</title>
759
760 <para>
761 The first stages of building a recipe are to fetch and unpack
762 the source code:
763 <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="4in" />
764 </para>
765
766 <para>
767 The <filename>do_fetch</filename> and
768 <filename>do_unpack</filename> tasks fetch the source files
769 and unpack them into a working directory.
770 By default, everything is accomplished in the
771 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>,
772 which has a defined structure.
773 For additional general information on the Build Directory,
774 see the
775 "<link linkend='structure-core-build'><filename>build/</filename></link>"
776 section.
777 </para>
778
779 <para>
780 Unpacked source source files are pointed to by the
781 <link linkend='var-S'><filename>S</filename></link> variable.
782 Each recipe has an area in the Build Directory where the
783 unpacked source code resides.
784 The name of directory for any given recipe is defined from
785 several different variables.
786 You can see the variables that define these directories
787 by looking at the figure:
788 <itemizedlist>
789 <listitem><para><link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>
790 </para></listitem>
791 <listitem><para><link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>
792 </para></listitem>
793 <listitem><para><link linkend='var-TARGET_OS'><filename>TARGET_OS</filename></link>
794 </para></listitem>
795 <listitem><para><link linkend='var-PN'><filename>PN</filename></link>
796 </para></listitem>
797 <listitem><para><link linkend='var-PV'><filename>PV</filename></link>
798 </para></listitem>
799 <listitem><para><link linkend='var-PR'><filename>PR</filename></link>
800 </para></listitem>
801 <listitem><para><link linkend='var-WORKDIR'><filename>WORKDIR</filename></link>
802 </para></listitem>
803 <listitem><para><link linkend='var-S'><filename>S</filename></link>
804 </para></listitem>
805 </itemizedlist>
806 </para>
807
808 <para>
809 Briefly, the <filename>S</filename> directory contains the
810 unpacked source files for a recipe.
811 The <filename>WORKDIR</filename> directory is where all the
812 building goes on for a given recipe.
813 </para>
814 </section>
815
816 <section id='patching-dev-environment'>
817 <title>Patching</title>
818
819 <para>
820 Once source code is fetched and unpacked, BitBake locates
821 patch files and applies them to the source files:
822 <imagedata fileref="figures/patching.png" align="center" width="6in" depth="5in" />
823 </para>
824
825 <para>
826 The <filename>do_patch</filename> task processes recipes by
827 using the
828 <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>
829 variable to locate applicable patch files, which by default
830 are <filename>*.patch</filename> or
831 <filename>*.diff</filename> files, or any file if
832 "apply=yes" is specified for the file in
833 <filename>SRC_URI</filename>.
834 </para>
835
836 <para>
837 BitBake finds and applies multiple patches for a single recipe
838 in the order in which it finds the patches.
839 Patches are applied to the recipe's source files located in the
840 <link linkend='var-S'><filename>S</filename></link> directory.
841 </para>
842
843 <para>
844 For more information on how the source directories are
845 created, see the
846 "<link linkend='source-fetching-dev-environment'>Source Fetching</link>"
847 section.
848 </para>
849 </section>
850
851 <section id='configuration-and-compilation-dev-environment'>
852 <title>Configuration and Compilation</title>
853
854 <para>
855 After source code is patched, BitBake executes tasks that
856 configure and compile the source code:
857 <imagedata fileref="figures/configuration-compile-autoreconf.png" align="center" width="7in" depth="5in" />
858 </para>
859
860 <para>
861 This step in the build process consists of three tasks:
862 <itemizedlist>
863 <listitem><para><emphasis><filename>do_configure</filename>:</emphasis>
864 This task configures the source by enabling and
865 disabling any build-time and configuration options for
866 the software being built.
867 Configurations can come from the recipe itself as well
868 as from an inherited class.
869 Additionally, the software itself might configure itself
870 depending on the target for which it is being built.
871 </para>
872
873 <para>The configurations handled by the
874 <filename>do_configure</filename> task are specific
875 to source code configuration for the source code
876 being built by the recipe.</para>
877
878 <para>If you are using
879 <link linkend='ref-classes-autotools'><filename>autotools.bbclass</filename></link>,
880 you can add additional configuration options by using
881 the <link linkend='var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></link>
882 variable.
883 For information on how this variable works within
884 that class, see the
885 <filename>meta/classes/autotools.bbclass</filename>.
886 </para></listitem>
887 <listitem><para><emphasis><filename>do_compile</filename>:</emphasis>
888 Once a configuration task has been satisfied, BitBake
889 compiles the source using the
890 <filename>do_compile</filename> task.
891 Compilation occurs in the directory pointed to by the
892 <link linkend='var-B'><filename>B</filename></link>
893 variable.
894 Realize that the <filename>B</filename> directory, by
895 default, is the same as the
896 <link linkend='var-S'><filename>S</filename></link>
897 directory.</para></listitem>
898 <listitem><para><emphasis><filename>do_install</filename>:</emphasis>
899 Once compilation is done, BitBake executes the
900 <filename>do_install</filename> task.
901 This task copies files from the <filename>B</filename>
902 directory and places them in a holding area pointed to
903 by the
904 <link linkend='var-D'><filename>D</filename></link>
905 variable.</para></listitem>
906 </itemizedlist>
907 </para>
908 </section>
909
910 <section id='package-splitting-dev-environment'>
911 <title>Package Splitting</title>
912
913 <para>
914 After source code is configured and compiled, the
915 OpenEmbedded build system analyzes
916 the results and splits the output into packages:
917 <imagedata fileref="figures/analysis-for-package-splitting.png" align="center" width="7in" depth="7in" />
918 </para>
919
920 <para>
921 The <filename>do_package</filename> and
922 <filename>do_packagedata</filename> tasks combine to analyze
923 the files found in the
924 <link linkend='var-D'><filename>D</filename></link> directory
925 and split them into subsets based on available packages and
926 files.
927 The analyzing process involves the following as well as other
928 items: splitting out debugging symbols,
929 looking at shared library dependencies between packages,
930 and looking at package relationships.
931 The <filename>do_packagedata</filename> task creates package
932 metadata based on the analysis such that the
933 OpenEmbedded build system can generate the final packages.
934 Working, staged, and intermediate results of the analysis
935 and package splitting process use these areas:
936 <itemizedlist>
937 <listitem><para><link linkend='var-PKGD'><filename>PKGD</filename></link>
938 </para></listitem>
939 <listitem><para><link linkend='var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></link>
940 </para></listitem>
941 <listitem><para><link linkend='var-PKGDESTWORK'><filename>PKGDESTWORK</filename></link>
942 </para></listitem>
943 <listitem><para><link linkend='var-PKGDEST'><filename>PKGDEST</filename></link>
944 </para></listitem>
945 </itemizedlist>
946 The <link linkend='var-FILES'><filename>FILES</filename></link>
947 variable defines the files that go into each package in
948 <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>.
949 If you want details on how this is accomplished, you can
950 look at
951 <link linkend='ref-classes-package'><filename>package.bbclass</filename></link>.
952 </para>
953
954 <para>
955 Depending on the type of packages being created (RPM, DEB, or
956 IPK), the <filename>do_package_write_*</filename> task
957 creates the actual packages and places them in the
958 Package Feed area, which is
959 <filename>${TMPDIR}/deploy</filename>.
960 You can see the
961 "<link linkend='package-feeds-dev-environment'>Package Feeds</link>"
962 section for more detail on that part of the build process.
963 <note>
964 Support for creating feeds directly from the
965 <filename>deploy/*</filename> directories does not exist.
966 Creating such feeds usually requires some kind of feed
967 maintenance mechanism that would upload the new packages
968 into an official package feed (e.g. the
969 Ångström distribution).
970 This functionality is highly distribution-specific
971 and thus is not provided out of the box.
972 </note>
973 </para>
974 </section>
975 </section>
976
977 <section id="package-feeds-dev-environment">
978 <title>Package Feeds</title>
979
980 <para>
981 When the OpenEmbedded build system generates an image or an SDK,
982 it gets the packages from a package feed area located in the
983 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
984 The main
985 <link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
986 figure shows this package feeds area in the upper-right corner.
987 </para>
988
989 <para>
990 This section looks a little closer into the package feeds area used
991 by the build system.
992 Here is a more detailed look at the area:
993 <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
994 </para>
995
996 <para>
997 Package feeds are an intermediary step in the build process.
998 BitBake generates packages whose type is defined by the
999 <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
1000 variable.
1001 Before placing the packages into package feeds,
1002 the build process validates them with generated output quality
1003 assurance checks through the
1004 <link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>
1005 class.
1006 </para>
1007
1008 <para>
1009 The package feed area resides in
1010 <filename>tmp/deploy</filename> of the Build Directory.
1011 Folders are created that correspond to the package type
1012 (IPK, DEB, or RPM) created.
1013 Further organization is derived through the value of the
1014 <link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>
1015 variable for each package.
1016 For example, packages can exist for the i586 or qemux86
1017 architectures.
1018 The package files themselves reside within the appropriate
1019 architecture folder.
1020 </para>
1021
1022 <para>
1023 BitBake uses the <filename>do_package_write_*</filename> task to
1024 place generated packages into the package holding area (e.g.
1025 <filename>do_package_write_ipk</filename> for IPK packages).
1026 </para>
1027 </section>
1028
1029 <section id='images-dev-environment'>
1030 <title>Images</title>
1031
1032 <para>
1033 The images produced by the OpenEmbedded build system
1034 are compressed forms of the
1035 root filesystems that are ready to boot on a target device.
1036 You can see from the main
1037 <link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
1038 figure that BitBake output in part consists of images.
1039 This section is going to look more closely at this output:
1040 <imagedata fileref="figures/images.png" align="center" width="6in" depth="5in" />
1041 </para>
1042
1043 <para>
1044 For a list of example images that the Yocto Project provides,
1045 the
1046 "<link linkend='ref-images'>Images</link>" chapter.
1047 </para>
1048
1049 <para>
1050 Images are written out to the
1051 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
1052 inside the <filename>deploy/images</filename> folder as shown
1053 in the figure.
1054 This folder contains any files expected to be loaded on the
1055 target device.
1056 The
1057 <link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>
1058 variable points to the <filename>deploy</filename> directory.
1059 <itemizedlist>
1060 <listitem><para><filename>&lt;kernel-image&gt;</filename>:
1061 A kernel binary file.
1062 The <link linkend='var-KERNEL_IMAGETYPE'><filename>KERNEL_IMAGETYPE</filename></link>
1063 variable setting determines the naming scheme for the
1064 kernel image file.
1065 Depending on that variable, the file could begin with
1066 a variety of naming strings.
1067 The <filename>deploy/images</filename> directory can
1068 contain multiple image files.</para></listitem>
1069 <listitem><para><filename>&lt;root-filesystem-image&gt;</filename>:
1070 Root filesystems for the target device (e.g.
1071 <filename>*.ext3</filename> or <filename>*.bz2</filename>
1072 files).
1073 The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
1074 variable setting determines the root filesystem image
1075 type.
1076 The <filename>deploy/images</filename> directory can
1077 contain multiple root filesystems.</para></listitem>
1078 <listitem><para><filename>&lt;kernel-modules&gt;</filename>:
1079 Tarballs that contain all the modules built for the kernel.
1080 Kernel module tarballs exist for legacy purposes and
1081 can be suppressed by setting the
1082 <link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link>
1083 variable to "0".
1084 The <filename>deploy/images</filename> directory can
1085 contain multiple kernel module tarballs.
1086 </para></listitem>
1087 <listitem><para><filename>&lt;bootloaders&gt;</filename>:
1088 Bootloaders supporting the image, if applicable to the
1089 target machine.
1090 The <filename>deploy/images</filename> directory can
1091 contain multiple bootloaders.
1092 </para></listitem>
1093 <listitem><para><filename>&lt;symlinks&gt;</filename>:
1094 The <filename>deploy/images</filename> folder contains
1095 a symbolic link that points to the most recently built file
1096 for each machine.
1097 These links might be useful for external scripts that
1098 need to obtain the latest version of each file.
1099 </para></listitem>
1100 </itemizedlist>
1101 </para>
1102 </section>
1103
1104 <section id='sdk-dev-environment'>
1105 <title>Application Development SDK</title>
1106
1107 <para>
1108 In the overview figure of the
1109 <link linkend='a-closer-look-at-the-yocto-project-development-environment'>Yocto Project Development Environment</link>
1110 the output labeled "Application Development SDK" represents an
1111 SDK.
1112 This section is going to take a closer look at this output:
1113 <imagedata fileref="figures/sdk.png" align="center" width="5in" depth="4in" />
1114 </para>
1115
1116 <para>
1117 The specific form of this output is a self-extracting
1118 SDK installer (<filename>*.sh</filename>) that, when run,
1119 installs the SDK, which consists of a cross-development
1120 toolchain, a set of libraries and headers, and an SDK
1121 environment setup script.
1122 Running this installer essentially sets up your
1123 cross-development environment.
1124 You can think of the cross-toolchain as the "host"
1125 part because it runs on the SDK machine.
1126 You can think of the libraries and headers as the "target"
1127 part because they are built for the target hardware.
1128 The setup script is added so that you can initialize the
1129 environment before using the tools.
1130 </para>
1131
1132 <note>
1133 <para>
1134 The Yocto Project supports several methods by which you can
1135 set up this cross-development environment.
1136 These methods include downloading pre-built SDK installers,
1137 building and installing your own SDK installer, or running
1138 an Application Development Toolkit (ADT) installer to
1139 install not just cross-development toolchains
1140 but also additional tools to help in this type of
1141 development.
1142 </para>
1143
1144 <para>
1145 For background information on cross-development toolchains
1146 in the Yocto Project development environment, see the
1147 "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
1148 section.
1149 For information on setting up a cross-development
1150 environment, see the
1151 "<ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>Installing the ADT and Toolchains</ulink>"
1152 section in the Yocto Project Application Developer's Guide.
1153 </para>
1154 </note>
1155
1156 <para>
1157 Once built, the SDK installers are written out to the
1158 <filename>deploy/sdk</filename> folder inside the
1159 <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
1160 as shown in the figure at the beginning of this section.
1161 Several variables exist that help configure these files:
1162 <itemizedlist>
1163 <listitem><para><link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>:
1164 Points to the <filename>deploy</filename>
1165 directory.</para></listitem>
1166 <listitem><para><link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>:
1167 Specifies the architecture of the machine
1168 on which the cross-development tools are run to
1169 create packages for the target hardware.
1170 </para></listitem>
1171 <listitem><para><link linkend='var-SDKIMAGE_FEATURES'><filename>SDKIMAGE_FEATURES</filename></link>:
1172 Lists the features to include in the "target" part
1173 of the SDK.
1174 </para></listitem>
1175 <listitem><para><link linkend='var-TOOLCHAIN_HOST_TASK'><filename>TOOLCHAIN_HOST_TASK</filename></link>:
1176 Lists packages that make up the host
1177 part of the SDK (i.e. the part that runs on
1178 the <filename>SDKMACHINE</filename>).
1179 When you use
1180 <filename>bitbake -c populate_sdk &lt;imagename&gt;</filename>
1181 to create the SDK, a set of default packages
1182 apply.
1183 This variable allows you to add more packages.
1184 </para></listitem>
1185 <listitem><para><link linkend='var-TOOLCHAIN_TARGET_TASK'><filename>TOOLCHAIN_TARGET_TASK</filename></link>:
1186 Lists packages that make up the target part
1187 of the SDK (i.e. the part built for the
1188 target hardware).
1189 </para></listitem>
1190 </itemizedlist>
1191 </para>
1192 </section>
1193</section>
1194
1195<section id="cross-development-toolchain-generation"> 156<section id="cross-development-toolchain-generation">
1196 <title>Cross-Development Toolchain Generation</title> 157 <title>Cross-Development Toolchain Generation</title>
1197 158