summaryrefslogtreecommitdiffstats
path: root/documentation/poky-ref-manual/technical-details.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/poky-ref-manual/technical-details.xml')
-rw-r--r--documentation/poky-ref-manual/technical-details.xml179
1 files changed, 77 insertions, 102 deletions
diff --git a/documentation/poky-ref-manual/technical-details.xml b/documentation/poky-ref-manual/technical-details.xml
index 1657431495..dfb5b726c0 100644
--- a/documentation/poky-ref-manual/technical-details.xml
+++ b/documentation/poky-ref-manual/technical-details.xml
@@ -493,111 +493,86 @@
493 493
494 <para> 494 <para>
495 The code in the Yocto Project that supports incremental builds is not 495 The code in the Yocto Project that supports incremental builds is not
496 simple code. 496 simple code.
497 Consequently, when things go wrong, debugging needs to be straightforward. 497 This section presents some tips and tricks that help you work around
498 Because of this, the Yocto Project team included strong debugging 498 issues related to shared state code.
499 tools. 499 </para>
500 </para> 500
501 501 <section id='debugging'>
502 <para> 502 <title>Debugging</title>
503 First, whenever a shared state package is written out, so is a 503
504 corresponding <filename>.siginfo</filename> file. 504 <para>
505 This practice results in a pickled python database of all 505 When things go wrong, debugging needs to be straightforward.
506 the metadata that went into creating the hash for a given shared state 506 Because of this, the Yocto Project team included strong debugging
507 package. 507 tools:
508 </para> 508 <itemizedlist>
509 509 <listitem><para>Whenever a shared state package is written out, so is a
510 <para> 510 corresponding <filename>.siginfo</filename> file.
511 Second, if BitBake is run with the <filename>--dump-signatures</filename> 511 This practice results in a pickled python database of all
512 (or <filename>-S</filename>) option, BitBake dumps out 512 the metadata that went into creating the hash for a given shared state
513 <filename>.siginfo</filename> files in 513 package.</para></listitem>
514 the stamp directory for every task it would have executed instead of 514 <listitem><para>If BitBake is run with the <filename>--dump-signatures</filename>
515 building the target package specified. 515 (or <filename>-S</filename>) option, BitBake dumps out
516 </para> 516 <filename>.siginfo</filename> files in
517 517 the stamp directory for every task it would have executed instead of
518 <para> 518 building the target package specified.</para></listitem>
519 Finally, there is a <filename>bitbake-diffsigs</filename> command that 519 <listitem><para>There is a <filename>bitbake-diffsigs</filename> command that
520 can process these <filename>.siginfo</filename> files. 520 can process these <filename>.siginfo</filename> files.
521 If one file is specified, it will dump out the dependency 521 If one file is specified, it will dump out the dependency
522 information in the file. 522 information in the file.
523 If two files are specified, it will compare the 523 If two files are specified, it will compare the
524 two files and dump out the differences between the two. 524 two files and dump out the differences between the two.
525 This allows the question of "What changed between X and Y?" to be 525 This allows the question of "What changed between X and Y?" to be
526 answered easily. 526 answered easily.</para></listitem>
527 </para> 527 </itemizedlist>
528 </para>
529 </section>
530
531 <section id='invalidating-shared-state'>
532 <title>Invalidating Shared State</title>
533
534 <para>
535 The Yocto Project shared state code uses checksums and shared state memory
536 cache to avoide building tasks that don't need built.
537 As with all schemes, this one has some drawbacks.
538 It is possible that you could make implicit changes that are not factored into the checksum
539 calculation, but do affect a task's output.
540 A good example is perhaps when a tool changes its output.
541 Let's say that the output of <filename>rpmdeps</filename> needed to change.
542 The result of the change should be that all the "package", "package_write_rpm",
543 and "package_deploy-rpm" shared state cache items would become invalid.
544 But, because this is a change that is external to the code and therefore implicit,
545 the associated shared state cache items do not become invalidated.
546 In this case, the build process would use the cache items rather than running the
547 task again.
548 Obviously, these types of implicit changes can cause problems.
549 </para>
550
551 <para>
552 To avoid these problems during the build, you need to understand the effects of any
553 change you make.
554 Note that any changes you make directly to a function automatically are factored into
555 the checksum calculation and thus, will invalidate the associated area of sstate cache.
556 You need to be aware of any implicit changes that are not obvious changes to the
557 code and could affect the output of a given task.
558 Once you are aware of such a change, you can take steps to invalidate the cache
559 and force the task to run.
560 The step to take is as simple as changing a function's comments in the source code.
561 For example, to invalidate package shared state files, change the comment statments
562 of <filename>do_package</filename> or the comments of one of the functions it calls.
563 The change is purely cosmetic, but it causes the checksum to be recalculated and
564 forces the task to be run again.
565 </para>
566
567 <note>
568 For an example of a commit that makes a cosmetic change to invalidate an sstate,
569 see this
570 <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
571 </note>
572 </section>
528 </section> 573 </section>
529</section> 574</section>
530 575
531<!--
532
533 <para>
534 The Yocto Project build process uses a shared state caching scheme to avoid having to
535 rebuild software when it is not necessary.
536 Because the build time for a Yocto image can be significant, it is helpful to try and
537 determine what really needs built and what can be skipped given a particular project's
538 build process.
539 </para>
540
541 <para>
542 The scheme that the Yocto Project uses involves checksum generation and comparison for
543 a task's inputs.
544 The scheme also employs an area of memory called the shared state cache that is
545 pointed to by the <filename>SSTATE_DIR</filename> variable.
546 This area contains task output generated from a previous build.
547 If a given task's checksum matches the checksum of a previous build for the same
548 task, the build process uses the state of the cache rather than rerunning that
549 task.
550 </para>
551
552 <para>
553 The previous paragraph is a simplistic explanation of how the build process
554 uses checksums and shared state memory cache to avoide building tasks that
555 don't need built.
556 If you want a bit more explanation on the topic,
557 see "<ulink url='https://lists.yoctoproject.org/pipermail/yocto/2011-March/003366.html'>Shared
558 State - What does it mean and why should I care?</ulink>" from the Yocto
559 Project discussion archives.
560 </para>
561
562 <para>
563 As with all schemes, this one has some drawbacks.
564 It is possible that you could make implicit changes that are not factored into the checksum
565 calculation, but do affect a task's output.
566 A good example is perhaps when a tool changes its output.
567 Let's say that the output of <filename>rpmdeps</filename> needed to change.
568 The result of the change should be that all the "package", "package_write_rpm",
569 and "package_deploy-rpm" sstate-cache items would become invalid.
570 But, because this is a change that is external to the code and therefore implicit,
571 the associated sstate-cache items do not become invalidated.
572 In this case, the build process would use the cache items rather than running the
573 task again.
574 Obviously, these types of implicit changes can cause problems.
575 </para>
576
577 <para>
578 To avoid these problems during the build, you need to understand the effects of any
579 change you make.
580 Note that any changes you make directly to a function automatically are factored into
581 the checksum calculation and thus, will invalidate the associated area of sstate cache.
582 You need to be aware of any implicit changes that are not obvious changes to the
583 code and could affect the output of a given task.
584 Once you are aware of such a change, you can take steps to invalidate the cache
585 and force the task to run.
586 The step to take is as simple as changing a function's comments in the source code.
587 For example, to invalidate package sstate files, change the comment statments
588 of <filename>do_package</filename> or one of the functions it calls.
589 The change is purely cosmetic, but it causes the checksum to be recalculated and
590 forces the task to be run again.
591 </para>
592
593 <note>
594 For an example of a commit that makes a cosmetic change to invalidate an sstate,
595 see this
596 <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
597 </note>
598</section>
599-->
600
601</chapter> 576</chapter>
602<!-- 577<!--
603vim: expandtab tw=80 ts=4 578vim: expandtab tw=80 ts=4