ref-manual: Suggested fleshing out of the sigdata/siginfo documentation

Fixes [YOCTO #10141]

Provided several fixes to address this situation:

 * Renamed "Debugging Build Failures" to "Debugging Tools and
   Techniques" as it fit better the subsections.

 * Renamed "Viewing Dependencies" to "Viewing Dependencies
   Between Recipes and Tasks" as it fit better the description.

 * Added a new "Viewing Task Variable Dependencies" section
   to describe how sigdata and siginfo stuff can be used.

 * Replaced the contents of "4.3.4.1 Debugging" with a shorter
   bit that now references into the new section on veiwing
   task variable dependencies.

(From yocto-docs rev: 539d76366055bed74ccc926519e969324cac470d)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

2 files changed, 130 insertions, 54 deletions

diff --git a/documentation/ref-manual/technical-details.xml b/documentation/ref-manual/technical-details.xml
index 5ef764c8e0..6bad547e50 100644
--- a/documentation/ref-manual/technical-details.xml
+++ b/documentation/ref-manual/technical-details.xml
@@ -901,56 +901,15 @@
             <title>Debugging</title>
 
             <para>
-                When things go wrong, debugging needs to be straightforward.
-                Because of this, the Yocto Project includes strong debugging
-                tools:
-                <itemizedlist>
-                    <listitem><para>Whenever a shared state package is written
-                        out into the
-                        <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>,
-                        a corresponding <filename>.siginfo</filename> file is
-                        also written.
-                        This file contains a pickled Python database of all
-                        the Metadata that went into creating the hash for a
-                        given shared state package.
-                        Whenever a stamp is written into the stamp directory
-                        <link linkend='var-STAMP'><filename>STAMP</filename></link>,
-                        a corresponding <filename>.sigdata</filename> file
-                        is created that contains the same hash data that
-                        represented the executed task.
-                        </para></listitem>
-                    <listitem><para>You can use BitBake to dump out the
-                        signature construction information without executing
-                        tasks by using either of the following BitBake
-                        command-line options:
-                        <literallayout class='monospaced'>
-     &dash;&dash;dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable>
-     -S <replaceable>SIGNATURE_HANDLER</replaceable>
-                        </literallayout>
-                        <note>
-                            Two common values for
-                            <replaceable>SIGNATURE_HANDLER</replaceable> are
-                            "none" and "printdiff" to only dump the signature
-                            or to compare the dumped signature with the
-                            cached one, respectively.
-                        </note>
-                        Using BitBake with either of these options causes
-                        BitBake to dump out <filename>.sigdata</filename> files
-                        in the stamp directory for every task it would have
-                        executed instead of building the specified target
-                        package.
-                        </para></listitem>
-                    <listitem><para>There is a
-                        <filename>bitbake-diffsigs</filename> command that
-                        can process <filename>.sigdata</filename> and
-                        <filename>.siginfo</filename> files.
-                        If you specify one of these files, BitBake dumps out
-                        the dependency information in the file.
-                        If you specify two files, BitBake compares the two
-                        files and dumps out the differences between the two.
-                        This more easily helps answer the question of "What
-                        changed between X and Y?"</para></listitem>
-                </itemizedlist>
+                Seeing what metadata went into creating the input signature
+                of a shared state (sstate) task can be a useful debugging aid.
+                This information is available in signature information
+                (<filename>siginfo</filename>) files in
+                <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>.
+                For information on how to view and interpret information in
+                <filename>siginfo</filename> files, see the
+                "<link linkend='usingpoky-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>"
+                section.
             </para>
         </section>
 
diff --git a/documentation/ref-manual/usingpoky.xml b/documentation/ref-manual/usingpoky.xml
index 15e1d47301..574de7b4b7 100644
--- a/documentation/ref-manual/usingpoky.xml
+++ b/documentation/ref-manual/usingpoky.xml
@@ -119,8 +119,8 @@
     </para>
 </section>
 
-<section id='usingpoky-debugging'>
-    <title>Debugging Build Failures</title>
+<section id='usingpoky-debugging-tools-and-techniques'>
+    <title>Debugging Tools and Techniques</title>
 
     <para>
         The exact method for debugging build failures depends on the nature of
@@ -247,8 +247,8 @@
         </para>
     </section>
 
-    <section id='usingpoky-viewing-dependencies'>
-        <title>Viewing Dependencies</title>
+    <section id='usingpoky-viewing-dependencies-between-recipes-and-tasks'>
+        <title>Viewing Dependencies Between Recipes and Tasks</title>
 
         <para>
             Sometimes it can be hard to see why BitBake wants to build other
@@ -348,6 +348,123 @@
         </para>
     </section>
 
+    <section id='usingpoky-viewing-task-variable-dependencies'>
+        <title>Viewing Task Variable Dependencies</title>
+
+        <para>
+            As mentioned in the
+            "<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>"
+            section of the BitBake User Manual, BitBake tries to automatically
+            determine what variables a task depends on so that it can rerun
+            the task if any values of the variables change.
+            This determination is usually reliable.
+            However, if you do things like construct variable names at runtime,
+            then you might have to manually declare dependencies on those
+            variables using <filename>vardeps</filename> as described in the
+            "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>"
+            section of the BitBake User Manual.
+        </para>
+
+        <para>
+            If you are unsure whether a variable dependency is being picked up
+            automatically for a given task, you can list the variable
+            dependencies BitBake has determined by doing the following:
+            <orderedlist>
+                <listitem><para>
+                    Build the recipe containing the task:
+                    <literallayout class='monospaced'>
+     $ bitbake <replaceable>recipename</replaceable>
+                    </literallayout>
+                    </para></listitem>
+                <listitem><para>
+                    Inside the
+                    <link linkend='var-STAMPS_DIR'><filename>STAMPS_DIR</filename></link>
+                    directory, find the signature data
+                    (<filename>sigdata</filename>) file that corresponds to the
+                    task.
+                    The <filename>sigdata</filename> files contain a pickled
+                    Python database of all the metadata that went into creating
+                    the input checksum for the task.
+                    As an example, for the
+                    <link linkend='ref-tasks-fetch'><filename>do_fetch</filename></link>
+                    task of the <filename>db</filename> recipe, the
+                    <filename>sigdata</filename> file might be found in the
+                    following location:
+                    <literallayout class='monospaced'>
+     ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
+                    </literallayout>
+                    For tasks that are accelerated through the shared state
+                    (<link linkend='shared-state-cache'>sstate</link>)
+                    cache, an additional <filename>siginfo</filename> file is
+                    written into
+                    <link linkend='var-SSTATE_DIR'><filename>SSTATE_DIR</filename></link>
+                    along with the cached task output.
+                    The <filename>siginfo</filename> files contain exactly the
+                    same information as <filename>sigdata</filename> files.
+                    </para></listitem>
+                <listitem><para>
+                    Run <filename>bitbake-dumpsigs</filename> on the
+                    <filename>sigdata</filename> or
+                    <filename>siginfo</filename> file.
+                    Here is an example:
+                    <literallayout class='monospaced'>
+     $ bitbake-dumpsigs ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1
+                    </literallayout>
+                    In the output of the above command, you will find a line
+                    like the following, which lists all the (inferred) variable
+                    dependencies for the task.
+                    This list also includes indirect dependencies from
+                    variables depending on other variables, recursively.
+                    <literallayout class='monospaced'>
+     Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch']
+                    </literallayout>
+                    <note>
+                        Functions (e.g. <filename>base_do_fetch</filename>)
+                        also count as variable dependencies.
+                        These functions in turn depend on the variables they
+                        reference.
+                    </note>
+                    The output of <filename>bitbake-dumpsigs</filename> also includes
+                    the value each variable had, a list of dependencies for each
+                    variable, and
+                    <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink>
+                    information.
+                    </para></listitem>
+            </orderedlist>
+        </para>
+
+        <para>
+            There is also a <filename>bitbake-diffsigs</filename> command for
+            comparing two <filename>siginfo</filename> or
+            <filename>sigdata</filename> files.
+            This command can be helpful when trying to figure out what changed
+            between two versions of a task.
+            If you call <filename>bitbake-diffsigs</filename> with just one
+            file, the command behaves like
+            <filename>bitbake-dumpsigs</filename>.
+        </para>
+
+        <para>
+            You can also use BitBake to dump out the signature construction
+            information without executing tasks by using either of the
+            following BitBake command-line options:
+            <literallayout class='monospaced'>
+     &dash;&dash;dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable>
+     -S <replaceable>SIGNATURE_HANDLER</replaceable>
+            </literallayout>
+            <note>
+                Two common values for
+                <replaceable>SIGNATURE_HANDLER</replaceable> are "none" and
+                "printdiff", which dump only the signature or compare the
+                dumped signature with the cached one, respectively.
+            </note>
+            Using BitBake with either of these options causes BitBake to dump
+            out <filename>sigdata</filename> files in the
+            <filename>stamps</filename> directory for every task it would have
+            executed instead of building the specified target package.
+        </para>
+    </section>
+
     <section id='usingpoky-debugging-taskrunning'>
         <title>Running Specific Tasks</title>