toaster-manual: Added new Toaster functionality descriptions.

Flow and usage information has changed.  Changes here reflect
that from the Toaster Team.

(From yocto-docs rev: 9774dec014b6d13623988b0d7c050c9e6deaad6f)

Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

3 files changed, 479 insertions, 11 deletions

diff --git a/documentation/toaster-manual/toaster-manual-intro.xml b/documentation/toaster-manual/toaster-manual-intro.xml
index ad9e08b50d..9f4c38b2d4 100644
--- a/documentation/toaster-manual/toaster-manual-intro.xml
+++ b/documentation/toaster-manual/toaster-manual-intro.xml
@@ -39,7 +39,7 @@
         <para>
             You can use Toaster in Analysis Mode or Build Mode:
             <itemizedlist>
-                <listitem><para><emphasis>Analysis Mode:</emphasis>
+                <listitem><para id='toaster-analysis-mode'><emphasis>Analysis Mode:</emphasis>
                     In Analysis Mode, you can record builds and statistics.
                     In this Mode, you directly access the
                     <filename>bitbake</filename> command, which you then use to
@@ -82,7 +82,7 @@
                             </para></listitem>
                     </itemizedlist>
                     </para></listitem>
-                <listitem><para><emphasis>Build Mode:</emphasis>
+                <listitem><para id='toaster-build-mode'><emphasis>Build Mode:</emphasis>
                     In Build Mode, Toaster handles the build configuration,
                     scheduling and execution.
                     In this mode, all your interaction with the build system
diff --git a/documentation/toaster-manual/toaster-manual-setup-and-use.xml b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
index 57a4343bc4..eb2de9b673 100644
--- a/documentation/toaster-manual/toaster-manual-setup-and-use.xml
+++ b/documentation/toaster-manual/toaster-manual-setup-and-use.xml
@@ -6,9 +6,469 @@
 
 <title>Setting Up and Using Toaster</title>
 
-    <section id='using-toaster-in-analysis-mode'>
+    <section id='starting-toaster-for-local-development'>
+        <title>Starting Toaster for Local Development</title>
+
+        <para>
+            Once you have set up the Yocto Project and installed the
+            Toaster system dependencies as described in
+            "<link linkend='toaster-manual-start'>Preparing to Use Toaster</link>",
+            you are ready to start Toaster.
+        </para>
+
+        <para>
+            If you want to configure and start your builds using the
+            Toaster web interface
+            (i.e. "<link linkend='toaster-build-mode'>Build Mode</link>"),
+            navigate to the root of your
+            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>
+            (e.g. <filename>poky</filename>):
+            <literallayout class='monospaced'>
+     $ cd poky
+            </literallayout>
+            Next, start Toaster:
+            <literallayout class='monospaced'>
+     $ bitbake/bin/toaster
+            </literallayout>
+            Open your favourite browser and enter the following:
+            <literallayout class='monospaced'>
+     http://127.0.0.1:8000
+            </literallayout>
+            If you would rather configure and start your builds
+            using the command line
+            (i.e. <link linkend='toaster-analysis-mode'>Analysis Mode</link>),
+            you can get Toaster to "listen"
+            to your builds and collect information about them.
+            To do that, navigate to the root of your Source Directory:
+            <literallayout class='monospaced'>
+     $ cd poky
+            </literallayout>
+            Once in that directory, source the build environment script:
+            <literallayout class='monospaced'>
+     $ source oe-init-build-env
+            </literallayout>
+            Next, from the build directory (e.g.
+            <filename>poky/build</filename>), start Toaster using this
+            command:
+            <literallayout class='monospaced'>
+     $ source ../bitbake/bin/toaster
+            </literallayout>
+            You can now run builds normally.
+        </para>
+
+        <para>
+            To see the build information provided by Toaster, open your
+            favorite browser and enter the following:
+            <literallayout class='monospaced'>
+     http://127.0.0.1:8000
+            </literallayout>
+        </para>
+    </section>
+
+    <section id='setting-a-different-port'>
+        <title>Setting a Different Port</title>
+
+        <para>
+            By default, Toaster starts on port 8000.
+            You can use the <filename>WEBPORT</filename> parameter to
+            set a different port.
+            For example, either of the following commands sets the
+            port to "8400":
+            <literallayout class='monospaced'>
+     $ bitbake/bin/toaster webport=8400
+            </literallayout>
+            or
+            <literallayout class='monospaced'>
+     $ source ../bitbake/bin/toaster webport=8400
+            </literallayout>
+        </para>
+    </section>
+
+    <section id='the-directory-for-cloning-layers'>
+        <title>The Directory for Cloning Layers</title>
+
+        <para>
+            If you are running Toaster in Build Mode, Toaster creates
+            a <filename>_toaster_clones</filename> directory inside
+            your Source Directory (i.e. <filename>poky</filename>).
+            For example, suppose you use this command to start Toaster:
+            <literallayout class='monospaced'>
+     poky/bitbake/bin/toaster
+            </literallayout>
+            In this example, Toaster creates and uses the
+            <filename>poky/_toaster_clones</filename>
+            directory to clone any layers needed for your builds.
+        </para>
+
+        <para>
+            Alternatively, if you would like all of your Toaster related
+            files and directories to be in a particular location other than
+            the default, you can set the <filename>TOASTER_DIR</filename>
+            environment variable, which takes precedence over your current
+            working directory.
+            Setting this environment variable causes Toaster to create and use
+            <filename>$TOASTER_DIR./_toaster_clones</filename>.
+        </para>
+    </section>
+
+    <section id='toaster-the-build-directory'>
+        <title>The Build Directory</title>
+
+        <para>
+            If you are running Toaster in Build Mode, Toaster creates a
+            build directory within your Source Directory (e.g.
+            <filename>poky</filename>).
+            For example, suppose you use this command to start Toaster:
+            <literallayout class='monospaced'>
+     poky/bitbake/bin/toaster
+            </literallayout>
+            In this example, Toaster creates and uses the
+            <filename>poky/_toaster_clones</filename>
+            directory to execute the builds.
+        </para>
+
+        <para>
+            Alternatively, if you would like all of your Toaster related files
+            and directories to be in a particular location, you can set
+            the <filename>TOASTER_DIR</filename> environment variable,
+            which takes precedence over your current working directory.
+            Setting this environment variable causes Toaster to use
+            <filename>$TOASTER_DIR./build</filename> as the build directory.
+        </para>
+    </section>
+
+    <section id='toaster-creating-a-django-super-user'>
+        <title>Creating a Django Superuser</title>
+
+        <para>
+            Toaster is built on the Django framework.
+            Django provides an administration interface you can use
+            to edit Toaster configuration parameters.
+        </para>
+
+        <para>
+            To access the Django administration interface, you must
+            create a superuser by following these steps:
+            <orderedlist>
+                <listitem><para>
+                    If you used <filename>virtualenv</filename>, which is
+                    recommended, to set up the Toaster system dependencies,
+                    you need be sure the virtual environment is activated.
+                    To activate this environment, use the following:
+                    <literallayout class='monospaced'>
+     $ source venv/bin/activate
+                    </literallayout>
+                    </para></listitem>
+                <listitem><para>
+                    From the root of your checkout directory, invoke the
+                    following command from <filename>manage.py</filename>:
+                    <literallayout class='monospaced'>
+     $ ./bitbake/lib/toaster/manage.py createsuperuser
+                    </literallayout>
+                    </para></listitem>
+                <listitem><para>
+                    Django prompts you for the username, which you need to
+                    provide.
+                    </para></listitem>
+                <listitem><para>
+                    Django prompts you for an email address, which is
+                    optional.
+                    </para></listitem>
+                <listitem><para>
+                    Django prompts you for a password, which you must provide.
+                    </para></listitem>
+                <listitem><para>
+                    Django prompts you to re-enter your password for verification.
+                    </para></listitem>
+            </orderedlist>
+            After completing these steps, the following confirmation message
+            appears:
+            <literallayout class='monospaced'>
+     Superuser created successfully.
+            </literallayout>
+        </para>
+
+        <para>
+            Creating a superuser allows you to access the Django administration
+            interface through a browser.
+            The URL for this interface is the same as the URL used for the
+            Toaster instance with "/admin" on the end.
+            For example, if you are running Toaster locally, use the
+            following URL:
+            <literallayout class='monospaced'>
+     http://127.0.0.1:8000/admin
+            </literallayout>
+            You can use the Django administration interface to set Toaster
+            configuration parameters such as the build directory, layer sources,
+            default variable values, and BitBake versions.
+        </para>
+    </section>
+
+    <section id='toaster-setting-up-a-production-instance-of-toaster'>
+        <title>Setting Up a Production Instance of Toaster</title>
+
+        <para>
+            You can use a production instance of Toaster to share the
+            Toaster instance with remote users, multiple users, or both.
+            The production instance is also the setup that can cope with
+            heavier loads on the web service.
+            Use the instructions in the following sections to set up
+            Toaster in
+            <link linkend='toaster-build-mode'>Build Mode</link>
+            where builds and projects are run,
+            viewed, and defined through the Toaster web interface.
+        </para>
+
+        <section id='toaster-production-instance-requirements'>
+            <title>Requirements</title>
+
+            <para>
+                Be sure you meet the following requirements:
+                <note>
+                    You must comply with all Apache,
+                    <filename>mod-wsgi</filename>, and Mysql requirements.
+                </note>
+                <itemizedlist>
+                    <listitem><para>
+                        Have all the build requirements as described in
+                        "<link linkend='toaster-setting-up-the-basic-system-requirements'>Setting Up the Basic System Requirements</link>"
+                        chapter.
+                        </para></listitem>
+                    <listitem><para>
+                        Have an Apache webserver.
+                        </para></listitem>
+                    <listitem><para>
+                        Have <filename>mod-wsgi</filename> for the Apache
+                        webserver.
+                        </para></listitem>
+                    <listitem><para>
+                        Use the Mysql database server.
+                        </para></listitem>
+                    <listitem><para>
+                        If you are using Ubuntu 14.04.3, run the following:
+                        <literallayout class='monospaced'>
+     $ sudo apt-get install apache2 libapache2-mod-wsgi mysql-server virtualenv libmysqlclient-dev
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        If you are using Fedora 22 or a RedHat distribution, run
+                        the following:
+                        <literallayout class='monospaced'>
+     $ sudo dnf install httpd mod_wsgi python-virtualenv gcc mysql-devel
+                        </literallayout>
+                        </para></listitem>
+                </itemizedlist>
+            </para>
+        </section>
+
+        <section id='toaster-installation-steps'>
+            <title>Installation</title>
+
+            <para>
+                Perform the following steps to install Toaster:
+                <orderedlist>
+                    <listitem><para>
+                        Checkout a copy of <filename>poky</filename>
+                        into the web server directory.
+                        You will be using <filename>/var/www/toaster</filename>:
+                        <literallayout class='monospaced'>
+     $ mkdir -p /var/www/toaster
+     $ cd /var/www/toaster/
+     $ git clone git://git.yoctoproject.org/poky
+     $ git checkout &DISTRO_NAME;
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        Initialize a virtual environment and install Toaster
+                        dependencies.
+                        Using a virtual environment keeps the Python packages
+                        isolated from your system-provided packages:
+                        <literallayout class='monospaced'>
+     $ cd /var/www/toaster/
+     $ virtualenv venv
+     $ source ./venv/bin/activate
+     $ pip install -r ./poky/bitbake/toaster-requirements.txt
+     $ pip install mysql
+     $ pip install MySQL-python
+                        </literallayout>
+                        <note>
+                            Isolating these packages is not required but is
+                            recommended.
+                            Alternatively, you can use your operating system's
+                            package manager to install the packages.
+                        </note>
+                        </para></listitem>
+                    <listitem><para>
+                        Configure Toaster by editing
+                        <filename>/var/www/toaster/poky/bitbake/lib/toaster/toastermain/settings.py</filename>
+                        as follows:
+                        <itemizedlist>
+                            <listitem><para>
+                                Edit the <filename>DATABASE</filename> settings:
+                                <literallayout class='monospaced'>
+     DATABASES = {
+         'default': {
+             'ENGINE': 'django.db.backends.mysql',
+             'NAME': 'toaster_data',
+             'USER': 'toaster',
+             'PASSWORD': 'yourpasswordhere',
+             'HOST': 'localhost',
+             'PORT': '3306',
+     }
+                                </literallayout>
+                                </para></listitem>
+                            <listitem><para>
+                                Edit the <filename>SECRET_KEY</filename>:
+                                <literallayout class='monospaced'>
+     SECRET_KEY = '<replaceable>your_secret_key</replaceable>'
+                                </literallayout>
+                                </para></listitem>
+                            <listitem><para>
+                                Edit the <filename>STATIC_ROOT</filename>:
+                                <literallayout class='monospaced'>
+     STATIC_ROOT = '/var/www/toaster/static_files/'
+                                </literallayout>
+                                </para></listitem>
+                            <listitem><para>
+                                Enable Build Mode by adding the following
+                                line to <filename>settings.py</filename>:
+                                <literallayout class='monospaced'>
+     BUILD_MODE=True
+                                </literallayout>
+                                </para></listitem>
+                        </itemizedlist>
+                        </para></listitem>
+                    <listitem><para>
+                        Add the database and user to the <filename>mysql</filename>
+                        server defined earlier:
+                        <literallayout class='monospaced'>
+     $ mysql -u root -p
+     mysql> CREATE DATABASE toaster;
+     mysql> CREATE USER 'toaster'@'localhost' identified by 'yourpasswordhere';
+     mysql> GRANT all on toaster_data.* to 'toaster'@'localhost';
+     mysql> quit
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        Get Toaster to create the database schema,
+                        default data, and gather the statically-served files:
+                        <literallayout class='monospaced'>
+     $ cd  /var/www/toaster/poky/
+     $ ./bitbake/lib/toaster/manage.py syncdb --migrate
+     $ ./bitbake/lib/toaster/manage.py loadconf ./meta-yocto/conf/toasterconf.json
+     $ ./bitbake/lib/toaster/manage.py lsupdates
+     $ ./bitbake/lib/toaster/manage.py collectstatic
+                        </literallayout>
+                        </para>
+
+                        <para>
+                            For the above set of commands, after moving to the
+                            <filename>poky</filename> directory,
+                            the <filename>syncdb</filename> command with the
+                            <filename>migrate</filename> option makes sure the database
+                            schema has had changes propagated correctly (i.e.
+                            migrations).
+                            See the
+                            <ulink url='https://south.readthedocs.org/en/latest/commands.html#syncdb'><filename>syncdb</filename></ulink>
+                            command for more information.
+                        </para>
+
+                        <para>
+                            The
+                            <link linkend='toaster-command-loadconf'><filename>loadconf</filename></link>
+                            command loads the
+                            <filename>./meta-yocto/conf/toasterconf.json</filename>
+                            JSON file.
+                        </para>
+
+                        <para>
+                            The <filename>lsupdates</filename> command fetches the
+                            initial recipes and layers data from the layer index.
+                        </para>
+
+                        <para>
+                            Finally, the <filename>collectstatic</filename> command
+                            is a Django framework command that collects all the
+                            statically served files into a designated directory to
+                            be served up by the Apache web server.
+                            For more information on this Django command, see
+                            <ulink url='https://docs.djangoproject.com/en/1.7/ref/contrib/staticfiles/'></ulink>.
+                        </para></listitem>
+                    <listitem><para>
+                        Add an Apache configuration file for Toaster to your Apache web
+                        server's configuration directory.
+                        If you are using Ubuntu or Debian, put the file here:
+                        <literallayout class='monospaced'>
+     /etc/apache2/conf-available/toaster.conf
+                        </literallayout>
+                        If you are using Fedora or RedHat, put it here:
+                        <literallayout class='monospaced'>
+     /etc/httpd/conf.d/toaster.conf
+                        </literallayout>
+                        Following is a sample Apache configuration for Toaster
+                        you can follow:
+                        <literallayout class='monospaced'>
+     Alias /static /var/www/toaster/static_files
+     &lt;Directory /var/www/toaster/static_files&gt;
+             Order allow,deny
+             Allow from all
+             Require all granted
+     &lt;/Directory&gt;
+
+     WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/venv/lib/python2.7/site-packages
+
+     WSGIScriptAlias / "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py"
+     &lt;Location /&gt;
+         WSGIProcessGroup toastern_wsgi
+     &lt;/Location&gt;
+                        </literallayout>
+                        If you are using Ubuntu or Debian,
+                        you will need to enable the config and module for Apache:
+                        <literallayout class='monospaced'>
+     $ sudo a2enmod wsgi
+     $ sudo a2enconf toaster
+                        </literallayout>
+                        Finally, restart Apache to make sure all new configuration
+                        is loaded.
+                        For Ubuntu and Debian use:
+                        <literallayout class='monospaced'>
+     $ sudo service apache2 restart
+                        </literallayout>
+                        For Fedora and RedHat use:
+                        <literallayout class='monospaced'>
+     $ sudo service httpd restart
+                        </literallayout>
+                        </para></listitem>
+                    <listitem><para>
+                        Install the build runner service.
+                        This service needs to be running in order to dispatch
+                        builds.
+                        Use this command:
+                        <literallayout class='monospaced'>
+     /var/www/toaster/poky/bitbake/lib/toaster/manage.py runbuilds
+                        </literallayout>
+                        Here is an example:
+                        <literallayout class='monospaced'>
+     #!/bin/sh
+     # toaster run builds dispatcher
+     cd /var/www/toaster/
+     source ./venv/bin/activate
+     ./bitbake/lib/toaster/manage.py runbuilds
+                        </literallayout>
+                        </para></listitem>
+                </orderedlist>
+                You can now open up a browser and start using Toaster.
+            </para>
+        </section>
+    </section>
+
+
+
+
+<!--    <section id='using-toaster-in-analysis-mode'>
         <title>Using Toaster in Analysis Mode</title>
 
+
         <para>
             This section describes how to use Toaster in Analysis Mode
             after setting Toaster up as a local instance or as a hosted
@@ -324,14 +784,14 @@
                     <listitem><para><emphasis>Start the BitBake Server:</emphasis>
                         Start the BitBake server using the following command:
                         <literallayout class='monospaced'>
-     $ bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:0 &amp;&amp; export BBSERVER=localhost:-1
+     $ bitbake &dash;&dash;postread conf/toaster.conf &dash;&dash;server-only -t xmlrpc -B localhost:0 &amp;&amp; export BBSERVER=localhost:-1
                         </literallayout>
                         </para></listitem>
                     <listitem><para><emphasis>Start the Logging Server:</emphasis>
                         Start the Toaster Logging Interface using the following
                         command:
                         <literallayout class='monospaced'>
-     $ nohup bitbake --observe-only -u toasterui >toaster_ui.log &amp;
+     $ nohup bitbake &dash;&dash;observe-only -u toasterui >toaster_ui.log &amp;
                         </literallayout>
                         <note>
                             No hard-coded ports are used in the BitBake options
@@ -810,6 +1270,7 @@
             </para>
         </section>
     </section>
+-->
 
     <section id='using-the-toaster-web-interface'>
         <title>Using the Toaster Web Interface</title>
@@ -875,6 +1336,12 @@
             <para>
                 Following are several videos that show how to use the Toaster GUI:
                 <itemizedlist>
+                    <listitem><para><emphasis>Build Custom Layers:</emphasis>
+                        This
+                        <ulink url='https://www.youtube.com/watch?v=QJzaE_XjX5c'>video</ulink>
+                        shows you how to build custom layers that are used with
+                        Toaster.
+                        </para></listitem>
                     <listitem><para><emphasis>Build Configuration:</emphasis>
                         This
                         <ulink url='https://www.youtube.com/watch?v=qYgDZ8YzV6w'>video</ulink>
diff --git a/documentation/toaster-manual/toaster-manual-start.xml b/documentation/toaster-manual/toaster-manual-start.xml
index 32be3a8473..daefa79090 100644
--- a/documentation/toaster-manual/toaster-manual-start.xml
+++ b/documentation/toaster-manual/toaster-manual-start.xml
@@ -15,12 +15,13 @@
         <title>Setting Up the Basic System Requirements</title>
 
         <para>
-            You first need to be sure your build system is set up to run
-            the Yocto Project.
-            See the
-            "<ulink url='&YOCTO_DOCS_QS_URL;#yp-resources'>Setting Up to Use the Yocto Project</ulink>"
-            section in the Yocto Project Quick Start for information on how
-            to set up your system for the Yocto Project.
+            Before you can use Toaster, you need to first set up your
+            build system to run the Yocto Project.
+            To do this, follow the instructions in the
+            "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Build Host Packages</ulink>"
+            and
+            "<ulink url='&YOCTO_DOCS_QS_URL;#releases'>Yocto Project Release</ulink>"
+            sections in the Yocto Project Quick Start.
         </para>
     </section>