1 files changed, 635 insertions, 0 deletions

diff --git a/documentation/toaster-manual/setup-and-use.rst b/documentation/toaster-manual/setup-and-use.rst
new file mode 100644
index 0000000000..2cb7884eb9
--- /dev/null
+++ b/documentation/toaster-manual/setup-and-use.rst
@@ -0,0 +1,635 @@
+.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
+.. Set default pygment highlighting to 'shell' for this document
+.. highlight:: shell
+
+****************************
+Setting Up and Using Toaster
+****************************
+
+Starting Toaster for Local Development
+======================================
+
+Once you have set up the Yocto Project and installed the Toaster system
+dependencies as described in the ":ref:`toaster-manual/start:Preparing to Use
+Toaster`" chapter, you are ready to start
+Toaster.
+
+Navigate to the root of your
+:term:`Source Directory` (e.g. ``poky``)::
+
+   $ cd poky
+
+Once in that directory, source the build environment script::
+
+   $ source oe-init-build-env
+
+Next, from the build directory (e.g.
+``poky/build``), start Toaster using this command::
+
+   $ source toaster start
+
+You can now run your builds from the command line, or with Toaster
+as explained in section
+":ref:`toaster-manual/setup-and-use:using the toaster web interface`".
+
+To access the Toaster web interface, open your favorite browser and
+enter the following::
+
+   http://127.0.0.1:8000
+
+Setting a Different Port
+========================
+
+By default, Toaster starts on port 8000. You can use the ``WEBPORT``
+parameter to set a different port. For example, the following command
+sets the port to "8400"::
+
+   $ source toaster start webport=8400
+
+Setting Up Toaster Without a Web Server
+=======================================
+
+You can start a Toaster environment without starting its web server.
+This is useful for the following:
+
+-  Capturing a command-line build's statistics into the Toaster database
+   for examination later.
+
+-  Capturing a command-line build's statistics when the Toaster server
+   is already running.
+
+-  Having one instance of the Toaster web server track and capture
+   multiple command-line builds, where each build is started in its own
+   "noweb" Toaster environment.
+
+The following commands show how to start a Toaster environment without
+starting its web server, perform BitBake operations, and then shut down
+the Toaster environment. Once the build is complete, you can close the
+Toaster environment. Before closing the environment, however, you should
+allow a few minutes to ensure the complete transfer of its BitBake build
+statistics to the Toaster database. If you have a separate Toaster web
+server instance running, you can watch this command-line build's
+progress and examine the results as soon as they are posted::
+
+   $ source toaster start noweb
+   $ bitbake target
+   $ source toaster stop
+
+Setting Up Toaster Without a Build Server
+=========================================
+
+You can start a Toaster environment with the "New Projects" feature
+disabled. Doing so is useful for the following:
+
+-  Sharing your build results over the web server while blocking others
+   from starting builds on your host.
+
+-  Allowing only local command-line builds to be captured into the
+   Toaster database.
+
+Use the following command to set up Toaster without a build server::
+
+   $ source toaster start nobuild webport=port
+
+Setting up External Access
+==========================
+
+By default, Toaster binds to the loop back address (i.e. ``localhost``),
+which does not allow access from external hosts. To allow external
+access, use the ``WEBPORT`` parameter to open an address that connects
+to the network, specifically the IP address that your NIC uses to
+connect to the network. You can also bind to all IP addresses the
+computer supports by using the shortcut "0.0.0.0:port".
+
+The following example binds to all IP addresses on the host::
+
+   $ source toaster start webport=0.0.0.0:8400
+
+This example binds to a specific IP address on the host's NIC::
+
+   $ source toaster start webport=192.168.1.1:8400
+
+The Directory for Cloning Layers
+================================
+
+Toaster creates a ``_toaster_clones`` directory inside your Source
+Directory (i.e. ``poky``) to clone any layers needed for your builds.
+
+Alternatively, if you would like all of your Toaster related files and
+directories to be in a particular location other than the default, you
+can set the ``TOASTER_DIR`` environment variable, which takes precedence
+over your current working directory. Setting this environment variable
+causes Toaster to create and use ``$TOASTER_DIR./_toaster_clones``.
+
+The Build Directory
+===================
+
+Toaster creates a build directory within your Source Directory (e.g.
+``poky``) to execute the builds.
+
+Alternatively, if you would like all of your Toaster related files and
+directories to be in a particular location, you can set the
+``TOASTER_DIR`` environment variable, which takes precedence over your
+current working directory. Setting this environment variable causes
+Toaster to use ``$TOASTER_DIR/build`` as the build directory.
+
+Creating a Django Superuser
+===========================
+
+Toaster is built on the `Django
+framework <https://www.djangoproject.com/>`__. Django provides an
+administration interface you can use to edit Toaster configuration
+parameters.
+
+To access the Django administration interface, you must create a
+superuser by following these steps:
+
+#. If you used ``pip3``, which is recommended, to set up the Toaster
+   system dependencies, you need be sure the local user path is in your
+   ``PATH`` list. To append the pip3 local user path, use the following
+   command::
+
+      $ export PATH=$PATH:$HOME/.local/bin
+
+#. From the directory containing the Toaster database, which by default
+   is the :term:`Build Directory`,
+   invoke the ``createsuperuser`` command from ``manage.py``::
+
+      $ cd ~/poky/build
+      $ ../bitbake/lib/toaster/manage.py createsuperuser
+
+#. Django prompts you for the username, which you need to provide.
+
+#. Django prompts you for an email address, which is optional.
+
+#. Django prompts you for a password, which you must provide.
+
+#. Django prompts you to re-enter your password for verification.
+
+After completing these steps, the following confirmation message
+appears::
+
+   Superuser created successfully.
+
+Creating a superuser allows you to access the Django administration
+interface through a browser. The URL for this interface is the same as
+the URL used for the Toaster instance with "/admin" on the end. For
+example, if you are running Toaster locally, use the following URL::
+
+   http://127.0.0.1:8000/admin
+
+You can use the Django administration interface to set Toaster configuration
+parameters such as the build directory, layer sources, default variable
+values, and BitBake versions.
+
+Setting Up a Production Instance of Toaster
+===========================================
+
+You can use a production instance of Toaster to share the Toaster
+instance with remote users, multiple users, or both. The production
+instance is also the setup that can handle heavier loads on the web
+service. Use the instructions in the following sections to set up
+Toaster to run builds through the Toaster web interface.
+
+Requirements
+------------
+
+Be sure you meet the following requirements:
+
+.. note::
+
+   You must comply with all Apache, ``mod-wsgi``, and Mysql requirements.
+
+-  Have all the build requirements as described in the ":ref:`toaster-manual/start:Preparing to
+   Use Toaster`" chapter.
+
+-  Have an Apache webserver.
+
+-  Have ``mod-wsgi`` for the Apache webserver.
+
+-  Use the Mysql database server.
+
+-  If you are using Ubuntu, run the following::
+
+      $ sudo apt-get install apache2 libapache2-mod-wsgi-py3 mysql-server python3-pip libmysqlclient-dev
+
+-  If you are using Fedora or a RedHat distribution, run the
+   following::
+
+      $ sudo dnf install httpd python3-mod_wsgi python3-pip mariadb-server mariadb-devel python3-devel
+
+-  If you are using openSUSE, run the following::
+
+      $ sudo zypper install apache2 apache2-mod_wsgi-python3 python3-pip mariadb mariadb-client python3-devel
+
+Installation
+------------
+
+Perform the following steps to install Toaster:
+
+#.  Create toaster user and set its home directory to
+    ``/var/www/toaster``::
+
+      $ sudo /usr/sbin/useradd toaster -md /var/www/toaster -s /bin/false
+      $ sudo su - toaster -s /bin/bash
+
+#.  Checkout a copy of ``poky`` into the web server directory. You will
+    be using ``/var/www/toaster``::
+
+      $ git clone git://git.yoctoproject.org/poky
+      $ git checkout &DISTRO_NAME_NO_CAP;
+
+#.  Install Toaster dependencies using the --user flag which keeps the
+    Python packages isolated from your system-provided packages::
+
+      $ cd /var/www/toaster/
+      $ pip3 install --user -r ./poky/bitbake/toaster-requirements.txt
+      $ pip3 install --user mysqlclient
+
+    .. note::
+
+       Isolating these packages is not required but is recommended.
+       Alternatively, you can use your operating system's package
+       manager to install the packages.
+
+#.  Configure Toaster by editing
+    ``/var/www/toaster/poky/bitbake/lib/toaster/toastermain/settings.py``
+    as follows:
+
+    -  Edit the
+       `DATABASES <https://docs.djangoproject.com/en/2.2/ref/settings/#databases>`__
+       settings:
+
+       .. code-block:: python
+
+         DATABASES = {
+            'default': {
+               'ENGINE': 'django.db.backends.mysql',
+               'NAME': 'toaster_data',
+               'USER': 'toaster',
+               'PASSWORD': 'yourpasswordhere',
+               'HOST': 'localhost',
+               'PORT': '3306',
+            }
+         }
+
+    -  Edit the
+       `SECRET_KEY <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-SECRET_KEY>`__:
+
+       .. code-block:: python
+
+         SECRET_KEY = 'your_secret_key'
+
+    -  Edit the
+       `STATIC_ROOT <https://docs.djangoproject.com/en/2.2/ref/settings/#std:setting-STATIC_ROOT>`__:
+
+      .. code-block:: python
+
+         STATIC_ROOT = '/var/www/toaster/static_files/'
+
+#.  Add the database and user to the ``mysql`` server defined earlier::
+
+      $ mysql -u root -p
+      mysql> CREATE DATABASE toaster_data;
+      mysql> CREATE USER 'toaster'@'localhost' identified by 'yourpasswordhere';
+      mysql> GRANT all on toaster_data.\* to 'toaster'@'localhost';
+      mysql> quit
+
+#.  Get Toaster to create the database schema, default data, and gather
+    the statically-served files::
+
+      $ cd /var/www/toaster/poky/
+      $ ./bitbake/lib/toaster/manage.py migrate
+      $ TOASTER_DIR=`pwd\` TEMPLATECONF='poky' \
+         ./bitbake/lib/toaster/manage.py checksettings
+      $ ./bitbake/lib/toaster/manage.py collectstatic
+
+
+    In the previous
+    example, from the ``poky`` directory, the ``migrate`` command
+    ensures the database schema changes have propagated correctly (i.e.
+    migrations). The next line sets the Toaster root directory
+    ``TOASTER_DIR`` and the location of the Toaster configuration file
+    ``TOASTER_CONF``, which is relative to ``TOASTER_DIR``. The
+    ``TEMPLATECONF`` value reflects the contents of
+    ``poky/.templateconf``, and by default, should include the string
+    "poky". For more information on the Toaster configuration file, see
+    the ":ref:`toaster-manual/reference:Configuring Toaster`" section.
+
+    This line also runs the ``checksettings`` command, which configures
+    the location of the Toaster :term:`Build Directory`.
+    The Toaster
+    root directory ``TOASTER_DIR`` determines where the Toaster build
+    directory is created on the file system. In the example above,
+    ``TOASTER_DIR`` is set as follows::
+
+       /var/www/toaster/poky
+
+
+    This setting causes the Toaster build directory to be::
+
+       /var/www/toaster/poky/build
+
+    Finally, the ``collectstatic`` command is a Django framework command
+    that collects all the statically served files into a designated
+    directory to be served up by the Apache web server as defined by
+    ``STATIC_ROOT``.
+
+#.  Test and/or use the Mysql integration with Toaster's Django web
+    server. At this point, you can start up the normal Toaster Django
+    web server with the Toaster database in Mysql. You can use this web
+    server to confirm that the database migration and data population
+    from the Layer Index is complete.
+
+    To start the default Toaster Django web server with the Toaster
+    database now in Mysql, use the standard start commands::
+
+      $ source oe-init-build-env
+      $ source toaster start
+
+    Additionally, if Django is sufficient for your requirements, you can use
+    it for your release system and migrate later to Apache as your
+    requirements change.
+
+#.  Add an Apache configuration file for Toaster to your Apache web
+    server's configuration directory. If you are using Ubuntu or Debian,
+    put the file here::
+
+      /etc/apache2/conf-available/toaster.conf
+
+
+    If you are using Fedora or RedHat, put it here::
+
+      /etc/httpd/conf.d/toaster.conf
+
+    If you are using OpenSUSE, put it here::
+
+      /etc/apache2/conf.d/toaster.conf
+
+    Following is a sample Apache configuration for Toaster you can follow:
+
+    .. code-block:: apache
+
+      Alias /static /var/www/toaster/static_files
+      <Directory /var/www/toaster/static_files>
+         <IfModule mod_access_compat.c>
+            Order allow,deny
+            Allow from all
+         </IfModule>
+         <IfModule !mod_access_compat.c>
+            Require all granted
+         </IfModule>
+      </Directory>
+    
+      <Directory /var/www/toaster/poky/bitbake/lib/toaster/toastermain>
+         <Files "wsgi.py">
+            Require all granted
+         </Files>
+      </Directory>
+    
+      WSGIDaemonProcess toaster_wsgi python-path=/var/www/toaster/poky/bitbake/lib/toaster:/var/www/toaster/.local/lib/python3.4/site-packages
+      WSGIScriptAlias / "/var/www/toaster/poky/bitbake/lib/toaster/toastermain/wsgi.py"
+      <Location />
+         WSGIProcessGroup toaster_wsgi
+      </Location>
+
+
+    If you are using Ubuntu or Debian, you will need to enable the config and
+    module for Apache::
+
+      $ sudo a2enmod wsgi
+      $ sudo a2enconf toaster
+      $ chmod +x bitbake/lib/toaster/toastermain/wsgi.py
+
+    Finally, restart Apache to make sure all new configuration is loaded. For Ubuntu,
+    Debian, and OpenSUSE use::
+
+      $ sudo service apache2 restart
+
+    For Fedora and RedHat use::
+
+      $ sudo service httpd restart
+
+#.  Prepare the systemd service to run Toaster builds. Here is a sample
+    configuration file for the service:
+
+    .. code-block:: ini
+
+      [Unit]
+      Description=Toaster runbuilds
+      
+      [Service]
+      Type=forking User=toaster
+      ExecStart=/usr/bin/screen -d -m -S runbuilds /var/www/toaster/poky/bitbake/lib/toaster/runbuilds-service.sh start
+      ExecStop=/usr/bin/screen -S runbuilds -X quit
+      WorkingDirectory=/var/www/toaster/poky
+      
+      [Install]
+      WantedBy=multi-user.target
+
+
+    Prepare the ``runbuilds-service.sh`` script that you need to place in the
+    ``/var/www/toaster/poky/bitbake/lib/toaster/`` directory by setting
+    up executable permissions::
+
+      #!/bin/bash
+      
+      #export http_proxy=http://proxy.host.com:8080
+      #export https_proxy=http://proxy.host.com:8080
+      #export GIT_PROXY_COMMAND=$HOME/bin/gitproxy
+      cd ~/poky/
+      source ./oe-init-build-env build
+      source ../bitbake/bin/toaster $1 noweb
+      [ "$1" == 'start' ] && /bin/bash
+
+#.  Run the service::
+
+       $ sudo service runbuilds start
+
+    Since the service is running in a detached screen session, you can
+    attach to it using this command::
+
+       $ sudo su - toaster
+       $ screen -rS runbuilds
+
+    You can detach from the service again using "Ctrl-a" followed by "d" key
+    combination.
+
+You can now open up a browser and start using Toaster.
+
+Using the Toaster Web Interface
+===============================
+
+The Toaster web interface allows you to do the following:
+
+-  Browse published layers in the `OpenEmbedded Layer
+   Index <http://layers.openembedded.org>`__ that are available for your
+   selected version of the build system.
+
+-  Import your own layers for building.
+
+-  Add and remove layers from your configuration.
+
+-  Set configuration variables.
+
+-  Select a target or multiple targets to build.
+
+-  Start your builds.
+
+-  See what was built (recipes and packages) and what packages were
+   installed into your final image.
+
+-  Browse the directory structure of your image.
+
+-  See the value of all variables in your build configuration, and which
+   files set each value.
+
+-  Examine error, warning and trace messages to aid in debugging.
+
+-  See information about the BitBake tasks executed and reused during
+   your build, including those that used shared state.
+
+-  See dependency relationships between recipes, packages and tasks.
+
+-  See performance information such as build time, task time, CPU usage,
+   and disk I/O.
+
+Toaster Web Interface Videos
+----------------------------
+
+Following are several videos that show how to use the Toaster GUI:
+
+-  *Build Configuration:* This
+   `video <https://www.youtube.com/watch?v=qYgDZ8YzV6w>`__ overviews and
+   demonstrates build configuration for Toaster.
+
+-  *Build Custom Layers:* This
+   `video <https://www.youtube.com/watch?v=QJzaE_XjX5c>`__ shows you how
+   to build custom layers that are used with Toaster.
+
+-  *Toaster Homepage and Table Controls:* This
+   `video <https://www.youtube.com/watch?v=QEARDnrR1Xw>`__ goes over the
+   Toaster entry page, and provides an overview of the data manipulation
+   capabilities of Toaster, which include search, sorting and filtering
+   by different criteria.
+
+-  *Build Dashboard:* This
+   `video <https://www.youtube.com/watch?v=KKqHYcnp2gE>`__ shows you the
+   build dashboard, a page providing an overview of the information
+   available for a selected build.
+
+-  *Image Information:* This
+   `video <https://www.youtube.com/watch?v=XqYGFsmA0Rw>`__ walks through
+   the information Toaster provides about images: packages installed and
+   root file system.
+
+-  *Configuration:* This
+   `video <https://www.youtube.com/watch?v=UW-j-T2TzIg>`__ provides
+   Toaster build configuration information.
+
+-  *Tasks:* This `video <https://www.youtube.com/watch?v=D4-9vGSxQtw>`__
+   shows the information Toaster provides about the tasks run by the
+   build system.
+
+-  *Recipes and Packages Built:* This
+   `video <https://www.youtube.com/watch?v=x-6dx4huNnw>`__ shows the
+   information Toaster provides about recipes and packages built.
+
+-  *Performance Data:* This
+   `video <https://www.youtube.com/watch?v=qWGMrJoqusQ>`__ shows the
+   build performance data provided by Toaster.
+
+Additional Information About the Local Yocto Project Release
+------------------------------------------------------------
+
+This section only applies if you have set up Toaster for local
+development, as explained in the
+":ref:`toaster-manual/setup-and-use:starting toaster for local development`"
+section.
+
+When you create a project in Toaster, you will be asked to provide a
+name and to select a Yocto Project release. One of the release options
+you will find is called "Local Yocto Project".
+
+.. image:: figures/new-project.png
+   :align: center
+   :scale: 75%
+
+When you select the "Local Yocto Project" release, Toaster will run your
+builds using the local Yocto Project clone you have in your computer:
+the same clone you are using to run Toaster. Unless you manually update
+this clone, your builds will always use the same Git revision.
+
+If you select any of the other release options, Toaster will fetch the
+tip of your selected release from the upstream :yocto_git:`Yocto Project
+repository <>` every time you run a build.
+Fetching this tip effectively means that if your selected release is
+updated upstream, the Git revision you are using for your builds will
+change. If you are doing development locally, you might not want this
+change to happen. In that case, the "Local Yocto Project" release might
+be the right choice.
+
+However, the "Local Yocto Project" release will not provide you with any
+compatible layers, other than the three core layers that come with the
+Yocto Project:
+
+-  `openembedded-core <http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/>`__
+
+-  `meta-poky <http://layers.openembedded.org/layerindex/branch/master/layer/meta-poky/>`__
+
+-  `meta-yocto-bsp <http://layers.openembedded.org/layerindex/branch/master/layer/meta-yocto-bsp/>`__
+
+.. image:: figures/compatible-layers.png
+   :align: center
+   :scale: 75%
+
+If you want to build any other layers, you will need to manually import
+them into your Toaster project, using the "Import layer" page.
+
+.. image:: figures/import-layer.png
+   :align: center
+   :scale: 75%
+
+Building a Specific Recipe Given Multiple Versions
+--------------------------------------------------
+
+Occasionally, a layer might provide more than one version of the same
+recipe. For example, the ``openembedded-core`` layer provides two
+versions of the ``bash`` recipe (i.e. 3.2.48 and 4.3.30-r0) and two
+versions of the ``which`` recipe (i.e. 2.21 and 2.18). The following
+figure shows this exact scenario:
+
+.. image:: figures/bash-oecore.png
+   :align: center
+   :scale: 75%
+
+By default, the OpenEmbedded build system builds one of the two recipes.
+For the ``bash`` case, version 4.3.30-r0 is built by default.
+Unfortunately, Toaster as it exists, is not able to override the default
+recipe version. If you would like to build bash 3.2.48, you need to set
+the
+:term:`PREFERRED_VERSION`
+variable. You can do so from Toaster, using the "Add variable" form,
+which is available in the "BitBake variables" page of the project
+configuration section as shown in the following screen:
+
+.. image:: figures/add-variable.png
+   :align: center
+   :scale: 75%
+
+To specify ``bash`` 3.2.48 as the version to build, enter
+"PREFERRED_VERSION_bash" in the "Variable" field, and "3.2.48" in the
+"Value" field. Next, click the "Add variable" button:
+
+.. image:: figures/set-variable.png
+   :align: center
+   :scale: 75%
+
+After clicking the "Add variable" button, the settings for
+``PREFERRED_VERSION`` are added to the bottom of the BitBake variables
+list. With these settings, the OpenEmbedded build system builds the
+desired version of the recipe rather than the default version:
+
+.. image:: figures/variable-added.png
+   :align: center
+   :scale: 75%