Updated the "Using NFV Access SDKs" section in NFV Access GuideHEADmaster

USERDOCAP-242 W/new content from D. Motrea

Signed-off-by: Miruna Paun <Miruna.Paun@enea.com>

1 files changed, 320 insertions, 43 deletions

diff --git a/doc/book-enea-nfv-access-guide/doc/using_nfv_access_sdks.xml b/doc/book-enea-nfv-access-guide/doc/using_nfv_access_sdks.xml
index 18602a1..e5c28ea 100644
--- a/doc/book-enea-nfv-access-guide/doc/using_nfv_access_sdks.xml
+++ b/doc/book-enea-nfv-access-guide/doc/using_nfv_access_sdks.xml
@@ -4,24 +4,61 @@
 <chapter id="workflow">
   <title>Using NFV Access SDKs</title>
 
-  <para>Enea NFV Access comes with two different toolchains, one for
-  developing applications for the host and one for applications running in the
-  guest VM. Each is wrapped together with an environment-setup script into a
-  shell archive and is available under the Download section on
-  portal.enea.com. They have self explanatory names.</para>
+  <para>Enea NFV Access comes with two standard SDKs which can be used to
+  develop user-applications and kernel modules for both host and guest
+  images.</para>
+
+  <para>A standard SDK consists of:</para>
 
   <itemizedlist>
     <listitem>
-      <para><literal>inteld1521/sdk/enea-glibc-x86_64-enea-image-virtualization-host-sdk-corei7-64-toolchain-7.0.sh</literal>
+      <para>Cross-Development Toolchain: cross-compiler and
+      cross-debugger</para>
+    </listitem>
+
+    <listitem>
+      <para>Libraries, Headers and Symbols that are specific to the
+      image</para>
+    </listitem>
+
+    <listitem>
+      <para>Environment Setup Script which defines the environment
+      variables</para>
+    </listitem>
+  </itemizedlist>
+
+  <para>To install the SDK on your host development machine, there are two
+  installation scripts available under the Download section on <emphasis
+  role="bold">portal.enea.com</emphasis>. They have self explanatory
+  names.</para>
+
+  <itemizedlist>
+    <listitem>
+      <para><literal>inteld1521/sdk/enea-glibc-x86_64-enea-image-virtualization-host-sdkcorei7-64-toolchain-7.0.sh</literal>
       - for host applications.</para>
     </listitem>
 
     <listitem>
-      <para><literal>qemux86-64/sdk/enea-glibc-x86_64-enea-image-virtualization-guest-sdk-core2-64-toolchain-7.0.sh</literal>
+      <para><literal>qemux86-64/sdk/enea-glibc-x86_64-enea-image-virtualization-guestsdk-core2-64-toolchain-7.0.sh</literal>
       - for guest applications.</para>
     </listitem>
   </itemizedlist>
 
+  <para>After installing the SDK, a developer will be able to compile and
+  generate executables for the preferred target machine. Cross-gdb
+  (<filename>x86_64-enea-linux-gdb</filename>) is created by the
+  Cross-Development toolchain. It can be used to debug applications on the
+  target platform from the development workstation. For kernel debugging,
+  <command>ftrace</command> and <command>kgdb</command> are enabled on the
+  host sdk image.</para>
+
+  <para>Various user-space tools helpful in the development process, are also
+  provided. The tools include <emphasis role="bold">LatencyTop</emphasis>,
+  <emphasis role="bold">Perf</emphasis>, <emphasis
+  role="bold">CrossTap</emphasis>, <emphasis role="bold">OProfile</emphasis>,
+  <emphasis role="bold">Lttng-ust</emphasis> and <emphasis
+  role="bold">GDBserver</emphasis>.</para>
+
   <section id="install-crosscomp">
     <title>Installing the Cross-Compilation Toolchain</title>
 
@@ -33,12 +70,12 @@
       <listitem>
         <para><programlisting>$ ./enea-glibc-x86_64-enea-image-virtualization-guest-sdk-core2-64-toolchain-7.0.sh</programlisting>When
         prompted, select to install the toolchain in the desired directory,
-        referred to as <literal>&lt;sdkdir&gt;</literal>. </para>
+        referred to as <literal>sdkdir</literal>.</para>
 
         <para>A default path where the toolchain will be installed will be
         shown in the prompt. The installer unpacks the environment setup
-        script in <literal>&lt;sdkdir&gt;</literal> and the toolchain under
-        <literal>&lt;sdkdir&gt;/sysroots</literal>.</para>
+        script in <literal>sdkdir</literal> and the toolchain under
+        <literal>sdkdir/sysroots</literal>.</para>
 
         <note>
           <para>Choose a unique directory for each toolchain. Installing a
@@ -50,7 +87,7 @@
 
       <listitem>
         <para>Setup the toolchain environment for your target by sourcing the
-        environment-setup script. Example: <programlisting>$ source &lt;sdkdir&gt;/environment-setup-core2-64-enea-linux</programlisting></para>
+        environment-setup script. Example: <programlisting>$ source sdkdir/environment-setup-core2-64-enea-linux</programlisting></para>
       </listitem>
     </orderedlist>
   </section>
@@ -71,7 +108,7 @@
         $(CC) -o helloworld helloworld.o
 clean:
         rm -f *.o helloworld
-#include &lt;stdio.h&gt;
+#include stdio.h;
 int main(void) {
         printf("Hello World\n");
         return 0;
@@ -100,7 +137,7 @@ hello world</programlisting>
     <para>Before cross-compiling kernle modules, you need to make sure the
     installed toolchain includes the kernel source tree, which should be
     available at:
-    <literal>&lt;sdkdir&gt;/sysroots/&lt;targetarch&gt;-enea-linux/usr/src/kernel</literal>.</para>
+    <literal>sdkdir/sysroots/targetarch-enea-linux/usr/src/kernel</literal>.</para>
 
     <para>Once the environment-setup script is sourced, you can make your
     kernel modules as usual and get them compiled for your target. Below you
@@ -114,7 +151,7 @@ hello world</programlisting>
         <programlisting>obj-m := hello.o
 PWD := $(shell pwd)
 
-KERNEL_SRC := &lt;full path to kernel source tree&gt;
+KERNEL_SRC := full path to kernel source tree
 
 all: scripts
     $(MAKE) -C $(KERNEL_SRC) M=$(PWD) LDFLAGS="" modules
@@ -122,9 +159,9 @@ scripts:
     $(MAKE) -C $(KERNEL_SRC) scripts
 clean:
     $(MAKE) -C $(KERNEL_SRC) M=$(PWD) LDFLAGS="" clean
-#include &lt;linux/module.h&gt;      /* Needed by all modules */
-#include &lt;linux/kernel.h&gt;      /* Needed for KERN_INFO */
-#include &lt;linux/init.h&gt;        /* Needed for the macros */
+#include linux/module.h     /* Needed by all modules */
+#include linux/kernel.h     /* Needed for KERN_INFO */
+#include linux/init.h       /* Needed for the macros */
 
 static int __init hello_start(void)
 {
@@ -168,36 +205,276 @@ MODULE_LICENSE("GPL");</programlisting>
   <section id="deploy-artifacts">
     <title>Deploying your artifacts</title>
 
-    <orderedlist>
-      <listitem>
-        <para>Deploying on host</para>
+    <section id="deploy_onhost">
+      <title>Deploying on host</title>
+
+      <para>You can use <literal>ssh</literal> to deploy your artifacts on the
+      host target. For this you will need a network connection to the target,
+      to use <literal>scp</literal> to copy to the desired location.</para>
+
+      <para><programlisting># scp helloworld root@&lt;target_ip_address&gt;:/tmp</programlisting></para>
+    </section>
+
+    <section id="deploy_onguest">
+      <title>Deploying on guest</title>
+
+      <para>You can deploy your artifacts onto the guest VM running on the
+      target, by using TAP Networking. You can use the
+      <filename>/etc/qemu-ifup</filename> script to create the tap interface
+      on the host and attach it to the existing virtual bridge
+      (<literal>virbr0</literal>). This bridge interface is created by the
+      <filename>libvirt</filename> library and can be used to connect to the
+      outside network. To be able to transfer files to the guest via
+      <literal>scp</literal>, port forwarding should be enabled on the host.
+      The script sets iptables rules to forward traffic from a host port to
+      the guest default SSH port (22).</para>
+
+      <para>Follow the steps below to create this setup:</para>
+
+      <orderedlist>
+        <listitem>
+          <para>On the host, run the <literal>qemu-ifup</literal> script
+          located in <literal>/etc</literal> directory:</para>
+
+          <para><programlisting># /etc/qemu-ifup &ndash;t tap0 &ndash;a 192.168.122.10 &ndash;p 1050 &ndash;g 22</programlisting></para>
+
+          <itemizedlist>
+            <listitem>
+              <para><emphasis role="bold">tap0</emphasis> - the tap interface
+              name which will be created and added to the virtual bridge
+              (virbr0).</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">192.168.122.10</emphasis> - the IP
+              address of the guest virtual network device. It has to be in the
+              same network with the IP address of the virbr0 interface.</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">1050</emphasis> - the host port
+              which is set to forward traffic from the host to the
+              guest.</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">22</emphasis> - the default SSH
+              guest port used in port forwarding.</para>
+            </listitem>
+          </itemizedlist>
+        </listitem>
+
+        <listitem>
+          <para>Launch the virtual machine specifying the newly created tap
+          interface:</para>
+
+          <para><programlisting>-device e1000,netdev=net0 \
+-netdev tap,id=net0,ifname=tap0,script=no,downscript=no</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>On the guest, after logging, configure the virtual network
+          device and set the default gateway as the <literal>virbr0</literal>
+          ip address:</para>
+
+          <para><programlisting>ip addr add 192.168.122.10/24 dev enp0s2
+ip link set enp0s2 up
+ip route add default via 192.168.122.1 dev enp0s2</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>Now you can use <literal>scp</literal> from your development
+          machine to deploy your artifacts on the guest, by giving the host
+          port for forwarding as a command parameter: <programlisting>scp -P 1050 helloworld root@target_ip_address:/tmp</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>On the host, after finishing the deployment session and
+          stopping the virtual machine, you can use the
+          <literal>qemu-ifdown</literal> script to clean up the configuration
+          on host. The following command will remove the tap interface and all
+          the iptables rules for the specific ip address: <programlisting># /etc/qemu-ifdown &ndash;t tap0 &ndash;a 192.168.122.10</programlisting>If
+          we need to remove only a particular port forwarding rule from
+          iptables, this should be run: <programlisting># /etc/qemu-ifdown &ndash;t tap0 &ndash;a 192.168.122.10 &ndash;p 1050 &ndash;g 22</programlisting></para>
+        </listitem>
+      </orderedlist>
+    </section>
+  </section>
+
+  <section id="crossdebugging">
+    <title>Cross-Debugging on Enea NFV Access</title>
+
+    <para>The cross-debugger (<literal>x86_64-enea-linux-gdb</literal>) is
+    created when installing the SDK on the development machine. It is helpful
+    for debugging both the kernel and user-applications. In order to perform
+    this task, we need the following tools to be available on the target
+    machine:</para>
 
-        <para>You can use <literal>ssh</literal> to deploy your artifacts on
-        the host target. For this you will need a network connection to the
-        target and to use <literal>scp</literal> to copy to the desired
-        location.</para>
+    <itemizedlist>
+      <listitem>
+        <para>Kgdb &ndash; for kernel cross-debugging</para>
       </listitem>
 
       <listitem>
-        <para>Deploying on guest</para>
-
-        <para>You can deploy your artifacts onto the guest VM running on the
-        target in two steps:</para>
-
-        <itemizedlist>
-          <listitem>
-            <para>Deploy the artifacts onto the target by using the method
-            described above or any other method.</para>
-          </listitem>
-
-          <listitem>
-            <para>On the target, copy the artifacts to the guest rootfs. For
-            this, you will need to shut down the guest VM, mount the file
-            system on the target, copy your files onto it, unmount it and then
-            restart the guest VM as usual.</para>
-          </listitem>
-        </itemizedlist>
+        <para>GDBServer &ndash; for application cross-debugging</para>
       </listitem>
-    </orderedlist>
+    </itemizedlist>
+
+    <para>The Host Development image provides both of these tools and has to
+    be booted on the target machine for cross-debugging sessions.</para>
+
+    <section id="ua_debug_host">
+      <title>User-application Cross-Debugging on Host</title>
+
+      <para>To debug a user-application on host, a TCP connection has to be
+      established between the host and development machines. GDBserver is the
+      program which runs on the target machine and allows you to run GDB on
+      your workstation. Below you can find how a simple
+      <filename>helloworld</filename> application can be debugged using
+      GDBServer and cross-gdb.</para>
+
+      <para>On target, launch the GDBServer, specifying how to communicate
+      with GDB and the name of your program:<programlisting># gdbserver :&lt;port_no&gt; /tmp/helloworld</programlisting>The
+      target will now be listening on the port given as a parameter to the
+      gdbserver. On the development machine, from the
+      <literal>&lt;sdkdir&gt;</literal>, start the cross-gdb:<programlisting># x86_64-enea-linux-gdb &lt;path_to_the_program&gt;/helloworld</programlisting>Connect
+      the GDB to the target: <programlisting>(gdb) target remote &lt;target_ip_address&gt;:&lt;port_no&gt;</programlisting>Now
+      remote debugging is started and the GDB commands are available to debug
+      your program from the target machine.</para>
+    </section>
+
+    <section id="us_debug_guest">
+      <title>User-application Cross-Debugging on Guest</title>
+
+      <para>To debug a user-application on guest, a TCP connection has to be
+      established between the host and development machines. Similarly, as
+      when deploying artifacts on guest, for a cross-debugging session, TAP
+      Networking is required.</para>
+
+      <para>A tap interface should be added to the existing virtual bridge
+      (<literal>virbr0</literal>), along with port forwarding rules created in
+      iptables. In order to do this, the script
+      <filename>/etc/qemu-ifup</filename> can pe used:</para>
+
+      <orderedlist>
+        <listitem>
+          <para>On the host, run the script <literal>qemu-ifup</literal>
+          located in <literal>/etc</literal> directory: <programlisting># /etc/qemu-ifup &ndash;t tap0 &ndash;a 192.168.122.10 &ndash;p 1051 &ndash;g 1025</programlisting></para>
+
+          <itemizedlist>
+            <listitem>
+              <para><emphasis role="bold">tap0</emphasis> - the tap interface
+              name which will be created and added to virtual bridge
+              (virbr0).</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">192.168.122.10</emphasis> - the IP
+              address of the guest virtual network device. It has to be in the
+              same network with the IP address of the virbr0 interface.</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">1051</emphasis> &ndash; the host
+              port which is set to forward traffic from the host to the guest
+              and is used by gdb target remote command.</para>
+            </listitem>
+
+            <listitem>
+              <para><emphasis role="bold">1025</emphasis>&ndash; the port used
+              by GDBServer on guest for listening.</para>
+            </listitem>
+          </itemizedlist>
+        </listitem>
+
+        <listitem>
+          <para>Launch the virtual machine, specifying the newly created tap
+          interface:<programlisting>-device e1000,netdev=net0 \
+-netdev tap,id=net0,ifname=tap0,script=no,downscript=no</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>On the guest, after logging, configure the virtual network
+          device and set the default gateway to virbr0 ip
+          address:<programlisting>ip addr add 192.168.122.10/24 dev enp0s2
+ip link set enp0s2 up
+ip route add default via 192.168.122.1 dev enp0s2</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>GDBserver is the program which runs on the guest VM and allows
+          you to run GDB on your workstation. On the guest, launch GBDserver
+          specifying how to communicate with GDB and the name of your program:
+          <programlisting># gdbserver :1025 /tmp/helloworld</programlisting>The
+          guest is now listening on port 1025, given as a parameter to the
+          gdbserver.</para>
+        </listitem>
+
+        <listitem>
+          <para>On the development machine, from the<filename>
+          &lt;sdkdir&gt;</filename>, start the cross-gdb:<programlisting># x86_64-enea-linux-gdb &lt;path_to_the_program&gt;/helloworld</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>Connect GDB to the target:<programlisting>(gdb) target remote &lt;target_ip_address&gt;:1051</programlisting>Now
+          remote debugging is started and the GDB commands are available to
+          debug your program from the guest VM.</para>
+        </listitem>
+
+        <listitem>
+          <para>On the host, after finishing the cross-debugging session and
+          stopping the virtual machine, you can use the
+          <filename>qemu-ifdown</filename> script to clean up the
+          configuration on host:<programlisting># /etc/qemu-ifdown -t tap0 -a 192.168.122.10</programlisting></para>
+        </listitem>
+      </orderedlist>
+    </section>
+
+    <section id="kernel_crossdebug">
+      <title>Kernel Cross-Debugging</title>
+
+      <para>In order to debug the kernel, a serial connection is required
+      between the development and target machines. Debugging commands will be
+      sent from your workstation to the target machine via a serial
+      port.</para>
+
+      <para>The KGDB kernel options are enabled in the Enea NFV Access Host
+      SDK image and the tool can be used in the following way:</para>
+
+      <itemizedlist>
+        <listitem>
+          <para>On target, once serial communication is established, configure
+          <literal>kgdboc</literal> after the kernel boots:<programlisting># echo ttyS0,115200 &gt; /sys/module/kgdboc/parameters/kgdboc</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>In order to connect to gdb via kgdboc, the kernel must first
+          be stopped:<programlisting># echo g &gt; /proc/sysrq-trigger</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>Close the console to the target, eg.: <command>Ctrl +
+          ]</command> for a telnet session.</para>
+        </listitem>
+
+        <listitem>
+          <para>On your development machine, start cross-gdb using the vmlinux
+          kernel image as a parameter. The image is located in
+          <filename>&lt;sdkdir&gt;/sysroots/corei7-64-enea-linux/boot/</filename>
+          and should be the same as the image found in the
+          <literal>/boot</literal> directory from the target.<programlisting># x86_64-enea-linux-gdb /
+./sysroots/corei7-64-enea-linux/boot/vmlinux-4.9.30-intel-pk-standard</programlisting></para>
+        </listitem>
+
+        <listitem>
+          <para>Connect GDB to the target machine using target command and the
+          serial device:<programlisting>(gdb) set remotebaud 115200
+(gdb) target remote /dev/ttyS0</programlisting></para>
+        </listitem>
+      </itemizedlist>
+
+      <para>The kernel can now be debugged in a similar manner as an
+      application program.</para>
+    </section>
   </section>
 </chapter>
\ No newline at end of file