summaryrefslogtreecommitdiffstats
path: root/documentation/sdk-manual/sdk-extensible.xml
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/sdk-manual/sdk-extensible.xml')
-rw-r--r--documentation/sdk-manual/sdk-extensible.xml608
1 files changed, 608 insertions, 0 deletions
diff --git a/documentation/sdk-manual/sdk-extensible.xml b/documentation/sdk-manual/sdk-extensible.xml
index f9f6260a90..3e11fc97d5 100644
--- a/documentation/sdk-manual/sdk-extensible.xml
+++ b/documentation/sdk-manual/sdk-extensible.xml
@@ -590,6 +590,614 @@
590 </section> 590 </section>
591</section> 591</section>
592 592
593<section id='sdk-a-closer-look-at-devtool-add'>
594 <title>A Closer Look at <filename>devtool add</filename></title>
595
596 <para>
597 The <filename>devtool add</filename> command automatically creates a
598 recipe based on the source tree with which you provide it.
599 Currently, the command has support for the following:
600 <itemizedlist>
601 <listitem><para>
602 Autotools (<filename>autoconf</filename> and
603 <filename>automake</filename>)
604 </para></listitem>
605 <listitem><para>
606 CMake
607 </para></listitem>
608 <listitem><para>
609 Scons
610 </para></listitem>
611 <listitem><para>
612 <filename>qmake</filename>
613 </para></listitem>
614 <listitem><para>
615 Plain <filename>Makefile</filename>
616 </para></listitem>
617 <listitem><para>
618 Out-of-tree kernel module
619 </para></listitem>
620 <listitem><para>
621 Binary package (i.e. "-b" option)
622 </para></listitem>
623 <listitem><para>
624 Node.js module through
625 <filename>npm</filename>
626 </para></listitem>
627 <listitem><para>
628 Python modules that use <filename>setuptools</filename>
629 or <filename>distutils</filename>
630 </para></listitem>
631 </itemizedlist>
632 </para>
633
634 <para>
635 Apart from binary packages, the determination of how a source tree
636 should be treated is automatic based on the files present within
637 that source tree.
638 For example, if a <filename>CMakeLists.txt</filename> file is found,
639 then the source tree is assumed to be using
640 CMake and is treated accordingly.
641 <note>
642 In most cases, you need to edit the automatically generated
643 recipe in order to make it build properly.
644 Typically, you would go through several edit and build cycles
645 until you can build the recipe.
646 Once the recipe can be built, you could use possible further
647 iterations to test the recipe on the target device.
648 </note>
649 </para>
650
651 <para>
652 The remainder of this section covers specifics regarding how parts
653 of the recipe are generated.
654 </para>
655
656 <section id='sdk-name-and-version'>
657 <title>Name and Version</title>
658
659 <para>
660 If you do not specify a name and version on the command
661 line, <filename>devtool add</filename> attempts to determine
662 the name and version of the software being built from
663 various metadata within the source tree.
664 Furthermore, the command sets the name of the created recipe
665 file accordingly.
666 If the name or version cannot be determined, the
667 <filename>devtool add</filename> command prints an error and
668 you must re-run the command with both the name and version
669 or just the name or version specified.
670 </para>
671
672 <para>
673 Sometimes the name or version determined from the source tree
674 might be incorrect.
675 For such a case, you must reset the recipe:
676 <literallayout class='monospaced'>
677 $ devtool reset -n <replaceable>recipename</replaceable>
678 </literallayout>
679 After running the <filename>devtool reset</filename> command,
680 you need to run <filename>devtool add</filename> again and
681 provide the name or the version.
682 </para>
683 </section>
684
685 <section id='sdk-dependency-detection-and-mapping'>
686 <title>Dependency Detection and Mapping</title>
687
688 <para>
689 The <filename>devtool add</filename> command attempts to
690 detect build-time dependencies and map them to other recipes
691 in the system.
692 During this mapping, the command fills in the names of those
693 recipes in the
694 <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>
695 value within the recipe.
696 If a dependency cannot be mapped, then a comment is placed in
697 the recipe indicating such.
698 The inability to map a dependency might be caused because the
699 naming is not recognized or because the dependency simply is
700 not available.
701 For cases where the dependency is not available, you must use
702 the <filename>devtool add</filename> command to add an
703 additional recipe to satisfy the dependency and then come
704 back to the first recipe and add its name to
705 <filename>DEPENDS</filename>.
706 </para>
707
708 <para>
709 If you need to add runtime dependencies, you can do so by
710 adding the following to your recipe:
711 <literallayout class='monospaced'>
712 RDEPENDS_${PN} += "dependency1 dependency2 ..."
713 </literallayout>
714 <note>
715 The <filename>devtool add</filename> command often cannot
716 distinguish between mandatory and optional dependencies.
717 Consequently, some of the detected dependencies might
718 in fact be optional.
719 When in doubt, consult the documentation or the configure
720 script for the software the recipe is building for further
721 details.
722 In some cases, you might find you can substitute the
723 dependency for an option to disable the associated
724 functionality passed to the configure script.
725 </note>
726 </para>
727 </section>
728
729 <section id='sdk-license-detection'>
730 <title>License Detection</title>
731
732 <para>
733 The <filename>devtool add</filename> command attempts to
734 determine if the software you are adding is able to be
735 distributed under a common open-source license and sets the
736 <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink>
737 value accordingly.
738 You should double-check this value against the documentation
739 or source files for the software you are building and update
740 that <filename>LICENSE</filename> value if necessary.
741 </para>
742
743 <para>
744 The <filename>devtool add</filename> command also sets the
745 <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink>
746 value to point to all files that appear to be license-related.
747 However, license statements often appear in comments at the top
748 of source files or within documentation.
749 Consequently, you might need to amend the
750 <filename>LIC_FILES_CHKSUM</filename> variable to point to one
751 or more of those comments if present.
752 Setting <filename>LIC_FILES_CHKSUM</filename> is particularly
753 important for third-party software.
754 The mechanism attempts to ensure correct licensing should you
755 upgrade the recipe to a newer upstream version in future.
756 Any change in licensing is detected and you receive an error
757 prompting you to check the license text again.
758 </para>
759
760 <para>
761 If the <filename>devtool add</filename> command cannot
762 determine licensing information, the
763 <filename>LICENSE</filename> value is set to "CLOSED" and the
764 <filename>LIC_FILES_CHKSUM</filename> vaule remains unset.
765 This behavior allows you to continue with development but is
766 unlikely to be correct in all cases.
767 Consequently, you should check the documentation or source
768 files for the software you are building to determine the actual
769 license.
770 </para>
771 </section>
772
773 <section id='sdk-adding-makefile-only-software'>
774 <title>Adding Makefile-Only Software</title>
775
776 <para>
777 The use of <filename>make</filename> by itself is very common
778 in both proprietary and open source software.
779 Unfortunately, Makefiles are often not written with
780 cross-compilation in mind.
781 Thus, <filename>devtool add</filename> often cannot do very
782 much to ensure that these Makefiles build correctly.
783 It is very common, for example, to explicitly call
784 <filename>gcc</filename> instead of using the
785 <filename>CC</filename> variable.
786 Usually, in a cross-compilation environment,
787 <filename>gcc</filename> is the compiler for the build host
788 and the cross-compiler is named something similar to
789 <filename>arm-poky-linux-gnueabi-gcc</filename> and might
790 require some arguments (e.g. to point to the associated sysroot
791 for the target machine).
792 </para>
793
794 <para>
795 When writing a recipe for Makefile-only software, keep the
796 following in mind:
797 <itemizedlist>
798 <listitem><para>
799 You probably need to patch the Makefile to use
800 variables instead of hardcoding tools within the
801 toolchain such as <filename>gcc</filename> and
802 <filename>g++</filename>.
803 </para></listitem>
804 <listitem><para>
805 The environment in which <filename>make</filename> runs
806 is set up with various standard variables for
807 compilation (e.g. <filename>CC</filename>,
808 <filename>CXX</filename>, and so forth) in a similar
809 manner to the environment set up by the SDK's
810 environment setup script.
811 One easy way to see these variables is to run the
812 <filename>devtool build</filename> command on the
813 recipe and then look in
814 <filename>oe-logs/run.do_compile</filename>.
815 Towards the top of this file you will see a list of
816 environment variables that are being set.
817 You can take advantage of these variables within the
818 Makefile.
819 </para></listitem>
820 <listitem><para>
821 If the Makefile sets a default for a variable using "=",
822 that default overrides the value set in the environment,
823 which is usually not desirable.
824 In this situation, you can either patch the Makefile
825 so it sets the default using the "?=" operator, or
826 you can alternatively force the value on the
827 <filename>make</filename> command line.
828 To force the value on the command line, add the
829 variable setting to
830 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'><filename>EXTRA_OEMAKE</filename></ulink>
831 within the recipe as follows:
832 <literallayout class='monospaced'>
833 EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'"
834 </literallayout>
835 In the above example, single quotes are used around the
836 variable settings as the values are likely to contain
837 spaces because required default options are passed to
838 the compiler.
839 </para></listitem>
840 <listitem><para>
841 Hardcoding paths inside Makefiles is often problematic
842 in a cross-compilation environment.
843 This is particularly true because those hardcoded paths
844 often point to locations on the build host and thus
845 will either be read-only or will introduce
846 contamination into the cross-compilation by virtue of
847 being specific to the build host rather than the target.
848 Patching the Makefile to use prefix variables or other
849 path variables is usually the way to handle this.
850 </para></listitem>
851 <listitem><para>
852 Sometimes a Makefile runs target-specific commands such
853 as <filename>ldconfig</filename>.
854 For such cases, you might be able to simply apply
855 patches that remove these commands from the Makefile.
856 </para></listitem>
857 </itemizedlist>
858 </para>
859 </section>
860
861 <section id='sdk-adding-native-tools'>
862 <title>Adding Native Tools</title>
863
864 <para>
865 Often, you need to build additional tools that run on the
866 build host system as opposed to the target.
867 You should indicate this using one of the following methods
868 when you run <filename>devtool add</filename>:
869 <itemizedlist>
870 <listitem><para>
871 Specify the name of the recipe such that it ends
872 with "-native".
873 Specifying the name like this produces a recipe that
874 only builds for the build host.
875 </para></listitem>
876 <listitem><para>
877 Specify the "&dash;&dash;also-native" option with the
878 <filename>devtool add</filename> command.
879 Specifying this option creates a recipe file that still
880 builds for the target but also creates a variant with
881 a "-native" suffix that builds for the build host.
882 </para></listitem>
883 </itemizedlist>
884 <note>
885 If you need to add a tool that is shipped as part of a
886 source tree that builds code for the target, you can
887 typically accomplish this by building the native and target
888 parts separately rather than within the same compilation
889 process.
890 Realize though that with the "&dash;&dash;also-native" option, you
891 can add the tool using just one recipe file.
892 </note>
893 </para>
894 </section>
895
896 <section id='sdk-adding-node-js-modules'>
897 <title>Adding Node.js Modules</title>
898
899 <para>
900 You can use the <filename>devtool add</filename> command in the
901 following form to add Node.js modules:
902 <literallayout class='monospaced'>
903 $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
904 </literallayout>
905 The name and version parameters are mandatory.
906 Lockdown and shrinkwrap files are generated and pointed to by
907 the recipe in order to freeze the version that is fetched for
908 the dependencies according to the first time.
909 This also saves checksums that are verified on future fetches.
910 Together, these behaviors ensure the reproducibility and
911 integrity of the build.
912 <note><title>Notes</title>
913 <itemizedlist>
914 <listitem><para>
915 You must use quotes around the URL.
916 The <filename>devtool add</filename> does not require
917 the quotes, but the shell considers ";" as a splitter
918 between multiple commands.
919 Thus, without the quotes,
920 <filename>devtool add</filename> does not receive the
921 other parts, which results in several "command not
922 found" errors.
923 </para></listitem>
924 <listitem><para>
925 In order to support adding
926 Node.js modules, a
927 <filename>nodejs</filename> recipe must be part of your
928 SDK in order to provide Node.js
929 itself.
930 </para></listitem>
931 </itemizedlist>
932 </note>
933 </para>
934 </section>
935</section>
936
937<section id='sdk-working-with-recipes'>
938 <title>Working With Recipes</title>
939
940 <para>
941 When building a recipe with <filename>devtool build</filename> the
942 typical build progression is as follows:
943 <orderedlist>
944 <listitem><para>
945 Fetch the source
946 </para></listitem>
947 <listitem><para>
948 Unpack the source
949 </para></listitem>
950 <listitem><para>
951 Configure the source
952 </para></listitem>
953 <listitem><para>
954 Compiling the source
955 </para></listitem>
956 <listitem><para>
957 Install the build output
958 </para></listitem>
959 <listitem><para>
960 Package the installed output
961 </para></listitem>
962 </orderedlist>
963 For recipes in the workspace, fetching and unpacking is disabled
964 as the source tree has already been prepared and is persistent.
965 Each of these build steps is defined as a function, usually with a
966 "do_" prefix.
967 These functions are typically shell scripts but can instead be written
968 in Python.
969 </para>
970
971 <para>
972 If you look at the contents of a recipe, you will see that the
973 recipe does not include complete instructions for building the
974 software.
975 Instead, common functionality is encapsulated in classes inherited
976 with the <filename>inherit</filename> directive, leaving the recipe
977 to describe just the things that are specific to the software to be
978 built.
979 A <ulink url='ref-classes-base'><filename>base</filename></ulink>
980 class exists that is implicitly inherited by all recipes and provides
981 the functionality that most typical recipes need.
982 </para>
983
984 <para>
985 The remainder of this section presents information useful when
986 working with recipes.
987 </para>
988
989 <section id='sdk-finding-logs-and-work-files'>
990 <title>Finding Logs and Work Files</title>
991
992 <para>
993 When you are debugging a recipe that you previously created using
994 <filename>devtool add</filename> or whose source you are modifying
995 by using the <filename>devtool modify</filename> command, after
996 the first run of <filename>devtool build</filename>, you will
997 find some symbolic links created within the source tree:
998 <filename>oe-logs</filename>, which points to the directory in
999 which log files and run scripts for each build step are created
1000 and <filename>oe-workdir</filename>, which points to the temporary
1001 work area for the recipe.
1002 You can use these links to get more information on what is
1003 happening at each build step.
1004 </para>
1005
1006 <para>
1007 These locations under <filename>oe-workdir</filename> are
1008 particularly useful:
1009 <itemizedlist>
1010 <listitem><para><filename>image/</filename>:
1011 Contains all of the files installed at the
1012 <ulink url='&YOCTO_DOCS_REF_URL;ref-tasks-install'><filename>do_install</filename></ulink>
1013 stage.
1014 Within a recipe, this directory is referred to by the
1015 expression
1016 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>.
1017 </para></listitem>
1018 <listitem><para><filename>sysroot-destdir/</filename>:
1019 Contains a subset of files installed within
1020 <filename>do_install</filename> that have been put into the
1021 shared sysroot.
1022 For more information, see the
1023 "<link linkend='sdk-sharing-files-between-recipes'>Sharing Files Between Recipes</link>"
1024 section.
1025 </para></listitem>
1026 <listitem><para><filename>packages-split/</filename>:
1027 Contains subdirectories for each package produced by the
1028 recipe.
1029 For more information, see the
1030 "<link linkend='sdk-packaging'>Packaging</link>" section.
1031 </para></listitem>
1032 </itemizedlist>
1033 </para>
1034 </section>
1035
1036 <section id='sdk-setting-configure-arguments'>
1037 <title>Setting Configure Arguments</title>
1038
1039 <para>
1040 If the software your recipe is building uses GNU autoconf,
1041 then a fixed set of arguments is passed to it to enable
1042 cross-compilation plus any extras specified by
1043 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink>
1044 set within the recipe.
1045 If you wish to pass additional options, add them to
1046 <filename>EXTRA_OECONF</filename>.
1047 Other supported build tools have similar variables
1048 (e.g.
1049 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE'><filename>EXTRA_OECMAKE</filename></ulink>
1050 for CMake,
1051 <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OESCONS'><filename>EXTRA_OESCONS</filename></ulink>
1052 for Scons, and so forth).
1053 If you need to pass anything on the <filename>make</filename>
1054 command line, you can use <filename>EXTRA_OEMAKE</filename> to do
1055 so.
1056 </para>
1057
1058 <para>
1059 You can use the <filename>devtool configure-help</filename> command
1060 to help you set the arguments listed in the previous paragraph.
1061 The command determines the exact options being passed, and shows
1062 them to you along with any custom arguments specified through
1063 <filename>EXTRA_OECONF</filename>.
1064 If applicable, the command also shows you the output of the
1065 configure script's "&dash;&dash;help" option as a reference.
1066 </para>
1067 </section>
1068
1069 <section id='sdk-sharing-files-between-recipes'>
1070 <title>Sharing Files Between Recipes</title>
1071
1072 <para>
1073 Recipes often need to use files provided by other recipes on
1074 the build host.
1075 For example, an application linking to a common library needs
1076 access to the library itself and its associated headers.
1077 The way this access is accomplished within the extensible SDK is
1078 through the sysroot.
1079 One sysroot exists per "machine" for which the SDK is being built.
1080 In practical terms, this means a sysroot exists for the target
1081 machine, and a sysroot exists for the build host.
1082 </para>
1083
1084 <para>
1085 Recipes should never write files directly into the sysroot.
1086 Instead, files should be installed into standard locations
1087 during the
1088 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
1089 task within the
1090 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>
1091 directory.
1092 A subset of these files automatically go into the sysroot.
1093 The reason for this limitation is that almost all files that go
1094 into the sysroot are cataloged in manifests in order to ensure
1095 they can be removed later when a recipe is modified or removed.
1096 Thus, the sysroot is able to remain free from stale files.
1097 </para>
1098 </section>
1099
1100 <section id='sdk-packaging'>
1101 <title>Packaging</title>
1102
1103 <para>
1104 Packaging is not always particularly relevant within the
1105 extensible SDK.
1106 However, if you examine how build output gets into the final image
1107 on the target device, it is important to understand packaging
1108 because the contents of the image are expressed in terms of
1109 packages and not recipes.
1110 </para>
1111
1112 <para>
1113 During the
1114 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink>
1115 task, files installed during the
1116 <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>
1117 task are split into one main package, which is almost always named
1118 the same as the recipe, and several other packages.
1119 This separation is done because not all of those installed files
1120 are always useful in every image.
1121 For example, you probably do not need any of the documentation
1122 installed in a production image.
1123 Consequently, for each recipe the documentation files are separated
1124 into a <filename>-doc</filename> package.
1125 Recipes that package software that has optional modules or
1126 plugins might do additional package splitting as well.
1127 </para>
1128
1129 <para>
1130 After building a recipe you can see where files have gone by
1131 looking in the <filename>oe-workdir/packages-split</filename>
1132 directory, which contains a subdirectory for each package.
1133 Apart from some advanced cases, the
1134 <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>
1135 and
1136 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>
1137 variables controls splitting.
1138 The <filename>PACKAGES</filename> variable lists all of the
1139 packages to be produced, while the <filename>FILES</filename>
1140 variable specifies which files to include in each package,
1141 using an override to specify the package.
1142 For example, <filename>FILES_${PN}</filename> specifies the files
1143 to go into the main package (i.e. the main package is named the
1144 same as the recipe and
1145 <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
1146 evaluates to the recipe name).
1147 The order of the <filename>PACKAGES</filename> value is significant.
1148 For each installed file, the first package whose
1149 <filename>FILES</filename> value matches the file is the package
1150 into which the file goes.
1151 Defaults exist for both the <filename>PACKAGES</filename> and
1152 <filename>FILES</filename> variables.
1153 Consequently, you might find you do not even need to set these
1154 variables in your recipe unless the software the recipe is
1155 building installs files into non-standard locations.
1156 </para>
1157 </section>
1158</section>
1159
1160<section id='sdk-restoring-the-target-device-to-its-original-state'>
1161 <title>Restoring the Target Device to its Original State</title>
1162
1163 <para>
1164 If you use the <filename>devtool deploy-target</filename>
1165 command to write a recipe's build output to the target, and
1166 you are working on an existing component of the system, then you
1167 might find yourself in a situation where you need to restore the
1168 original files that existed prior to running the
1169 <filename>devtool deploy-target</filename> command.
1170 Because the <filename>devtool deploy-target</filename> command
1171 backs up any files it overwrites, you can use the
1172 <filename>devtool undeploy-target</filename> to restore those files
1173 and remove any other files the recipe deployed.
1174 Consider the following example:
1175 <literallayout class='monospaced'>
1176 $ devtool undeploy-target lighttpd root@192.168.7.2
1177 </literallayout>
1178 If you have deployed multiple applications, you can remove them
1179 all at once thus restoring the target device back to its
1180 original state:
1181 <literallayout class='monospaced'>
1182 $ devtool undeploy-target -a root@192.168.7.2
1183 </literallayout>
1184 Information about files deployed to the target as well as any
1185 backed up files are stored on the target itself.
1186 This storage of course requires some additional space
1187 on the target machine.
1188 <note>
1189 The <filename>devtool deploy-target</filename> and
1190 <filename>devtool undeploy-target</filename> command do not
1191 currently interact with any package management system on the
1192 target device (e.g. RPM or OPKG).
1193 Consequently, you should not intermingle operations
1194 <filename>devtool deploy-target</filename> and the package
1195 manager operations on the target device.
1196 Doing so could result in a conflicting set of files.
1197 </note>
1198 </para>
1199</section>
1200
593<section id='sdk-installing-additional-items-into-the-extensible-sdk'> 1201<section id='sdk-installing-additional-items-into-the-extensible-sdk'>
594 <title>Installing Additional Items Into the Extensible SDK</title> 1202 <title>Installing Additional Items Into the Extensible SDK</title>
595 1203