From d4bdafa8dfc26f75895117fbef82d03675e6c145 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Sat, 26 Mar 2016 11:10:13 -0700 Subject: sdk-manual: Added sections in Appendix B. (From yocto-docs rev: 955f08c8d49fabd6022570e4d0a7442f06f6049b) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../sdk-manual/sdk-appendix-customizing.xml | 607 +++++++++++++++++++++ 1 file changed, 607 insertions(+) (limited to 'documentation') diff --git a/documentation/sdk-manual/sdk-appendix-customizing.xml b/documentation/sdk-manual/sdk-appendix-customizing.xml index d38320f8bb..9fca16571f 100644 --- a/documentation/sdk-manual/sdk-appendix-customizing.xml +++ b/documentation/sdk-manual/sdk-appendix-customizing.xml @@ -383,6 +383,613 @@ +
+ A Closer Look at <filename>devtool add</filename> + + + The devtool add command automatically creates a + recipe based on the source tree with which you provide it. + Currently, the command has support for the following: + + + Autotools (autoconf and + automake) + + + CMake + + + Scons + + + qmake + + + Plain Makefile + + + Out-of-tree kernel module + + + Binary package (i.e. "-b" option) + + + Node.js module through + npm + + + Python modules that use setuptools + or distutils + + + + + + Apart from binary packages, the determination of how a source tree + should be treated is automatic based on the files present within + that source tree. + For example, if a CMakeLists.txt file is found, + then the source tree is assumed to be using + CMake and is treated accordingly. + + In most cases, you need to edit the automatically generated + recipe in order to make it build properly. + Typically, you would go through several edit and build cycles + until you can build the recipe. + Once the recipe can be built, you could use possible further + iterations to test the recipe on the target device. + + + + + The remainder of this section covers specifics regarding how parts + of the recipe are generated. + + +
+ Name and Version + + + If you do not specify a name and version on the command + line, devtool add attempts to determine + the name and version of the software being built from + various metadata within the source tree. + Furthermore, the command sets the name of the created recipe + file accordingly. + If the name or version cannot be determined, the + devtool add command prints an error and + you must re-run the command with both the name and version + or just the name or version specified. + + + + Sometimes the name or version determined from the source tree + might be incorrect. + For such a case, you must run the following commands: + + $ devtool reset -n recipename + + After running the devtool reset command, + you need to run devtool add again and + provide the name or the version. + +
+ +
+ Dependency Detection and Mapping + + + The devtool add command attempts to + detect build-time dependencies and map them to other recipes + in the system. + During this mapping, the command fills in the names of those + recipes in the + DEPENDS + value within the recipe. + If a dependency cannot be mapped, then a comment is placed in + the recipe indicating such. + The inability to map a dependency might be caused because the + naming is not recognized or because the dependency simply is + not available. + For cases where the dependency is not available, you must use + the devtool add command to add an + additional recipe to satisfy the dependency and then come + back to the first recipe and add its name to + DEPENDS. + + + + If you need to add runtime dependencies, you can do so by + adding the following to your recipe: + + RDEPENDS_${PN} += "dependency1 dependency2 ..." + + + The devtool add command often cannot + distinguish between mandatory and optional dependencies. + Consequently, some of the detected dependencies might + in fact be optional. + When in doubt, consult the documentation or the configure + script for the software the recipe is building for further + details. + In some cases, you might find you can substitute the + dependency for an option to disable the associated + functionality passed to the configure script. + + +
+ +
+ License Detection + + + The devtool add command attempts to + determine if the software you are adding is able to be + distributed under a common open-source license and sets the + LICENSE + value accordingly. + You should double-check this value against the documentation + or source files for the software you are building and update + that LICENSE value if necessary. + + + + The devtool add command also sets the + LIC_FILES_CHKSUM + value to point to all files that appear to be license-related. + However, license statements often appear in comments at the top + of source files or within documentation. + Consequently, you might need to amend the + LIC_FILES_CHKSUM variable to point to one + or more of those comments if present. + Setting LIC_FILES_CHKSUM is particularly + important for third-party software. + The command attempts to ensure correct licensing should you + upgrade the recipe to a newer upstream version in future. + Any change in licensing is detected and you receive an error + prompting you to check the license text again. + + + + If the devtool add command cannot + determine licensing information, the + LICENSE value is set to "CLOSED" and the + LIC_FILES_CHKSUM vaule remains unset. + This behavior allows you to continue with development but is + unlikely to be correct in all cases. + Consequently, you should check the documentation or source + files for the software you are building to determine the actual + license. + +
+ +
+ Adding Makefile-Only Software + + + The use of make by itself is very common + in both proprietary and open source software. + Unfortunately, Makefiles are often not written with + cross-compilation in mind. + Thus, devtool add often cannot do very + much to ensure that these Makefiles build correctly. + It is very common, for example, to explicitly call + gcc instead of using the + CC variable. + Usually, in a cross-compilation environment, + gcc is the compiler for the build host + and the cross-compiler is named something similar to + arm-poky-linux-gnueabi-gcc and might + require some arguments (e.g. to point to the associated sysroot + for the target machine). + + + + When writing a recipe for Makefile-only software, keep the + following in mind: + + + You probably need to patch the Makefile to use + variables instead of hardcoding tools within the + toolchain such as gcc and + g++. + + + The environment in which make runs + is set up with various standard variables for + compilation (e.g. CC, + CXX, and so forth) in a similar + manner to the environment set up by the SDK's + environment setup script. + One easy way to see these variables is to run the + devtool build command on the + recipe and then look in + oe-logs/run.do_compile. + Towards the top of this file you will see a list of + environment variables that are being set. + You can take advantage of these variables within the + Makefile. + + + If the Makefile sets a default for a variable, that + default overrides the value set in the environment, + which is usually not desirable. + In this situation, you can either patch the Makefile + so it sets the default using the "?=" operator, or + you can alternatively force the value on the + make command line. + To force the value on the command line, add the + variable setting to + EXTRA_OEMAKE + within the recipe as follows: + + EXTRA_OEMAKE += "'CC=${CC}' 'CXX=${CXX}'" + + In the above example, single quotes are used around the + variable settings as the values are likely to contain + spaces because required default options are passed to + the compiler. + + + Hardcoding paths inside Makefiles is often problematic + in a cross-compilation environment. + This is particularly true because those hardcoded paths + often point to locations on the build host and thus + will either be read-only or will introduce + contamination into the cross-compilation by virtue of + being specific to the build host rather than the target. + Patching the Makefile to use prefix variables or other + path variables is usually the way to handle this. + + + Sometimes a Makefile runs target-specific commands such + as ldconfig. + For such cases, you might be able to simply apply + patches that remove these commands from the Makefile. + + + +
+ +
+ Adding Native Tools + + + Often, you need to build additional tools that run on the + build host system as opposed to the target. + You should indicate this using one of the following methods + when you run devtool add: + + + Specify the name of the recipe such that it ends + with "-native". + Specifying the name like this produces a recipe that + only builds for the build host. + + + Specify the "‐‐also-native" option with the + devtool add command. + Specifying this option creates a recipe file that still + builds for the target but also creates a variant with + a "-native" suffix that builds for the build host. + + + + If you need to add a tool that is shipped as part of a + source tree that builds code for the target, you can + typically accomplish this by building the native and target + parts separately rather than within the same compilation + process. + Realize though that with the "‐‐also-native" option, you + can add the tool using just one recipe file. + + +
+ +
+ Adding <filename>Node.js</filename> Modules + + + You can use the devtool add command in the + following form to add Node.js modules: + + $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1" + + The name and version parameters are mandatory. + Lockdown and shrinkwrap files are generated and pointed to by + the recipe in order to freeze the version that is fetched for + the dependencies according to the first time. + This also saves checksums that are verified on future fetches. + Together, these behaviors ensure the reproducibility and + integrity of the build. + Notes + + + You must use quotes around the URL. + The devtool add does not require + the quotes, but the shell considers ";" as a splitter + between multiple commands. + Thus, devtool add does not receive + the other parts resulting in several "command not found" + errors. + + + In order to support adding + Node.js modules, a + nodejs recipe must be part of your + SDK in order to provide Node.js + itself. + + + + +
+
+ +
+ Working With Recipes + + + When building a recipe with devtool build the + typical workflow is as follows: + + + Fetch the source + + + Unpack the source + + + Configure the source + + + Compiling the source + + + Install the build output + + + Package the installed output + + + For recipes in the workspace, fetching and unpacking is disabled + as the source tree has already been prepared and is persistent. + Each of these build steps is defined as a function, usually with a + "do_" prefix. + These functions are typically shell scripts but can instead be written + in Python. + + + + If you look at the contents of a recipe, you will see that the + recipe does not include complete instructions for building the + software. + Instead, common functionality is encapsulated in classes inherited + with the inherit directive, leaving the recipe + to describe just the things that are specific to the software to be + built. + A base + class exists that is implicitly inherited by all recipes and provides + the functionality that most typical recipes need. + + + + The remainder of this section presents information useful when + working with recipes. + + +
+ Finding Logs and Work Files + + + When you are debugging a recipe that you previously created using + devtool add or whose source you are modifying + by using the devtool modify command, after + the first run of devtool build, you will + find some symbolic links created within the source tree: + oe-logs, which points to the directory in + which log files and run scripts for each build step are created + and oe-workdir, which points to the temporary + work area for the recipe. + You can use these links to get more information on what is + happening at each build step. + + + + These locations under oe-workdir are + particularly useful: + + image/: + Contains all of the files installed at the + do_install + stage. + Within a recipe, this directory is referred to by the + expression + ${D}. + + sysroot-destdir/: + Contains a subset of files installed within + do_install that have been put into the + shared sysroot. + For more information, see the + "Sharing Files Between Recipes" + section. + + packages-split/: + Contains subdirectories for each package produced by the + recipe. (more on "Packaging" below) + For more information, see the + "Packaging" section. + + + +
+ +
+ Setting Configure Arguments + + + If the software your recipe is building uses GNU autoconf, + then a fixed set of arguments is passed to it to enable + cross-compilation plus any extras specified by + EXTRA_OECONF + set within the recipe. + If you wish to pass additional options, add them to + EXTRA_OECONF. + Other supported build tools have similar variables + (e.g. + EXTRA_OECMAKE + for CMake, + EXTRA_OESCONS + for Scons, and so forth). + If you need to pass anything on the make + command line, you can use EXTRA_OEMAKE to do + so. + + + + You can use the devtool configure-help command + to help you set the arguments listed in the previous paragraph. + The command determines the exact options being passed, and shows + them to you along with any custom arguments specified through + EXTRA_OECONF. + If applicable, the command also shows you the output of the + configure script's "‐‐help" option as a reference. + +
+ +
+ Sharing Files Between Recipes + + + Recipes often need to use files provided by other recipes on + the build host. + For example, an application linking to a common library needs + access to the library itself and its associated headers. + The way this access is accomplished within the extensible SDK is + through the sysroot. + One sysroot exists per "machine" for which the SDK is being built. + In practical terms, this means a sysroot exists for the target + machine, and a sysroot exists for the build host. + + + + Recipes should never write files directly into the sysroot. + Instead, files should be installed into standard locations + during the + do_install + task within the + ${D} + directory. + A subset of these files automatically go into the sysroot. + The reason for this limitation is that almost all files that go + into the sysroot are cataloged in manifests in order to ensure + they can be removed later when a a recipe is modified or removed. + Thus, the sysroot is able to remain free from stale files. + +
+ +
+ Packaging + + + Packaging is not always particularly relevant within the + extensible SDK. + However, if you examine build output gets into the final image on + the target device, it is important to understand packaging + because the contents of the image are expressed in terms of + packages ... not recipes. + + + + During the + do_package + task, files installed during the + do_install + task are split into one main package, which is almost always named + the same as the recipe, and several other packages. + This separation is done because not all of those installed files + are always useful in every image. + For example, you probably do not need any of the documentation + installed in a production image. + Consequently, for each recipe the documentation files are separated + into a -doc package. + Recipes that package software that has optional modules or + plugins might do additional package splitting as well. + + + + After building a recipe you can see where files have gone by + looking in the oe-workdir/packages-split + directory, which contains a subdirectory for each package. + Apart from some advanced cases, the + PACKAGES + and + FILES + variables controls splitting. + The PACKAGES variable lists all of the + packages to be produced, while the FILES + variable specifies which files to include in each package, + using an override to specify the package. + For example, FILES_${PN} specifies the files + to go into the main package (i.e. the main package is named the + same as the recipe and + ${PN} + evaluates to the recipe name). + The order of the PACKAGES value is significant. + For each installed file, the first package whose + FILES value matches the file is the package + into which the file goes. + Defaults exist for both the PACKAGES and + FILES variables. + Consequently, you might find you do not even need to set these + variables in your recipe unless the software the recipe is + building installs files into non-standard locations. + +
+ +
+ Restoring the Target Device to its Original State + + + If you use the devtool deploy-target + command to write a recipe's build output to the target, and + you are working on an existing component of the system, then you + might find yourself in a situation where you need to restore the + original files that existed prior to running the + devtool deploy-target command. + Because the devtool deploy-target command + backs up any files it overwrites, you can use the + devtool undeploy-target to restore those files + and remove any other files the recipe deployed. + Consider the following example: + + $ devtool undeploy-target lighttpd root@192.168.7.2 + + If you have deployed multiple applications, you can remove them + all at once thus restoring the target device back to its + original state: + + $ devtool undeploy-target -a root@192.168.7.2 + + Information about files deployed to the target as well as any + backed up files are stored on the target itself. + This storage of course requires some additional space + on the target machine. + + The devtool deploy-target and + devtool undeploy-target command do not + currently interact with any package management system on the + target device (e.g. RPM or OPKG). + Consequently, you should not intermingle operations + devtool deploy-target and the package + manager operations on the target device. + Doing so could result in a conflicting set of files. + + +
+
+