From 4a1ea28ce9103c4f243669cbb7250d33eb97d676 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Mon, 26 Jan 2015 07:12:53 -0800 Subject: documentation: First draft devtools. Completed draft of the new devtool material. This commit affects some linking and figures across the YP doc set. * Added the build-workspace-directory.png file to both the dev-manual and the mega-manual. * Updated the Makefile to include the new figure in the TARFILE for dev-manual and mega-manual. * Re-did the section organization to pull the quilt and finding source file out of the "Modifying Source Files" section. (From yocto-docs rev: 4585b9fbb1636ae3c65e2e3ad2d1d6baca636e59) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../dev-manual/dev-manual-common-tasks.xml | 8 +- documentation/dev-manual/dev-manual-model.xml | 872 ++++++++++++++++----- .../figures/build-workspace-directory.png | Bin 0 -> 24458 bytes 3 files changed, 668 insertions(+), 212 deletions(-) create mode 100644 documentation/dev-manual/figures/build-workspace-directory.png (limited to 'documentation/dev-manual') diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index e5526ef65e..b8669f8daa 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -4838,7 +4838,7 @@ ${S}/linux - See the "Finding the Temporary Source Code" + See the "Finding Temporary Source Code" section and the S variable for more information about where source is kept during a build. @@ -4857,8 +4857,8 @@ Two methods exist by which you can create the patch: - Git workflow and - Quilt workflow. + devtool and + Quilt. For kernel patches, the Git workflow is more appropriate. This section assumes the Git workflow and shows the steps specific to this example. @@ -9104,7 +9104,7 @@ File Makefile.am added to patch patches/parallelmake.patch For more information on using Quilt, see the - "Using a Quilt Workflow" + "Using Quilt in Your Workflow" section. diff --git a/documentation/dev-manual/dev-manual-model.xml b/documentation/dev-manual/dev-manual-model.xml index 59d6adb3fc..15b815d417 100644 --- a/documentation/dev-manual/dev-manual-model.xml +++ b/documentation/dev-manual/dev-manual-model.xml @@ -1242,7 +1242,7 @@ aclocal, autoconf, autoheader, - automake --a, and + automake ‐‐a, and ./configure. Click on the "Console" tab beneath your source code to see the results of reconfiguring your project. @@ -1668,104 +1668,597 @@ -
- Modifying Temporary Source Code +
+ Modifying Source Code - You might - find it helpful during development to modify the temporary source code used by recipes - to build packages. - For example, suppose you are developing a patch and you need to experiment a bit - to figure out your solution. - After you have initially built the package, you can iteratively tweak the - source code, which is located in the - Build Directory, and then - you can force a re-compile and quickly test your altered code. - Once you settle on a solution, you can then preserve your changes in the form of - patches. - You can accomplish these steps all within either a - Quilt or - Git workflow. + A common development workflow consists of modifying project source + files that are external to the Yocto Project and then integrating + that project's build output into an image built using the Yocto + Project. + Given this scenario, development engineers typically want to stick + to their familiar project development tools and methods, which allows + them to just focus on the project. -
- Finding the Temporary Source Code + + Several workflows exist that allow you to develop, build, and test + code that is going to be integrated into an image built using the + Yocto Project. + This section describes two: + + devtool: + A set of tools and + enhancements put together by the Yocto Project team that + makes it easier for you to modify code that is external to + the Yocto Project. + Section + "Using devtool in Your Workflow" + describes this workflow. + If you want more information that showcases the workflow, click + here + for an excellent presentation by Trevor Woerner that + provides detailed background information and a complete + working tutorial. + + Quilt: + A powerful tool that allows you to capture source + code changes without having a clean source tree. + While Quilt is not the preferred workflow of the two, this + section includes it for users that are committed to using + the tool. + See the + "Using Quilt in Your Workflow" + section for more information. + + + + +
+ Using <filename>devtool</filename> in Your Workflow - During a build, the unpacked temporary source code used by recipes - to build packages is available in the Build Directory as - defined by the - S variable. - Below is the default value for the S variable as defined in the - meta/conf/bitbake.conf configuration file in the - Source Directory: - - S = "${WORKDIR}/${BP}" - - You should be aware that many recipes override the S variable. - For example, recipes that fetch their source from Git usually set - S to ${WORKDIR}/git. + As mentioned earlier, devtool helps + you easily develop projects whose build output must be part of + an image built using the Yocto Project. + The remainder of this section presents the workflow you would + use that includes devtool. - The - BP - represents the base recipe name, which consists of the name and version: - - BP = "${BPN}-${PV}" - + The workflow considers the entire build process for the + image and not just modification of the external source + code. - - The path to the work directory for the recipe - (WORKDIR) - is defined as follows: - - ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} - - The actual directory depends on several things: - - TMPDIR: - The top-level build output directory - MULTIMACH_TARGET_SYS: - The target system identifier - PN: - The recipe name - EXTENDPE: - The epoch - (if - PE - is not specified, which is usually the case for most - recipes, then EXTENDPE is blank) - PV: - The recipe version - PR: - The recipe revision - - +
+ Establish the Reference Image - - As an example, assume a Source Directory top-level folder - name poky, a default Build Directory at - poky/build, and a - qemux86-poky-linux machine target - system. - Furthermore, suppose your recipe is named - foo_1.3.0.bb. - In this case, the work directory the build system uses to - build the package would be as follows: - - poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 - - + + Local repositories for both the Yocto Project and your + project must exist in addition to the image built by + the OpenEmbedded build system. + The steps to clone the poky Git + repository, build out an image, and test it using QEMU + are well documented as follows: + + + For information on how to set up a local copy of the + poky repository and on how to + build a Yocto Project image, see the + "Building an Image" + section in the Yocto Project Quick Start. + + + For information on how to test an image using QEMU, see + the + "Using the Quick EMUlator (QEMU)" + section. + + + + + + Before you start making modifications to your project's + source code, you should be sure you have the appropriate + local repositories and have a base image built using + BitBake that you can run on QEMU. + +
+ +
+ Update Your External Source + + + Part of the development flow using + devtool of course involves updating + your source files. + Several opportunities exist in the workflow to extract and + work on the files. + For now, just realize that you need to be able to have + access to and edit files in your layer. + One obvious solution is to initially extract the code into its + own layer in preparation for modification. + + + + Another option is to use the devtool + command. + devtool makes use of a + "workspace" layer where much of the transitional work + occurs, which is needed for setting up Metadata used by the + OpenEmbedded build system that lets you build your software. + Options exist using devtool that + enable you to use the tool to extract source code. + +
+ +
+ Use <filename>dev-tool</filename> to Integrate Your Code with the Image + + + devtool automatically + generates the needed Metadata that allows the OpenEmbedded + build system to build your code into the image. + Use the following command form: + + $ devtool add your-project-name path-to-source/your-project-name + + Running devtool modifies the + bblayers.conf used to build the + image in the Yocto Project. + For more information on the bblayers.conf, + see the + "BSP Layers" + section in the Yocto Project Board Support Package (BSP) Developer's Guide. + + + + Running devtool adds a new workspace + layer to the bblayers.conf file that + is based on your project's location: + + path-to-source/build-directory/workspace-layer + + By default, the name of the workspace layer is "workplace". + + + + For details on the workspace layer created in the + build-directory, + see the + "Adding a New Recipe to the Workspace Layer" + section. + + + + Of course, each layer must have a + layer.conf configuration file. + devtool also creates this configuration + file: + + $ cat workspace/conf/layer.conf + # ### workspace layer auto­generated by devtool ### + BBPATH =. "${LAYERDIR}:" + BBFILES += "${LAYERDIR}/recipes/*/*.bb \ + ${LAYERDIR}/appends/*.bbappend" + BBFILE_COLLECTIONS += "workspacelayer" + BBFILE_PATTERN_workspacelayer = "^${LAYERDIR}/" + BBFILE_PATTERN_IGNORE_EMPTY_workspacelayer = "1" + BBFILE_PRIORITY_workspacelayer = "99" + + + + + Running devtool also automatically + generates your recipe: + + $ cat workspace/recipes/your-project-name/your-project-name.bb + # Recipe created by recipetool + # This is the basis of a recipe and may need further editing in order to be fully functional. + # (Feel free to remove these comments when editing.) + # + # Unable to find any files that looked like license statements. Check the accompanying + # documentation and source headers and set LICENSE and LIC_FILES_CHKSUM accordingly. + LICENSE = "CLOSED" + LIC_FILES_CHKSUM = "" + + # No information for SRC_URI yet (only an external source tree was + # specified) + SRC_URI = "" + + DEPENDS = "libx11" + # NOTE: if this software is not capable of being built in a separate build directory + # from the source, you should replace autotools with autotools­-brokensep in the + # inherit line + inherit autotools + + # Specify any options you want to pass to the configure script using EXTRA_OECONF: + EXTRA_OECONF = "" + + + + + Lastly, the devtool creates the + .bbappend file: + + $ cat workspace/appends/your-project-name.bbappend + inherit externalsrc + EXTERNALSRC = "/path-to-source/your-project-name" + + # initial_rev: commit-ID + + +
+ +
+ Build Your Project + + + You can use BitBake or devtool to build + your modified project. + + + + To use BitBake, use the following: + + $ bitbake your-project-name + + To use devtool, run the tool with the + build option: + + $ devtool build your-project-name + + +
+ +
+ Build the Image + + + The final step before testing is to rebuild the base image + with your project changes as part of the image. + Simply run BitBake again on your image. + Here is an example that assumes + MACHINE + is set to "qemux86": + + $ bitbake qemux86 image + + +
+ +
+ Testing the Image + + + Once you have the image, you can test it using QEMU. + Here is an example assuming "qemux86": + + $ runqemu qemux86 image + + For information on how to test an image using QEMU, see the + "Using the Quick EMUlator (QEMU)" + section. + +
+
+ +
+ <filename>devtool</filename> Quick Reference - Now that you know where to locate the directory that has the temporary source code, - you can use a Quilt or Git workflow to make your edits, test the changes, - and preserve the changes in the form of patches. + devtool has more functionality than simply + adding a new recipe and the supporting Metadata to a temporary + workspace layer. + This section provides a short reference on + devtool for most of the commands. + +
+ Getting Help + + + The easiest way to get help with the + devtool command is using the + ‐‐help option: + + $ devtool ‐‐help + usage: devtool [-h] [‐‐basepath BASEPATH] [-d] [-q] + [‐‐color {auto,always,never}] + {create-workspace,deploy-target,undeploy-target,add,modify,extract,update-recipe,status,build,reset} + ... + + OpenEmbedded development tool + + positional arguments: + {create-workspace,deploy-target,undeploy-target,add,modify,extract,update-recipe,status,build,reset} + create-workspace Set up a workspace + deploy-target Deploy recipe output files to live target machine + undeploy-target Undeploy recipe output files in live target machine + add Add a new recipe + modify Modify the source for an existing recipe + extract Extract the source for an existing recipe + update-recipe Apply changes from external source tree to recipe + status Show status + build Build recipe + reset Remove a recipe from your workspace + + optional arguments: + -h, ‐‐help show this help message and exit + ‐‐basepath BASEPATH Base directory of SDK / build directory + -d, ‐‐debug Enable debug output + -q, ‐‐quiet Print only errors + ‐‐color {auto,always,never} + Colorize output + + Use devtool <command> ‐‐help to get help on a specific command + + + + + As directed in the general help output, you can get more + syntax on a specific command by providing the command + name and using ‐‐help: + + $ devtool add ‐‐help + usage: devtool add [-h] [‐‐version VERSION] recipename srctree + + positional arguments: + recipename Name for new recipe to add + srctree Path to external source tree + + optional arguments: + -h, ‐‐help show this help message and exit + ‐‐version VERSION, -V VERSION + + +
+ +
+ Adding a New Recipe to the Workspace Layer + + + Use the add command to add a new recipe + to the workspace layer. + The recipe you add should not exist - + devtool creates it for you. + The source files the recipe uses should exist in an external + area. + + + + The following example creates and adds a new recipe named + jackson-2.0 to the workspace layer. + The source code built by the recipes resides in + /home/scottrif/sources/jackson: + + $ devtool add jackson-2.0 /home/scottrif/sources/jackson + + + For complete syntax, use the + devtool add ‐‐help command. + + + + + If you add a recipe and the workspace layer does not exist, + the command creates the layer and populates it as follows: + + + + + + + + + README - Provides information on what is in workspace layer and how to + manage it. + + appends - A directory that contains *.bbappend files. + + conf - A configuration directory that contains the layer.conf file. + + recipes - A directory containing recipes. This directory contains a + folder for each directory added whose name matches that of the + added recipe. devtool places the recipe.bb file + within that sub-directory. + + + + + Running devtool add when the + workspace layer exists causes the tool to add the recipe + and append files into the existing workspace layer. + +
+ +
+ Creating the Workspace Layer + + + Use the create-workspace command to + create a new workspace layer in your + Build Directory. + When you create a new workspace layer, it is populated with the + README file and the + conf directory only. + + + + The following example creates a new workspace layer in your + current working and by default names the workspace layer + "workspace": + + $ devtool create-workspace + + + For complete syntax, use the + devtool create-workspace ‐‐help command. + + + + + You can create a workspace layer anywhere by supplying + a pathname with the command. + The following command creates a new workspace layer named + "new-workspace": + + $ devtool create-workspace /home/scottrif/new-workspace + + +
+ +
+ Modifying a Recipe + + + Use the modify command to configure + an external recipe in the workspace layer. + This command is very similar to the + add + command except that it does not physically create the + recipe in the workspace layer because the recipe already + exists in an external layer. + + + + The modify command provides options + that allow you to extract the source and provide a + development branch name where you can do your work. + You can use the following command to checkout the source + files: + + $ devtool modify -x recipe path-to-source + + Using the above command form, the default development branch + would be "devtool". + + + + If you want to name a development branch, use the + -b option with the + -x option: + + $ devtool modify -x -b branch recipe path-to-source + + + For complete syntax, use the + devtool modify ‐‐help command. + + +
+ +
+ Resetting a Recipe + + + Use the reset command to remove a + recipe and its configuration (e.g. the corresponding + .bbappend file) from the workspace layer. + Realize that this command deletes the recipe and and the + append file. + The command does not physically move them for you. + Consequently, you must be sure to physically relocate your + updated recipe and the append file outside of the workspace + layer before running the reset command. + + For complete syntax, use the + devtool reset ‐‐help command. + + +
+ +
+ Updating a Recipe + + + Use the update-recipe command to cause + devtool to update your recipe with patches + that reflect changes you make to the source files. + For example, if you know you are going to work on some + code, you could first use the + modify + command to extract the code and set up the workspace. + After which, you could modify, compile, and test the code + in its own layer to which it was extracted. + When you are satisfied with the results you can then + run the update-recipe to create the + patches and update the recipe in the external layer: + + $ devtool update-recipe recipe + + + For complete syntax, use the + devtool update-recipe ‐‐help command. + + +
+ +
+ Building Your Software + + + Use the build command to cause the + OpenEmbedded build system to build your software based + on the recipe file: + + $ devtool build recipe + + + For complete syntax, use the + devtool update-recipe ‐‐help command. + + Building your software using build is + identical to using BitBake to build the software. + +
+ +
+ Deploying Your Software on the Target Machine + + + Use the deploy-target command to deploy + the recipe's build output to the the live target machine: + + $ devtool deploy-target recipe target + + The target is the actual + target running on an SSH server (i.e. + user@hostname[:destdir]. + + For complete syntax, use the + devtool deploy-target ‐‐help command. + + +
+ +
+ Removing Your Software from the Target Machine + + + Use the undeploy-target command to remove + deployed build output from the target machine. + For undeploy-target command to work, + you must have previously used the + deploy-target + command. + + $ devtool undeploy-target recipe target + + The target is the actual + target running on an SSH server (i.e. + user@hostname. + + For complete syntax, use the + devtool undeploy-target ‐‐help command. + + +
- Using a Quilt Workflow + Using Quilt in Your Workflow Quilt @@ -1782,7 +2275,7 @@ The temporary source code used by the OpenEmbedded build system is kept in the Build Directory. See the - "Finding the Temporary Source Code" + "Finding Temporary Source Code" section to learn how to locate the directory that has the temporary source code for a particular package. Change Your Working Directory: @@ -1815,7 +2308,7 @@ $ bitbake -c compile -f name_of_package - The -f or --force + The -f or ‐‐force option forces the specified task to execute. If you find problems with your code, you can just keep editing and re-testing iteratively until things work as expected. @@ -1863,133 +2356,99 @@ PR value in the recipe since the resulting packages have changed. -
+ +
+ +
+ Finding Temporary Source Code -
- Using a Git Workflow - Git is an even more powerful tool that allows you to capture source code changes without having - a clean source tree. - This section outlines the typical workflow you can use to modify temporary source code, - test changes, and then preserve the changes in the form of a patch all using Git. - For general information on Git as it is used in the Yocto Project, see the - "Git" section. + You might + find it helpful during development to modify the temporary source code used by recipes + to build packages. + For example, suppose you are developing a patch and you need to experiment a bit + to figure out your solution. + After you have initially built the package, you can iteratively tweak the + source code, which is located in the + Build Directory, and then + you can force a re-compile and quickly test your altered code. + Once you settle on a solution, you can then preserve your changes in the form of + patches. + You can accomplish these steps all within either a + Quilt or + Git workflow. - - This workflow uses Git only for its ability to manage local changes to the source code - and produce patches independent of any version control system used with the Yocto Project. - + + During a build, the unpacked temporary source code used by recipes + to build packages is available in the Build Directory as + defined by the + S variable. + Below is the default value for the S variable as defined in the + meta/conf/bitbake.conf configuration file in the + Source Directory: + + S = "${WORKDIR}/${BP}" + + You should be aware that many recipes override the S variable. + For example, recipes that fetch their source from Git usually set + S to ${WORKDIR}/git. + + The + BP + represents the base recipe name, which consists of the name and version: + + BP = "${BPN}-${PV}" + + + - Follow these general steps: - - Find the Source Code: - The temporary source code used by the OpenEmbedded build system is kept in the - Build Directory. - See the - "Finding the Temporary Source Code" - section to learn how to locate the directory that has the temporary source code for a - particular package. - Change Your Working Directory: - You need to be in the directory that has the temporary source code. - That directory is defined by the - S - variable. - If needed, initialize a Git Repository: - If the recipe you are working with does not use a Git fetcher, - you need to set up a Git repository as follows: - - $ git init - $ git add * - $ git commit -m "initial revision" - - The above Git commands initialize a Git repository that is based on the - files in your current working directory, stage all the files, and commit - the files. - At this point, your Git repository is aware of all the source code files. - Any edits you now make to files can be committed later and will be tracked by - Git. - Edit the Files: - Make your changes to the temporary source code. - Test Your Changes: - Once you have modified the source code, the easiest way - to test your changes is by calling the - do_compile task as shown in the - following example: - - $ bitbake -c compile -f name_of_package - - The -f or --force - option forces the specified task to execute. - If you find problems with your code, you can just keep editing and - re-testing iteratively until things work as expected. - All the modifications you make to the temporary source code - disappear once you -c clean, -c cleansstate, - or -c cleanall with BitBake for the package. - Modifications will also disappear if you use the rm_work - feature as described in the - "Building an Image" - section of the Yocto Project Quick Start. - - See the List of Files You Changed: - Use the git status command to see what files you have actually edited. - The ability to have Git track the files you have changed is an advantage that this - workflow has over the Quilt workflow. - Here is the Git command to list your changed files: - - $ git status - - Stage the Modified Files: - Use the git add command to stage the changed files so they - can be committed as follows: - - $ git add file1.c file2.c file3.c - - Commit the Staged Files and View Your Changes: - Use the git commit command to commit the changes to the - local repository. - Once you have committed the files, you can use the git log - command to see your changes: - - $ git commit -m "commit-summary-message" - $ git log - - The name of the patch file created in the next step is based on your - commit-summary-message. - Generate the Patch: - Once the changes are committed, use the git format-patch - command to generate a patch file: - - $ git format-patch -1 - - Specifying "-1" causes Git to generate the - patch file for the most recent commit. - At this point, the patch file has all your edits made - to the file1.c, file2.c, and - file3.c files. - You can find the resulting patch file in the current directory and it - is named according to the git commit summary line. - The patch file ends with .patch. - Copy the Patch File: - For simplicity, copy the patch file into a directory named files, - which you can create in the same directory that holds the recipe - (.bb) file or the - append (.bbappend) file. - Placing the patch here guarantees that the OpenEmbedded build system will find - the patch. - Next, add the patch into the - SRC_URI - of the recipe. - Here is an example: - - SRC_URI += "file://0001-commit-summary-message.patch" - - Increment the Recipe Revision Number: - Finally, don't forget to 'bump' the - PR - value in the recipe since the resulting packages have changed. - + The path to the work directory for the recipe + (WORKDIR) + is defined as follows: + + ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} + + The actual directory depends on several things: + + TMPDIR: + The top-level build output directory + MULTIMACH_TARGET_SYS: + The target system identifier + PN: + The recipe name + EXTENDPE: + The epoch - (if + PE + is not specified, which is usually the case for most + recipes, then EXTENDPE is blank) + PV: + The recipe version + PR: + The recipe revision + + + + + As an example, assume a Source Directory top-level folder + name poky, a default Build Directory at + poky/build, and a + qemux86-poky-linux machine target + system. + Furthermore, suppose your recipe is named + foo_1.3.0.bb. + In this case, the work directory the build system uses to + build the package would be as follows: + + poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 + + + + + Now that you know where to locate the directory that has the temporary source code, + you can use a Quilt or Git workflow to make your edits, test the changes, + and preserve the changes in the form of patches.
@@ -2109,6 +2568,3 @@
- diff --git a/documentation/dev-manual/figures/build-workspace-directory.png b/documentation/dev-manual/figures/build-workspace-directory.png new file mode 100644 index 0000000000..f561f8fee6 Binary files /dev/null and b/documentation/dev-manual/figures/build-workspace-directory.png differ -- cgit v1.2.3-54-g00ecf