summaryrefslogtreecommitdiffstats
path: root/scripts/lib
diff options
context:
space:
mode:
authorTom Zanussi <tom.zanussi@linux.intel.com>2013-08-20 20:55:16 +0000
committerRichard Purdie <richard.purdie@linuxfoundation.org>2013-10-01 22:56:03 +0100
commit53a1d9a788fd9f970af980da2ab975cca60685c4 (patch)
treea7c0cd0c0483b6c8af7941a024173ce2b01239eb /scripts/lib
parente5d09762c26d31f7f26e921b743d39a50696cbb5 (diff)
downloadpoky-53a1d9a788fd9f970af980da2ab975cca60685c4.tar.gz
wic: Initial code for wic (OpenEmbedded Image Creator)
Initial implementation of the 'wic' command. The 'wic' command generates partitioned images from existing OpenEmbedded build artifacts. Image generation is driven by partitioning commands contained in an 'Openembedded kickstart' (.wks) file specified either directly on the command-line or as one of a selection of canned .wks files (see 'wic list images'). When applied to a given set of build artifacts, the result is an image or set of images that can be directly written onto media and used on a particular system. 'wic' is based loosely on the 'mic' (Meego Image Creator) framework, but heavily modified to make direct use of OpenEmbedded build artifacts instead of package installation and configuration, things already incorporated int the OE artifacts. The name 'wic' comes from 'oeic' with the 'oe' diphthong promoted to the letter 'w', because 'oeic' is impossible to remember or pronounce. This covers the mechanics of invoking and providing help for the command and sub-commands; it contains hooks for future commits to connect with the actual functionality, once implemented. Help is integrated into the 'wic' command - see that for details on usage. (From OE-Core rev: 95455ae4251e06d66e60945092b784d2d9ef165c) Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'scripts/lib')
-rw-r--r--scripts/lib/image/__init__.py22
-rw-r--r--scripts/lib/image/engine.py256
-rw-r--r--scripts/lib/image/help.py311
3 files changed, 589 insertions, 0 deletions
diff --git a/scripts/lib/image/__init__.py b/scripts/lib/image/__init__.py
new file mode 100644
index 0000000000..1ff814e761
--- /dev/null
+++ b/scripts/lib/image/__init__.py
@@ -0,0 +1,22 @@
1#
2# OpenEmbedded Image tools library
3#
4# Copyright (c) 2013, Intel Corporation.
5# All rights reserved.
6#
7# This program is free software; you can redistribute it and/or modify
8# it under the terms of the GNU General Public License version 2 as
9# published by the Free Software Foundation.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License along
17# with this program; if not, write to the Free Software Foundation, Inc.,
18# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
19#
20# AUTHORS
21# Tom Zanussi <tom.zanussi (at] linux.intel.com>
22#
diff --git a/scripts/lib/image/engine.py b/scripts/lib/image/engine.py
new file mode 100644
index 0000000000..a9b530cc04
--- /dev/null
+++ b/scripts/lib/image/engine.py
@@ -0,0 +1,256 @@
1# ex:ts=4:sw=4:sts=4:et
2# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
3#
4# Copyright (c) 2013, Intel Corporation.
5# All rights reserved.
6#
7# This program is free software; you can redistribute it and/or modify
8# it under the terms of the GNU General Public License version 2 as
9# published by the Free Software Foundation.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License along
17# with this program; if not, write to the Free Software Foundation, Inc.,
18# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
19#
20# DESCRIPTION
21
22# This module implements the image creation engine used by 'wic' to
23# create images. The engine parses through the OpenEmbedded kickstart
24# (wks) file specified and generates images that can then be directly
25# written onto media.
26#
27# AUTHORS
28# Tom Zanussi <tom.zanussi (at] linux.intel.com>
29#
30
31import os
32import sys
33from abc import ABCMeta, abstractmethod
34import shlex
35import json
36import subprocess
37import shutil
38
39import os, sys, errno
40
41
42def verify_build_env():
43 """
44 Verify that the build environment is sane.
45
46 Returns True if it is, false otherwise
47 """
48 try:
49 builddir = os.environ["BUILDDIR"]
50 except KeyError:
51 print "BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)"
52 sys.exit(1)
53
54 return True
55
56
57def get_line_val(line, key):
58 """
59 Extract the value from the VAR="val" string
60 """
61 if line.startswith(key + "="):
62 stripped_line = line.split('=')[1]
63 stripped_line = stripped_line.replace('\"', '')
64 return stripped_line
65 return None
66
67
68def find_artifacts(image_name):
69 """
70 Gather the build artifacts for the current image (the image_name
71 e.g. core-image-minimal) for the current MACHINE set in local.conf
72 """
73 bitbake_env_cmd = "bitbake -e %s" % image_name
74 rc, bitbake_env_lines = exec_cmd(bitbake_env_cmd)
75 if rc != 0:
76 print "Couldn't get '%s' output, exiting." % bitbake_env_cmd
77 sys.exit(1)
78
79 for line in bitbake_env_lines.split('\n'):
80 if (get_line_val(line, "IMAGE_ROOTFS")):
81 rootfs_dir = get_line_val(line, "IMAGE_ROOTFS")
82 continue
83 if (get_line_val(line, "STAGING_KERNEL_DIR")):
84 kernel_dir = get_line_val(line, "STAGING_KERNEL_DIR")
85 continue
86 if (get_line_val(line, "HDDDIR")):
87 hdddir = get_line_val(line, "HDDDIR")
88 continue
89 if (get_line_val(line, "STAGING_DATADIR")):
90 staging_data_dir = get_line_val(line, "STAGING_DATADIR")
91 continue
92 if (get_line_val(line, "STAGING_DIR_NATIVE")):
93 native_sysroot = get_line_val(line, "STAGING_DIR_NATIVE")
94 continue
95
96 return (rootfs_dir, kernel_dir, hdddir, staging_data_dir, native_sysroot)
97
98
99CANNED_IMAGE_DIR = "lib/image/canned-wks" # relative to scripts
100
101def find_canned_image(scripts_path, wks_file):
102 """
103 Find a .wks file with the given name in the canned files dir.
104
105 Return False if not found
106 """
107 canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
108
109 for root, dirs, files in os.walk(canned_wks_dir):
110 for file in files:
111 if file.endswith("~") or file.endswith("#"):
112 continue
113 if file.endswith(".wks") and wks_file + ".wks" == file:
114 fullpath = os.path.join(canned_wks_dir, file)
115 return fullpath
116 return None
117
118
119def list_canned_images(scripts_path):
120 """
121 List the .wks files in the canned image dir, minus the extension.
122 """
123 canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
124
125 for root, dirs, files in os.walk(canned_wks_dir):
126 for file in files:
127 if file.endswith("~") or file.endswith("#"):
128 continue
129 if file.endswith(".wks"):
130 fullpath = os.path.join(canned_wks_dir, file)
131 f = open(fullpath, "r")
132 lines = f.readlines()
133 for line in lines:
134 desc = ""
135 idx = line.find("short-description:")
136 if idx != -1:
137 desc = line[idx + len("short-description:"):].strip()
138 break
139 basename = os.path.splitext(file)[0]
140 print " %s\t\t%s" % (basename, desc)
141
142
143def list_canned_image_help(scripts_path, fullpath):
144 """
145 List the help and params in the specified canned image.
146 """
147 canned_wks_dir = os.path.join(scripts_path, CANNED_IMAGE_DIR)
148
149 f = open(fullpath, "r")
150 lines = f.readlines()
151 found = False
152 for line in lines:
153 if not found:
154 idx = line.find("long-description:")
155 if idx != -1:
156 print
157 print line[idx + len("long-description:"):].strip()
158 found = True
159 continue
160 if not line.strip():
161 break
162 idx = line.find("#")
163 if idx != -1:
164 print line[idx + len("#:"):].rstrip()
165 else:
166 break
167
168
169def wic_create(args, wks_file, rootfs_dir, bootimg_dir, kernel_dir,
170 native_sysroot, hdddir, staging_data_dir, scripts_path,
171 image_output_dir, properties_file, properties=None):
172 """
173 Create image
174
175 wks_file - user-defined OE kickstart file
176 rootfs_dir - absolute path to the build's /rootfs dir
177 bootimg_dir - absolute path to the build's boot artifacts directory
178 kernel_dir - absolute path to the build's kernel directory
179 native_sysroot - absolute path to the build's native sysroots dir
180 hdddir - absolute path to the build's HDDDIR dir
181 staging_data_dir - absolute path to the build's STAGING_DATA_DIR dir
182 scripts_path - absolute path to /scripts dir
183 image_output_dir - dirname to create for image
184 properties_file - use values from this file if nonempty i.e no prompting
185 properties - use values from this string if nonempty i.e no prompting
186
187 Normally, the values for the build artifacts values are determined
188 by 'wic -e' from the output of the 'bitbake -e' command given an
189 image name e.g. 'core-image-minimal' and a given machine set in
190 local.conf. If that's the case, the variables get the following
191 values from the output of 'bitbake -e':
192
193 rootfs_dir: IMAGE_ROOTFS
194 kernel_dir: STAGING_KERNEL_DIR
195 native_sysroot: STAGING_DIR_NATIVE
196 hdddir: HDDDIR
197 staging_data_dir: STAGING_DATA_DIR
198
199 In the above case, bootimg_dir remains unset and the image
200 creation code determines which of the passed-in directories to
201 use.
202
203 In the case where the values are passed in explicitly i.e 'wic -e'
204 is not used but rather the individual 'wic' options are used to
205 explicitly specify these values, hdddir and staging_data_dir will
206 be unset, but bootimg_dir must be explicit i.e. explicitly set to
207 either hdddir or staging_data_dir, depending on the image being
208 generated. The other values (rootfs_dir, kernel_dir, and
209 native_sysroot) correspond to the same values found above via
210 'bitbake -e').
211
212 """
213 try:
214 oe_builddir = os.environ["BUILDDIR"]
215 except KeyError:
216 print "BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)"
217 sys.exit(1)
218
219
220def wic_list(args, scripts_path, properties_file):
221 """
222 Print the complete list of properties defined by the image, or the
223 possible values for a particular image property.
224 """
225 if len(args) < 1:
226 return False
227
228 if len(args) == 1:
229 if args[0] == "images":
230 list_canned_images(scripts_path)
231 return True
232 elif args[0] == "properties":
233 return True
234 else:
235 return False
236
237 if len(args) == 2:
238 if args[0] == "properties":
239 wks_file = args[1]
240 print "print properties contained in wks file: %s" % wks_file
241 return True
242 elif args[0] == "property":
243 print "print property values for property: %s" % args[1]
244 return True
245 elif args[1] == "help":
246 wks_file = args[0]
247 fullpath = find_canned_image(scripts_path, wks_file)
248 if not fullpath:
249 print "No image named %s found, exiting. (Use 'wic list images' to list available images, or specify a fully-qualified OE kickstart (.wks) filename)\n" % wks_file
250 sys.exit(1)
251 list_canned_image_help(scripts_path, fullpath)
252 return True
253 else:
254 return False
255
256 return False
diff --git a/scripts/lib/image/help.py b/scripts/lib/image/help.py
new file mode 100644
index 0000000000..cb3112cf08
--- /dev/null
+++ b/scripts/lib/image/help.py
@@ -0,0 +1,311 @@
1# ex:ts=4:sw=4:sts=4:et
2# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
3#
4# Copyright (c) 2013, Intel Corporation.
5# All rights reserved.
6#
7# This program is free software; you can redistribute it and/or modify
8# it under the terms of the GNU General Public License version 2 as
9# published by the Free Software Foundation.
10#
11# This program is distributed in the hope that it will be useful,
12# but WITHOUT ANY WARRANTY; without even the implied warranty of
13# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14# GNU General Public License for more details.
15#
16# You should have received a copy of the GNU General Public License along
17# with this program; if not, write to the Free Software Foundation, Inc.,
18# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
19#
20# DESCRIPTION
21# This module implements some basic help invocation functions along
22# with the bulk of the help topic text for the OE Core Image Tools.
23#
24# AUTHORS
25# Tom Zanussi <tom.zanussi (at] linux.intel.com>
26#
27
28import subprocess
29import logging
30
31
32def subcommand_error(args):
33 logging.info("invalid subcommand %s" % args[0])
34
35
36def display_help(subcommand, subcommands):
37 """
38 Display help for subcommand.
39 """
40 if subcommand not in subcommands:
41 return False
42
43 help = subcommands.get(subcommand, subcommand_error)[2]
44 pager = subprocess.Popen('less', stdin=subprocess.PIPE)
45 pager.communicate(help)
46
47 return True
48
49
50def wic_help(args, usage_str, subcommands):
51 """
52 Subcommand help dispatcher.
53 """
54 if len(args) == 1 or not display_help(args[1], subcommands):
55 print(usage_str)
56
57
58def invoke_subcommand(args, parser, main_command_usage, subcommands):
59 """
60 Dispatch to subcommand handler borrowed from combo-layer.
61 Should use argparse, but has to work in 2.6.
62 """
63 if not args:
64 logging.error("No subcommand specified, exiting")
65 parser.print_help()
66 elif args[0] == "help":
67 wic_help(args, main_command_usage, subcommands)
68 elif args[0] not in subcommands:
69 logging.error("Unsupported subcommand %s, exiting\n" % (args[0]))
70 parser.print_help()
71 else:
72 usage = subcommands.get(args[0], subcommand_error)[1]
73 subcommands.get(args[0], subcommand_error)[0](args[1:], usage)
74
75
76##
77# wic help and usage strings
78##
79
80wic_usage = """
81
82 Create a customized OpenEmbedded image
83
84 usage: wic [--version] [--help] COMMAND [ARGS]
85
86 Current 'wic' commands are:
87 create Create a new OpenEmbedded image
88 list List available values for options and image properties
89
90 See 'wic help COMMAND' for more information on a specific command.
91"""
92
93wic_help_usage = """
94
95 usage: wic help <subcommand>
96
97 This command displays detailed help for the specified subcommand.
98"""
99
100wic_create_usage = """
101
102 Create a new OpenEmbedded image
103
104 usage: wic create <wks file or image name> [-o <DIRNAME> | --outdir <DIRNAME>]
105 [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
106 [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
107 [-k, --kernel-dir] [-n, --native-sysroot] [-s, --skip-build-check]
108
109 This command creates an OpenEmbedded image based on the 'OE kickstart
110 commands' found in the <wks file>.
111
112 The -o option can be used to place the image in a directory with a
113 different name and location.
114
115 See 'wic help create' for more detailed instructions.
116"""
117
118wic_create_help = """
119
120NAME
121 wic create - Create a new OpenEmbedded image
122
123SYNOPSIS
124 wic create <wks file or image name> [-o <DIRNAME> | --outdir <DIRNAME>]
125 [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]
126 [-e | --image-name] [-r, --rootfs-dir] [-b, --bootimg-dir]
127 [-k, --kernel-dir] [-n, --native-sysroot] [-s, --skip-build-check]
128
129DESCRIPTION
130 This command creates an OpenEmbedded image based on the 'OE
131 kickstart commands' found in the <wks file>.
132
133 In order to do this, wic needs to know the locations of the
134 various build artifacts required to build the image.
135
136 Users can explicitly specify the build artifact locations using
137 the -r, -b, -k, and -n options. See below for details on where
138 the corresponding artifacts are typically found in a normal
139 OpenEmbedded build.
140
141 Alternatively, users can use the -e option to have 'mic' determine
142 those locations for a given image. If the -e option is used, the
143 user needs to have set the appropriate MACHINE variable in
144 local.conf, and have sourced the build environment.
145
146 The -e option is used to specify the name of the image to use the
147 artifacts from e.g. core-image-sato.
148
149 The -r option is used to specify the path to the /rootfs dir to
150 use as the .wks rootfs source.
151
152 The -b option is used to specify the path to the dir containing
153 the boot artifacts (e.g. /EFI or /syslinux dirs) to use as the
154 .wks bootimg source.
155
156 The -k option is used to specify the path to the dir containing
157 the kernel to use in the .wks bootimg.
158
159 The -n option is used to specify the path to the native sysroot
160 containing the tools to use to build the image.
161
162 The -s option is used to skip the build check. The build check is
163 a simple sanity check used to determine whether the user has
164 sourced the build environment so that the -e option can operate
165 correctly. If the user has specified the build artifact locations
166 explicitly, 'wic' assumes the user knows what he or she is doing
167 and skips the build check.
168
169 When 'wic -e' is used, the locations for the build artifacts
170 values are determined by 'wic -e' from the output of the 'bitbake
171 -e' command given an image name e.g. 'core-image-minimal' and a
172 given machine set in local.conf. In that case, the image is
173 created as if the following 'bitbake -e' variables were used:
174
175 -r: IMAGE_ROOTFS
176 -k: STAGING_KERNEL_DIR
177 -n: STAGING_DIR_NATIVE
178 -b: HDDDIR and STAGING_DATA_DIR (handlers decide which to use)
179
180 If 'wic -e' is not used, the user needs to select the appropriate
181 value for -b (as well as -r, -k, and -n).
182
183 The -o option can be used to place the image in a directory with a
184 different name and location.
185
186 As an alternative to the wks file, the image-specific properties
187 that define the values that will be used to generate a particular
188 image can be specified on the command-line using the -i option and
189 supplying a JSON object consisting of the set of name:value pairs
190 needed by image creation.
191
192 The set of properties available for a given image type can be
193 listed using the 'wic list' command.
194"""
195
196wic_list_usage = """
197
198 List available OpenEmbedded image properties and values
199
200 usage: wic list images
201 wic list <image> help
202 wic list properties
203 wic list properties <wks file>
204 wic list property <property>
205 [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
206
207 This command enumerates the set of available canned images as well as
208 help for those images. It also can be used to enumerate the complete
209 set of possible values for a specified option or property needed by
210 the image creation process.
211
212 The first form enumerates all the available 'canned' images.
213
214 The second form lists the detailed help information for a specific
215 'canned' image.
216
217 The third form enumerates all the possible values that exist and can
218 be specified in an OE kickstart (wks) file.
219
220 The fourth form enumerates all the possible options that exist for
221 the set of properties specified in a given OE kickstart (ks) file.
222
223 The final form enumerates all the possible values that exist and can
224 be specified for any given OE kickstart (wks) property.
225
226 See 'wic help list' for more details.
227"""
228
229wic_list_help = """
230
231NAME
232 wic list - List available OpenEmbedded image properties and values
233
234SYNOPSIS
235 wic list images
236 wic list <image> help
237 wic list properties
238 wic list properties <wks file>
239 wic list property <property>
240 [-o <JSON PROPERTY FILE> | --outfile <JSON PROPERTY_FILE>]
241
242DESCRIPTION
243 This command enumerates the complete set of possible values for a
244 specified option or property needed by the image creation process.
245
246 This command enumerates the set of available canned images as well
247 as help for those images. It also can be used to enumerate the
248 complete set of possible values for a specified option or property
249 needed by the image creation process.
250
251 The first form enumerates all the available 'canned' images.
252 These are actually just the set of .wks files that have been moved
253 into the /scripts/lib/image/canned-wks directory).
254
255 The second form lists the detailed help information for a specific
256 'canned' image.
257
258 The third form enumerates all the possible values that exist and
259 can be specified in a OE kickstart (wks) file. The output of this
260 can be used by the third form to print the description and
261 possible values of a specific property.
262
263 The fourth form enumerates all the possible options that exist for
264 the set of properties specified in a given OE kickstart (wks)
265 file. If the -o option is specified, the list of properties, in
266 addition to being displayed, will be written to the specified file
267 as a JSON object. In this case, the object will consist of the
268 set of name:value pairs corresponding to the (possibly nested)
269 dictionary of properties defined by the input statements used by
270 the image. Some example output for the 'list <wks file>' command:
271
272 $ wic list test.ks
273 "part" : {
274 "mountpoint" : "/"
275 "fstype" : "ext3"
276 }
277 "part" : {
278 "mountpoint" : "/home"
279 "fstype" : "ext3"
280 "offset" : "10000"
281 }
282 "bootloader" : {
283 "type" : "efi"
284 }
285 .
286 .
287 .
288
289 Each entry in the output consists of the name of the input element
290 e.g. "part", followed by the properties defined for that
291 element enclosed in braces. This information should provide
292 sufficient information to create a complete user interface with.
293
294 The final form enumerates all the possible values that exist and
295 can be specified for any given OE kickstart (wks) property. If
296 the -o option is specified, the list of values for the given
297 property, in addition to being displayed, will be written to the
298 specified file as a JSON object. In this case, the object will
299 consist of the set of name:value pairs corresponding to the array
300 of property values associated with the property.
301
302 $ wic list property part
303 ["mountpoint", "where the partition should be mounted"]
304 ["fstype", "filesytem type of the partition"]
305 ["ext3"]
306 ["ext4"]
307 ["btrfs"]
308 ["swap"]
309 ["offset", "offset of the partition within the image"]
310
311"""