summaryrefslogtreecommitdiffstats
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* oe-selftest: fix behaviour if oe-selftest.log is a dangling symlinkPaul Eggleton2017-01-091-1/+2
| | | | | | | | | | | | | If you delete the log file that the oe-selftest.log symlink points to but not the symlink itself, because we were using os.path.exists() here the code assumed that the symlink didn't exist when in fact it still did. Use os.path.lexists() instead. (From OE-Core rev: 263af91a0efd21e041ecdb0c40f9b2d4e735f67d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* gummiboot: Remove/change gummiboot references with systemd-bootAlejandro Hernandez2017-01-092-25/+23
| | | | | | | | | | | | | | | After systemd-boot was introduced, its been tested for a while with no major issues being found until now, this patch completely replaces all gummiboot instances with systemd-boot ones, taking the next step into cleaning up systemd-boot/gummiboot. [YOCTO #10332] (From OE-Core rev: f9a61d3400ad9068a6d83b8eb6aefe3098c58e68) Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie2017-01-092-2/+2
| | | | | | | | | | | | | | | | | There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). (From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: Improve BUILDDIR environment handlingRichard Purdie2017-01-051-6/+13
| | | | | | | | | | | | | | | | Its possible something (like bitbake/tinfoil2) may mess around with the environment and using the enviroment as a global variable store isn't particularly nice anyway. This patch changes the BUILDDIR usages so that the environment isn't used as a global store and a global variable is used instead. Whilst that is still not perfect, it does avoid the current double and triple backtraces we're seeing where tinfoil2/bitbake has trampled the enviroment leading to failures of failures making debugging even harder. (From OE-Core rev: 689b676bbf2f1a5fadb04aeb41d5e68e35356545) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: let command line parameters override defaultsPatrick Ohly2017-01-051-1/+1
| | | | | | | | | | | | | | | | It may be necessary to override the parameters gathered for the qemu invocation. For example, the qemux86 machine configuration sets "-vga vmware", but when using OVMF as BIOS, only "-vga std" is supported. By putting the parameters derived from custom runqemu parameters like "qemuparams" after the parameters derived from the machine configuration the user gets the possibility to override those. (From OE-Core rev: b6feb7578d60289c8b6e376cfaac8a3ee45e72f9) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* example-recipe: Fix LDFLAGS compilation issue on newly created recipesAlejandro Hernandez2016-12-221-1/+1
| | | | | | | | (From meta-yocto rev: aa3e99e24c3234b6f3f19169939d4cd498d0ce36) Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add kickstart file for MPC8315Ed Bartosh2016-12-221-0/+6
| | | | | | | | | | | | | Added kickstart file to produce partitioned image for MPC8315 reference hardware. [YOCTO #8719] (From OE-Core rev: 8c873b4fd92b72adfc49d20bbfab4779857fb6c9) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: add --fixed-size wks optionMaciej Borzecki2016-12-225-42/+105
| | | | | | | | | | | | | | | | | Added new option --fixed-size to wks. The option can be used to indicate the exact size of a partition. The option cannot be added together with --size, in which case an error will be raised. Other options that influence automatic partition size (--extra-space, --overhead-factor), if specifiec along with --fixed-size, will raise an error. If it partition data is larger than the amount of space specified with --fixed-size option wic will raise an error. (From OE-Core rev: fdd217ba874bd480e0180830fe2e6bd54dde19d9) Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: modify: fix usage on the kernelPaul Eggleton2016-12-221-0/+2
| | | | | | | | | | | | | | | | | | | When using devtool modify on the kernel, we have to do a bit of a dance with tinfoil instances because we only find out that we're working on a kernel recipe after tinfoil is initialised, but then we need to build kern-tools-native which we're doing just by running bitbake directly. With the tinfoil2 changes, a datastore for the recipe that we were keeping around across the opening and closing of tinfoil is no longer able to be used. Re-parse the recipe to avoid this problem. (In future this whole thing will be able to be done in the same tinfoil instance thanks to tinfoil2, but that refactoring is yet to be done.) (From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: look for wks files in <layer>/wicEd Bartosh2016-12-221-2/+5
| | | | | | | | | | | | | | | | Currently wic looks for wks files in <layer dir>/scripts/lib/wic/canned-wks/ directories. This path is too nested and doesn't look consistent with the naming scheme of layer directories. Added <layer>/wic directory to the list of paths to look for wks files. (From OE-Core rev: 803b5fa798cf7580c7c10401eb04a1cccf51b8ab) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: import git module only when neededEd Bartosh2016-12-201-1/+1
| | | | | | | | | | | | | | | | | git module is not included into standard Python library and therefore causes import errors on the systems where PythonGit is not installed. As git module only used in the code implementing --repository functionality it's better to import git only in the scope that requires it. [YOCTO #10821] (From OE-Core rev: 66be32c1a075201d6ee0e9b9e10b84e6a2ace745) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: obey the rootfs size from the metadataChristopher Larson2016-12-171-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When no --size is specified for the rootfs in the .wks, we want to obey the rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result in image construction failure, if the size determined by du was insufficient to hold the files without the aforementioned extra space. This fallback from --size to ROOTFS_SIZE was already implemented when --rootfs-dir is specified in the .wks, but it did not occur otherwise, neither when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used. This made a certain amount of sense, as this fallback logic happened at such a level that it wasn't able to identify which partitions were rootfs partitions otherwise. Rather than doing it at that level, we can do it in prepare_rootfs(), which is run by the rootfs source plugins. Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now both be applied when --size isn't specified in the .wks. A warning is added about this, though a user won't see it unless wic fails or they examine the do_image_wic log. Fixes [YOCTO #10815] (From OE-Core rev: 1d50e11286722c4114c1ae0bc285f846cd85fc4c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: remove True option to getVarFlag callsJoshua Lock2016-12-162-4/+4
| | | | | | | | | | | | | | | getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) (From OE-Core rev: 3e4806063fe11092b2307f113a6c0b0f04104091) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: remove True option to getVar callsJoshua Lock2016-12-1622-106/+106
| | | | | | | | | | | | | getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) (From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/runqemu: Allow to use qemu from host.Mariano Lopez2016-12-161-0/+15
| | | | | | | | | | | | | | | This will add support to use qemu from the running host, with this is possible to put qemu-native in ASSUME_PROVIDED variable. By default it will try to get qemu from the build sysroot, and only if it fails will try to use the host's qemu. (From OE-Core rev: fe7fd2cd3a9c4fb5b31bd3cab81c96a3b81cb540) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: prevent BBHandledException from showing tracebackPaul Eggleton2016-12-141-4/+7
| | | | | | | | | | | | If we don't catch this then attempting to run devtool in non-memres mode when bitbake is already running will produce a traceback instead of just an error message. (From OE-Core rev: e01b75dff599ffa2b66e6608b28bbb3564365eee) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: extract: disable basehash mismatch errorsPaul Eggleton2016-12-141-0/+1
| | | | | | | | | | | | | Using the setVariable commands here followed by buildFile will result in "basehash mismatch" errors, and that's expected since we are deviating *at runtime* from what was previously seen by changing these variable values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors. (From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix extraction of source to work in memres modePaul Eggleton2016-12-142-69/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | Extracting the source for a recipe (as used by devtool's extract, modify and upgrade subcommands) requires us to run do_fetch, do_unpack, do_patch and any tasks that the recipe has inserted inbetween, and do so with a modified datastore primarily so that we can redirect WORKDIR and STAMPS_DIR in order to have the files written out to a place of our choosing and avoid stamping the tasks as having executed in a real build context respectively. However, this all gets much more difficult when in memres mode since we can't call internal functions such as bb.build.exec_func() directly - instead we need to execute the tasks on the server. To do this we use the buildFile command which already exists for the purpose of supporting bitbake -b, and setVariable commands to set up the appropriate datastore. (I did look at passing the modified datastore to the buildFile command instead of using setVar() on the main datastore, however its use of databuilder makes that very difficult, and we'd also need a different method of getting the changes in the datastore over to the worker as well.) (From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: add OE lib pathPaul Eggleton2016-12-141-0/+1
| | | | | | | | | | | The autotools code imports oe.package; we weren't experiencing a problem with this probably due to OE itself adding that path previously. (From OE-Core rev: a61d7bf8447b2d2c65eb34315c86086ff35c8bc9) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/patch: move in logic to commit for additional tasksPaul Eggleton2016-12-141-36/+1
| | | | | | | | | | | | | | | If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for additional tasks between do_unpack and do_patch, make a git commit. This logic was previously implemented in devtool itself, but it makes more sense for it to be implemented in the patch class since that's where the rest of the logic is for this (or in lib/oe/patch.py). It also makes it possible for this to work with tinfoil2. (From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool / recipetool: use tinfoil parsing APIPaul Eggleton2016-12-147-53/+16
| | | | | | | | | | | | | Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of the recipeutils equivalents, and replace any local duplicate implementations. This not only tidies up the code but also allows these calls to work in memres mode. (From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: package: don't try to initialise tinfoil twicePaul Eggleton2016-12-141-3/+1
| | | | | | | | | | | | | | | setup_tinfoil() already calls prepare(), we don't need to call it again ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of bitbake should be run against a build directory". Calling prepare() twice should probably still be allowed, so that ought to be fixed separately, but in the mean time this code is still wrong so fix it here. (From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-pkgdata-util: Make read-value handle override variablesOla x Nilsson2016-12-141-8/+8
| | | | | | | | | | | | | | | Some variables in pkgdata files have a package-name override. When the bare variable can not be found, try with the override-variant. PKGSIZE is one such variable, and already had special code to handle this. Test included. (From OE-Core rev: 6df99cda894033cba68bc6ab91e47f67e0d788a5) Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-buildenv-internal: show usage outputEd Bartosh2016-12-131-0/+12
| | | | | | | | | | | | | Show usage text if script is not sourced. Tested in bash, zsh and dash. [YOCTO #10751] (From OE-Core rev: ac7a905b18acb8bd9b2412b6682afbe1d7e18d7b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-find-native-sysroot: create usage outputEd Bartosh2016-12-131-0/+14
| | | | | | | | | | | | Created usage output for oe-find-native-sysroot script. [YOCTO #10751] (From OE-Core rev: 8ddfc48c7f3e2ca45c035cec492fdc31c6ad484f) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-git-proxy: create usage outputEd Bartosh2016-12-131-0/+21
| | | | | | | | | | | | Created usage output for oe-git-proxy script. [YOCTO #10751] (From OE-Core rev: ec0fdb5e896fc20dbafcc8ae507b17c011dc56fd) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oepydevshell-internal.py: standardize usage outputEd Bartosh2016-12-131-4/+11
| | | | | | | | | | | | | Made usage output of oepydevshell-internal.py to look similar to the output of other oe scripts. [YOCTO #10751] (From OE-Core rev: e6480af22a7a12c655efed14f8f1aea658f26b1c) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-setup-builddir: create usage outputEd Bartosh2016-12-131-0/+8
| | | | | | | | | | | | Created usage output for oe-setup-builddir script. [YOCTO #10751] (From OE-Core rev: 77606455df7d45fd014c3603e1cf1b24efd37695) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-setup-rpmrepo: standardize usage outputEd Bartosh2016-12-131-11/+17
| | | | | | | | | | | | | Made usage output of oe-setup-rpmrepo to look similar to the output of other oe scripts. [YOCTO #10751] (From OE-Core rev: 5423c9a412c680b781417a64b412838845b5d075) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-trim-schemas: create usage outputEd Bartosh2016-12-131-0/+9
| | | | | | | | | | | | Created usage output for oe-trim-schemas script. [YOCTO #10751] (From OE-Core rev: a9fac12157ed434a6711de1af77eda1f4a8e9e8a) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-run-native: standardize usage outputEd Bartosh2016-12-131-2/+9
| | | | | | | | | | | | | Made usage output of oe-run-native to look similar to the output of other oe scripts. [YOCTO #10751] (From OE-Core rev: e1c96125ea674509fbc9b36dc671b7a53bd848ac) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: Load plugins in a well defined orderOla x Nilsson2016-12-131-2/+2
| | | | | | | | | | | | | | | To allow recipetool plugins in one layer to shadow another in a well defined way, first search BBPATH/lib/recipetool directories and then scripts/lib/recipetool and load only the first found. The previous search and load loop would load all found plugins with the ones found later replacing any found before. (From OE-Core rev: 7a8726a3662a3909dc6ef6e8d1029d0b1aa938c3) Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Load plugins in a well defined orderOla x Nilsson2016-12-132-3/+8
| | | | | | | | | | | | | | | To allow devtool plugins in one layer to shadow another in a well defined way, first search BBPATH/lib/devtool directories and then scripts/lib/devool and load only the first found. The previous search and load loop would load all found plugins with the ones found later replacing any found before. (From OE-Core rev: 1b2b8a0a80de17ea053002fdd124055d2798029a) Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: rename command line option -p -> -sEd Bartosh2016-12-131-1/+1
| | | | | | | | | | | | Short variant of wic command line option --skip-build-check is incorretly named -p. It's named -s in wic help and Yocto documentation. (From OE-Core rev: 8d4121459334349f10bfb0f4cda00a8461aab245) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Create a logical partition only when it is really mandatoryAlessio Igor Bogani2016-12-131-3/+4
| | | | | | | | | | | Don't worth bother with logical partition on MBR partition type (aka msdos) if disk image generated by wic should have 4 partitions. (From OE-Core rev: 36a558fbdc96094626e7de1a3510691e30885368) Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-selftest: Add option to submit test result to a git repository.Mariano Lopez2016-12-131-0/+102
| | | | | | | | | | | | | | | | | | | | | | This new option allows to commit the result to a git repository, along with the results it will add a metadata file for information of the current selftest run, such as: hostname, machine, distro, distro version, host version, and layers. This implementation will have a branch per different hostname, testing branch, and machine. To use this feature use: oe-selftest <options> --repository <repository_link> [YOCTO #9954] (From OE-Core rev: 758e4b7e619f9aaa79e3910b5539ff8fb2d1064a) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: fix encoding-related errors creating python recipesPaul Eggleton2016-12-081-2/+2
| | | | | | | | | | | | Yet another instance of us expecting a string back from subprocess when in Python 3 what you get back is bytes. Just decode the output within run_command() so we avoid this everywhere. (From OE-Core rev: 103faae78cdff5280c7b7cdb7ca01e0868d02ec9) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui: support reading reduced /proc logsPatrick Ohly2016-12-071-0/+31
| | | | | | | | | | | | | | | Pre-processing /proc data during the build considerably reduces the amount of data written to disk: 176KB instead of 4.7MB for a 20 minuted build. Parsing also becomes faster. buildstats.bbclass only writes the reduced logs now, but support for the full /proc files is kept around as reference. (From OE-Core rev: b5e47df9af1ebbb477074587fdeae17eb2f55582) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui: simplify drawing of memory usagePatrick Ohly2016-12-073-7/+23
| | | | | | | | | | | | | The internal representation after parsing now matches exactly what the drawing code needs, thus speeding up drawing a bit. However, the main motivation is to store exactly that required information in a more compact file. (From OE-Core rev: ca06e67a0bb5820b38fda4c8dfee20764c1e59ae) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui: render disk space usagePatrick Ohly2016-12-073-0/+99
| | | | | | | | | | | | | | | | | | | | | | | This adds a new, separate chart showing the amount of disk space used over time for each volume monitored during the build. The hight of the graph entries represents the delta between current usage and minimal usage during the build. That's more useful than showing just the current usage, because then a graph showing changes in the order of MBs in a volume that is several GB large would be just flat. The legend shows the maximum of those deltas, i.e. maximum amount of space needed for the build. Minor caveat: sampling of disk space usage starts a bit later than the initial task, so the displayed value may be slightly lower than the actual amount of space needed because sampling does not record the actual initial state. (From OE-Core rev: 263d189d066b578debf08b2bd07494a69b70f70d) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui/parsing.py: fix error handling in meminfo parserPatrick Ohly2016-12-071-1/+1
| | | | | | | | | | | When matching fails, m.group(0) is invalid and can't be used in the error message. (From OE-Core rev: ddfea21e06a2e6b1a1e766969f8c134a6de7388a) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui/draw.py: skip empty CPU and disk usage chartsPatrick Ohly2016-12-071-48/+50
| | | | | | | | | | | The only real change is the addition of two if checks that skips the corresponding drawing code when there is no data. (From OE-Core rev: 1658fd5e9ca1ba793cae604c2a395d54e3ec9056) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui: show system utilizationPatrick Ohly2016-12-072-20/+57
| | | | | | | | | | | | | | | This enables rendering of the original bootchart charts for CPU, disk and memory usage. It depends on the /proc samples recorded by the updated buildstats.bbclass. Currently, empty charts CPU and disk usage charts are drawn if that data is not present; the memory chart already gets skipped when there's no data, which will also have to be added for the other two. (From OE-Core rev: 233d3e50b361feea07803a9c0f2a691e687c6cd5) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui/draw.py: fix drawing of samples not starting at zeroPatrick Ohly2016-12-071-1/+1
| | | | | | | | | | | | The code did not handle x scaling correctly when drawing starts at some time larger than zero, i.e. it worked for normal bootchart data, but not for the system statistics recorded by buildstats.bbclass. (From OE-Core rev: 166f8f9aaa1f01fc6d6a5451f8f06b815c51ffae) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* pybootchartgui/draw.py: allow moving process chart up and downPatrick Ohly2016-12-071-1/+1
| | | | | | | | | | | | | | | | | | | | Substracting curr_y when determining the hight of the process chart is wrong because the height is independent of the position where the chart is about to be drawn. It happens to work at the moment because curr_y is always 10 when render_processes_chart() gets called. But it leads to a negative height when other charts are drawn above it, and then the grid gets drawn on top of those other charts. Substracting some constant is relevant because otherwise the box is slightly larger than the process bars. Not sure exactly where that comes from (text height?); leg_s seems a suitable constant and happens to be 10, so everything still gets rendered exactly as before. (From OE-Core rev: b6bb690728c329ae448f89a1b68298c6dd8a573a) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/send-pull-request: Avoid multiple chain headersPatrick Ohly2016-12-071-2/+7
| | | | | | | | | | | | | | | | | | | | | | | | When creating a patch set with cover letter using the send-pull-request script, both the "In-Reply-To" and "References" headers are appended twice in patch 2 and subsequent. That's because git-format-patch already inserted them and then git-send-email repeats that. Suppressing mail threading in git-send-email with --no-thread avoids the problem and is the right solution because it works regardless whether git-send-email is called once or twicee. Repeating these headers is a violation of RFC 2822 and can confuse mail programs. For example, Patchwork does not detect a patch series problem when there are these extra headers. [YOCTO #10718] (From OE-Core rev: 303a1aa3df43eb0b693d8602062fa33c4a08fdd6) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb-perf: plot histograms base on buildstats dataLeonardo Sandoval2016-11-301-0/+157
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Scripts that produces script data to be consumed by gnuplot. There are two possible plots depending if either the -S parameter is present or not: * without -S: Produces a histogram listing top N recipes/tasks versus stats. The first stat defined in the -s parameter is the one taken into account for ranking * -S: Produces a histogram listing tasks versus stats. In this case, the value of each stat is the sum for that particular stat in all recipes found. Stats values are in descending order defined by the first stat defined on -s EXAMPLES 1. Top recipes' tasks taking into account utime $ buildstats-plot.sh -s utime | gnuplot -p 2. Tasks versus utime:stime $ buildstats-plot.sh -s utime:stime -S | gnuplot -p 3. Tasks versus IO write_bytes:IO read_bytes $ buildstats-plot.sh -s 'IO write_bytes:IO read_bytes' -S | gnuplot -p (From OE-Core rev: 74408c19fba89de54c093fccf65b3a072d6a197b) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* python3: Build and package precompiled modulesDominic Sacré2016-11-301-0/+10
| | | | | | | | | | | | | | | | | | | | | Remove the patch that was applied in the python3 and python3-native recipes to skip compilation of python modules. Modify generate-manifest-3.5.py to match '__pycache__' directories in FILES_*. This is necessary because Python3 puts .pyc files in '__pycache__' subdirectories one level below the corresponding .py files, whereas in Python2 they used to be right next to the sources. This change significantly reduces the startup overhead of Python3 scripts. For example, on a Cortex-A9, "python3 -c pass" took 0.40s before, and 0.19s after. (From OE-Core rev: bb4d689769703177dbb0df0935e15016b879f42b) Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix handling of unicode characters from subprocess stdoutJiajie Hu2016-11-231-2/+3
| | | | | | | | | | | | | | | | | | In previous implementation, a UnicodeDecodeError exception will be raised if multi-byte encoded characters are printed by the subprocess. As an example, the following command will fail in an en_US.UTF-8 environment because wget quotes its saving destination with '‘'(0xE2 0x80 0x98), while just the first byte is provided for decoding: devtool add recipe http://example.com/source.tar.xz The patch fixes the issue by avoiding such kind of incomplete decoding. (From OE-Core rev: 1875ea92546d23abcab1b40b562477a0016f712d) Signed-off-by: Jiajie Hu <jiajie.hu@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Specify the stats to take into accountLeonardo Sandoval2016-11-231-17/+82
| | | | | | | | | | | | | | | | | | | | | | | | | There are many more stats on buildstats that 'Elapsed time', so make the script more flexible to support all stats. Some cmd line examples: $ buildstats.sh -s 'utime' Buildstats' data covers proc's stats in different areas, including CPU times, IO, program system resources and child program system resources. In order to print values on each of these sets from command line, one can use the following: $ buildstats.sh -H -s 'TIME' | less $ buildstats.sh -H -s 'IO' | less and 'RUSAGE' and 'CHILD_RUSAGE' for program and program's child system resources. (From OE-Core rev: 81479b191287ccbf4cf94fa2d0ad46813091bca1) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>