summaryrefslogtreecommitdiffstats
path: root/scripts/lib/devtool
Commit message (Collapse)AuthorAgeFilesLines
* devtool: do not leave behind source trees in workspace/sourcesAlexander Kanavin2022-08-211-3/+13
| | | | | | | | | | | | | | | | | | | | These are typically auto-extracted with modify/upgrade from recipes and can be easily recreated. On the rare occasions where they need to be reused, they are still available under workspace/attic (which is already used for old recipes and appends), so nothing gets lost. This avoids the annoyance of devtool refusing to proceed because there is a previous source tree in workspace/sources. For independent source trees behave as before: do nothing. Adjust the test that previously deleted those trees by hand. (From OE-Core rev: 9bfb95d070d68d5ab5adfe0ea096f5fbf9cad8b0) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts: Add copyright statements to files without oneRichard Purdie2022-08-121-0/+2
| | | | | | | | | | Where there isn't a copyright statement, add one to make it explicit. Also drop editor config lines where they were present and add license identifiers as MIT if there isn't one. (From OE-Core rev: deb3ccec53e0bd63bc4235cf2b0d3fc781687361) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/upgrade: catch bb.fetch2.decodeurl errorsAlexander Kanavin2022-08-041-1/+4
| | | | | | | | | | | Otherwise, workspace cleanup (removing bogus recipe and source tree) will not happen, leaving breakage behind. (From OE-Core rev: 74774f9b67580a8c56f605dfd4cc7b856bbeeae8) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/upgrade: correctly clean up when recipe filename isn't yet knownAlexander Kanavin2022-08-041-15/+13
| | | | | | | | | | | | | | | There is a coding error in the second invocation of _upgrade_error: rf is passed into it before it is initialized in the try: block. And so bogus recipes are left behind in the workspace, causing breakage. Instead, rewrite the functions to take the recipe directory name in the workspace layer, which can be calculated in advance. (From OE-Core rev: e653996369c1d2b5ac8367ad85f4816d679b6c98) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish: handle patching when S points to subdir of a git repoPaul Eggleton2022-07-141-6/+19
| | | | | | | | | | | | If devtool finish needs to create a patch and have it applied to the sources for a recipe where S points to a subdirectory of the sources, then the patch needs to be applied at the root of the repo i.e. we need to add a patchdir= parameter to the SRC_URI entry. (From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f) Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: ignore pn- overrides when determining SRC_URI overridesPaul Eggleton2022-07-121-1/+3
| | | | | | | | | | | | | | | | | | | | If (perhaps foolishly) at your configuration level you have e.g. SRC_URI_append_pn-recipename = " file://patchname.patch" and then run devtool modify on a different recipe, an error occurs: INFO: SRC_URI contains some conditional appends/prepends - will create branches to represent these ... ERROR: [Errno 2] No such file or directory: '/path/to/downloads/patchname.patch' pn- overrides would not constitute an alternative configuration that we should handle in this context, so just ignore them to avoid the issue. (From OE-Core rev: 3f2a812ade42ece0bb59b2d303125a91b29936dd) Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix _copy_file() TypeErrorXiaobing Luo2022-06-111-1/+1
| | | | | | | | | | | | | | | | when devtool finish, the _copy_file() failed. -------------------------------------------- TypeError: _copy_file() got an unexpected keyword argument 'base_outdir' -------------------------------------------- Fixes: 05f2d5d2ce00 ("devtool: finish: add dry-run option") (From OE-Core rev: a45d9dc089fb2719ca69b92870917f8c0925f632) Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: explicitly set main or master branches in upgrades when availableAlexander Kanavin2022-02-211-7/+8
| | | | | | | | | | | | | In particular this resolves devtool's inability to pick a branch when the same tagged commit is avaiable in main and in a release maintenance branch. Thanks to Peter Kjellerstedt for the suggestion. (From OE-Core rev: bcb21ee2760a2c76039412a56c6cda43fbf96fd0) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Remove stripped binaries in pseudo contextFlorian Amstutz2022-02-071-1/+1
| | | | | | | | | | | deploy-target may fail the second time with "pseudo abort" because devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot context. (From OE-Core rev: 2338a33b690b0bbe279cde3f73764911b239cb50) Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: sdk-update: Remove an unnecessary \n from SSTATE_MIRRORSPeter Kjellerstedt2022-02-031-1/+1
| | | | | | | | | | | Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror entries) there is no need to separate the entries in SSTATE_MIRRORS with "\n". (From OE-Core rev: 6e66ec3cd201d32d3d97df2222f41bb8c475fb69) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Drop useless classRichard Purdie2021-09-231-4/+0
| | | | | | | | | | | | | | | This class was added by me back in 2007 and has changed one for a whitespace issue since. It only has two lines and neither are particularly useful, one was replaced by the nopackages class, the other adding recursive dependencies also is now mainly problematic adding tons of unneeded dependencies. The name is hard to understand and the class doesn't have a clear purpose. Drop it. Remove the references in devtool (which may be the one reason to keep it around but in my view still not worth it). (From OE-Core rev: 6e2cbfc561dac89bf9183d24d90e52f7d9117826) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* package/scripts: Fix FILES_INFO handlingRichard Purdie2021-08-171-2/+3
| | | | | | | | | | | | | | | There is a long standing bug where FILES_INFO isn't written into pkgdata with a package suffix. This means if the files are read into the datastore as intended, the last one "wins". Fix this to work as intended. Most of the call sites using the data need to be updated to handle this and the overrides change correctly. Also fix some other problematic references noticed along the way. (From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: print a warning on upgrades if PREFERRED_VERSION is setAlexander Kanavin2021-08-021-0/+3
| | | | | | | (From OE-Core rev: 0b0f53eed0aadbf45d9eead96ebf7725cc7447e6) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta: Manual override fixesRichard Purdie2021-08-021-4/+4
| | | | | | | | | | The automated conversion of OE-Core to use the new override sytax isn't perfect. This patches some mis-converted lines and some lines which were missed by the automation. (From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Convert to new override syntaxRichard Purdie2021-08-025-12/+12
| | | | | | | | | | | | This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". (From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: correctly handle non-standard source tree locations in upgradesAlexander Kanavin2021-07-161-3/+12
| | | | | | | | | | | | | | | When S is set to a sub-directory of upstream source, the license checks and the bbappend writing (specifically, setting EXTERNALSRC) need to operate on that sub-directory. 'devtool modify' already has similar logic, and it was copied from there and adjusted. (From OE-Core rev: fd77e356d4507405fde352c8bba7d3842518bbdd) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Fix preserving attributes when using --stripFlorian Amstutz2021-07-021-1/+1
| | | | | | | | | | | | | | Commit a2db4fa127a3347fc6df31f895fb0b552669119e added ${WORKDIR}/deploy-* to PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to deploy-target-stripped. Use the directory devtool-deploy-target-stripped instead. [YOCTO #14451] (From OE-Core rev: 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9) Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool upgrade: rebase override-only patches as wellAlexander Kanavin2021-06-261-15/+14
| | | | | | | | | | | | | | | | | | | | | | There's a number of recipes (such as nettle or glib) which were upgraded incorrectly due to rebasing only the default set of patches in main SRC_URI. Native-only or musl-only patches were not handled, causing do_patch fails and overall misery. Unfortunately, this also necessitates aborting any incomplete rebases as devtool needs to be able to return to the original branch to complete the upgrade command. Also, do not add devtool-patched-%version tags, as they are not used anywhere, and it's unclear how to extend them to cover multiple override branches. (From OE-Core rev: 00131cbc969f6a73caffe068fcb983287cbe577f) (From OE-Core rev: 588d04748dd89b5371ca9144666e97ec82dd6204) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* classes/lib/scripts: Use bb.utils.rename() instead of os.rename()Devendra Tewari2021-05-062-5/+6
| | | | | | | | | | | | | | | | | | Incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than adding fallback code to every call site, use a new wrapper in bitbake which detects this case and falls back to shutil.move which is slower but will handtle the overlay docker filesystems correctly. [YOCTO #14301] (From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix do_kernel_configme taskAlejandro Hernandez Samaniego2021-03-141-1/+6
| | | | | | | | | | | | | | | | | | | | | | | The do_kernel_configme task is no longer part of SRCTREECOVEREDTASKS, its been removed from the kernel-yocto.bbclass since b72dbb2e4, but there wasnt a matching patch for devtool for those changes. This patch enables us to invoke the do_kernel_configme task when using a devtool workspace, it also prepends a check for an existing .config file in the source directory and moves it if thats the case, since when using devtool modify a .config is created and do_kernel_configme complains about it, this is not the case when using bitbake since the .config file would be on B instead. Alowing do_kernel_configme to run also fixes the flow where testing a new config fragment from devtool workspace isnt added properly (config queue shows it as ///frg.cfg) and as a side effect it never gets merged into the final config. (From OE-Core rev: 08dcc0e68095dcf2a159546a48b29d40c9aabc0b) Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix file:// fetcher symlink directory structureTomasz Dziendzielski2021-01-231-1/+5
| | | | | | | | | | | Add relative path if file is under directory. [YOCTO #13738] (From OE-Core rev: 19ddacc1b38f9ebb86a9359963ccc3c707f7125e) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Fix source extraction for gcc shared sourceTomasz Dziendzielski2021-01-201-4/+26
| | | | | | | | | | | | If do_patch task is disabled then prepare do_configure dependencies to fetch external sources and create symlink to ${S} in devtool workspace. [YOCTO #13036] (From OE-Core rev: 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d) Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: gitsm:// should be handled same as git:// in upgradesAlexander Kanavin2020-12-311-2/+2
| | | | | | | (From OE-Core rev: e7076f1742bb83e76aa4e90aad861546536166cf) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: make sure .git/info exists before writing to .git/info/excludesRoland Hieber2020-10-171-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | If nothing else is specified, 'git init' uses its default repository template from the install location (e.g. /usr/share/git-core/templates), which already includes an info/ subdirectory. However, when setting init.templateDir to a different template path in ~/.gitconfig, this isn't necessarily the case, and it can lead to setup_git_repo() failing with stack traces like: File: '.../scripts/lib/devtool/__init__.py', lineno: 234, function: setup_git_repo 0230: pass 0231: if 'singletask.lock\n' not in excludes: 0232: excludes.append('singletask.lock\n') 0233: bb.warn("try writing excludefile") *** 0234: with open(excludefile, 'w') as f: 0235: for line in excludes: 0236: f.write(line) 0237: 0238: bb.process.run('git checkout -b %s' % devbranch, cwd=repodir) Exception: FileNotFoundError: [Errno 2] No such file or directory: '.../devtooltmp-6m36b181/workdir/foobar-1.0.1/.git/info/exclude' Fix this edge case by creating the .git/info/ directory first. Fixes: 334ba846c795fc0d8c73 (2018-02-01, "devtool: set up git repos so that singletask.lock is ignored") (From OE-Core rev: 148a23e4d5ceaf655ccacb52deca4ba501f12975) Signed-off-by: Roland Hieber <rhi@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: remove unused variableRoss Burton2020-10-081-1/+0
| | | | | | | | | This variable isn't used at all. (From OE-Core rev: 9676ee24d190b144cb50be6ebde3990644a6df2a) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: Fix size calculation for hard linksMichael Tretter2020-09-021-1/+7
| | | | | | | | | | | | | | | | | | | | | | If a package contains hard links to a file, the file size is added for each hard link instead of once for the file. Therefore, the calculated size may be much larger than the actual package size. For example, the mesa-megadriver package contains several hard links to the same library. Keep track of the inode numbers when listing the files that are installed and use the actual size only for the first occurrence of an inode. All further hard links to the same inode are added to the file list, but accounted with size 0. All file names need to be added to the file list, because the list is used for preserving the files/hard links on the target. (From OE-Core rev: 17e92572278980d1a7f06de9d72c68baf57698f1) Signed-off-by: Michael Tretter <m.tretter@pengutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: expand SRC_URI when guessing recipe update modeMartin Jansa2020-08-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * I have recipes which use variable inside SRC_URI, e.g.: ROS_BRANCH ?= "branch=release/melodic/swri_nodelet" SRC_URI = "git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https" and devtool modify works fine, but devtool finish fails with: $ devtool finish --force-patch-refresh swri-nodelet meta-ros/meta-ros1-melodic/ ... Traceback (most recent call last): File "/OE/openembedded-core/scripts/devtool", line 334, in <module> ret = main() File "/OE/openembedded-core/scripts/devtool", line 321, in main ret = args.func(args, config, basepath, workspace) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 2082, in finish updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1737, in _update_recipe mode = _guess_recipe_update_mode(srctree, rd) File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1721, in _guess_recipe_update_mode params = bb.fetch.decodeurl(uri)[5] File "/OE/bitbake/lib/bb/fetch2/__init__.py", line 390, in decodeurl raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s)) bb.fetch2.MalformedUrl: The URL: 'git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https' is invalid: parameter ${ROS_BRANCH} does not specify a value (missing '=') let it expand the SRC_URI before trying to decode it. (From OE-Core rev: 3535cfdbf3d77f550b804276f957acf859da484f) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: default to empty string, if LIC_FILES_CHKSUM is not availableBjarne Michelsen2020-07-081-2/+2
| | | | | | | | | | | | | | | | | | [Bug 13971] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13971 A recipe using LICENSE=CLOSED would normally not have LIC_FILES_CHKSUM set. If LIC_FILES_CHKSUM was not set, it would cause `devtool upgrade` to fail. This fixes the issue by defaulting to an empty string if LIC_FILES_CHKSUM is not found. [YOCTO #13971] (From OE-Core rev: 95e112f0f465868e98285509ea0d1a624a439ac2) Signed-off-by: Bjarne Michelsen <bjarne_michelsen@hotmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix typoPaul Eggleton2020-06-231-2/+2
| | | | | | | | | specifiy -> specify (From OE-Core rev: 194555aae3f1f10468c1800c93bd17c54ce217c8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: use -f and don't use --exclude-standard when adding files to workspaceMartin Jansa2020-06-042-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * I see a case where a tarball contains .gitignore and bunch of files which are normally ignored in git, but still included in the tarball (e.g. configure script next to configure.ac) * when devtool is creating a git repo in workspace it won't include these files from tarball in the initial devtool-base commit, because git ls-files won't list them * but then the first .patch file (without git headers) when applied with GitApplyTree._applypatch() will add all these still ignored files to a commit which used to only modify some files, because it's using -f: # Add all files shellcmd = ["git", "add", "-f", "-A", "."] output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir) at least in this case it would be better to add all ignored files in the initial devtool-base commit and then --force-patch-refresh will just include the small modification as before instead of adding unrelated files, just because they were initially ignored - this behavior will also match with the do_patch task in the actual build where the .gitignore is ignored when unpacking some tarball * my use-case is fixed in setup_git_repo, but similar function is in devtool upgrade, I've changed it there as well (From OE-Core rev: 06a24a615549af3550302a56ea08147000a608f3) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: do not write md5sums into upgraded recipesAlexander Kanavin2020-05-211-2/+2
| | | | | | | | | | | | This will drop them md5sums from recipes that still have them, and will not re-add them for recipes where they're already removed. [RP: Added fix for the test recipe] (From OE-Core rev: 1609e77a4db04a4c01cbbff17261cd57e8f816ee) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard.py: Allow recipe to disable menuconfig logicTom Hochstein2020-02-021-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | u-boot.inc supports u-boot recipes with or without menuconfig [1]. However, running devtool on a u-boot recipe that does not support menuconfig results in an error: cp: cannot stat '/home/r60874/upstream/fsl-xwayland/tmp/work/imx8mmevk-fsl-linux/u-boot-imx/2018.03-r0/u-boot-imx-2018.03//.config': No such file or directory The problem is the devtool logic assumes that any recipe with a do_menuconfig task will generate a .config in do_configure(). Fix the problem by removing the assumption with a flag that the recipe can control, like this: do_configure() { if [ menuconfig-supported ]; then ... else DEVTOOL_DISABLE_MENUCONFIG=true fi } [1] https://github.com/openembedded/openembedded-core/commit/11278e3b2c75be80645b9841763a97dbb35daadc (From OE-Core rev: 6cc17e658c3dbadc772353f0f200561d2d988658) Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard: npm: update the append fileJean-Marie LEMETAYER2020-01-271-8/+4
| | | | | | | | | | | | | | | | | When creating a recipe using devtool, a workspace is created to store the new recipe, the recipe source and some append files. These append files are used by devtool to build the recipe using externalsrc (to use the source which are in the workspace). They can also have some additional actions according to the class of the recipe. This commit updates the append file for the npm recipes. The devtool / externalsrc files are removed in the npm package directory instead of the install directory. (From OE-Core rev: a213a76e56e9bc215a61399ae35a3602863f5850) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: npm: rename npm command line optionsJean-Marie LEMETAYER2020-01-271-3/+3
| | | | | | | | | | This commit renames the '--fetch-dev' option into '--npm-dev' as it is a npm only option. (From OE-Core rev: 2b75cc848ceebee4067788a621299bfd5fb62231) Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix devtool upgrade with reproducible_builds classPaul Eggleton2019-11-211-2/+3
| | | | | | | | | | | | | | | | | | If the reproducible_build class is inherited then there may be a "source-date-epoch" subdirectory in a fetched source tree; devtool upgrade was not expecting that in the upgraded source. Take a small snippet of code from recipetool create which already handles this, and make it a shared function that can be used in both places. Additionally, fix an assumption that the source is always in a subdirectory in the cleanup code that blocked debugging this. [YOCTO #13635] (From OE-Core rev: 0d642861cd9cf034b8d4951433980addc215d4fd) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Revert "devtool/standard.py: Not filtering devtool workspace for devtool finish"Alexander Kanavin2019-11-211-1/+1
| | | | | | | | | | | | | This reverts commit 41d225f4a37d02e9f79bdbfb79caac8cd3d291ce. Unfortunately this change broke 'devtool upgrade' functionality, causing 'devtool finish' to write out an upgraded recipe that no longer includes the original upstream source in SRC_URI. (From OE-Core rev: 2d6e55192dba0bf7f6e23e5ab5b3dbc68835bb28) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: add ssh key option to deploy-target paramEduardo Abinader2019-10-231-0/+8
| | | | | | | | (From OE-Core rev: 729da1414098c2dd103d9749cf39117881cafeec) Signed-off-by: Eduardo Abinader <eduardoabinader@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Add --remove-work option for devtool reset commandSai Hari Chandana Kalluri2019-10-231-7/+19
| | | | | | | | | | | | | | | | | | | | | | | | | Enable --remove-work option for devtool reset command that allows user to clean up source directory within workspace. Currently devtool reset command only removes recipes and user is forced to manually remove the sources directory within the workspace before running devtool modify again. Using devtool reset -r or devtool reset --remove-work option, user can cleanup the sources directory along with the recipe instead of manually cleaning it. syntax: devtool reset -r <recipename> Ex: devtool reset -r zip devtool finish -r <recipename> <layer-name> Ex: devtool finish -r zip meta-yocto-bsp (From OE-Core rev: 0d6dd01b8633ad63d77c24568a4b20852aaa366f) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard.py: Not filtering devtool workspace for devtool finishJaewon Lee2019-10-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | All devtool commands right now are filtering out the devtool workspace bbappends in build/workspace/appends when calling parse_recipe. While this may make sense for devtool add and modify, we need devtool finish to include those appends. A specific breakage that is caused because devtool finish filters devtool appends is the cmake/cml1 flow where a file is created in the WORKDIR that finish needs access to, to commit those files. Particularly for git packages with SRCPV in PV, SRCPV is only changed to 999 when using external source, hence when creating the cfg or cmake config files using for instance bitbake -c diffconfig, these files are created in the git999 workdir correctly (as in the devtool bbapends, we are inheriting externalsrc class). But when devtool finish is run, the devtool appends are not parsed, hence SRCPV is not changed to 999 and devtool is looking for the fragment files in the wrong WORKDIR. Changing the parse_recipe call just in devtool finish to not filter out the devtool workspace. Fixes [YOCTO #13533] (From OE-Core rev: aa94f00eec64ef936acc80dfa826fd309daa294f) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish: Add suppport for the --no-clean optionPeter Kjellerstedt2019-09-271-1/+2
| | | | | | | | | | This works just like the already existing --no-clean option to the `devtool reset` command. (From OE-Core rev: 4433d3133ce53cdf7db17f5e7914fe8c4d78dba3) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish: Keep patches ordered when updating bbappendNiclas Svensson2019-09-271-4/+4
| | | | | | | | | | | | The _get_patchset_revs() function returns the patches in an OrderedDict to keep them ordered. However, this information was lost when the patches were added to the bbappend file. (From OE-Core rev: 69b7a2ba3af1280cc220ab236032f8466246ee93) Signed-off-by: Niclas Svensson <niclas.svensson@axis.com> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: Avoid failure for recipes with S == WORKDIR and no local filesPeter Kjellerstedt2019-09-011-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When extracting the sources for a recipe that has S == WORKDIR and no local files in the SRC_URI (which, e.g., can happen for a recipe with a URI that has the unpack=false attribute), the extraction fails with the following backtrace: Traceback (most recent call last): File ".../scripts/devtool", line 344, in <module> ret = main() File ".../scripts/devtool", line 331, in main ret = args.func(args, config, basepath, workspace) File ".../poky/scripts/lib/devtool/standard.py", line 762, in modify initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) File ".../poky/scripts/lib/devtool/standard.py", line 647, in _extract_source bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree) File ".../poky/bitbake/lib/bb/process.py", line 178, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git commit -a -m "Committing local file symlinks %% ignore"' failed with exit code 1: On branch devtool nothing to commit, working tree clean This is because no files were found in the oe-local-files directory and consequently no symbolic links were added using `git add`, but the `git commit` command was still executed. (From OE-Core rev: 3fdf304e72a1fb5de8bf9bc21e5b598fefb08648) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: build: Also run deploy for devtool build if applicableJaewon Lee2019-08-211-1/+12
| | | | | | | | | | | | | | | | Right now `devtool build` runs populate_sysroot and packagedata tasks. Adding deploy to this list, if the recipe has the deploy task, so that the newly built artifacts are available in the deploy directory. Applicable only for packages with deploy task, such as kernel. [YOCTO#13382] (From OE-Core rev: b38a1328f0c7bc4b4102a05daee4058fd3214489) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: remove temp dir in upgradeChen Qi2019-07-221-0/+1
| | | | | | | | | | For now, the temp dir is left in system, although the temporary source directory has been cleaned up. So we clean it up too. (From OE-Core rev: 8a0602327d5afcf4f36850d3f05c9721305852af) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: provide support for devtool menuconfig commandSai Hari Chandana Kalluri2019-07-172-0/+84
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All packages that support the menuconfig task will be able to run devtool menuconfig command. This would allow the user to modify the current configure options and create a config fragment which can be added to a recipe using devtool finish. 1. The patch checks if devtool menuconfig command is called for a valid package. 2. It checks for oe-local-files dir within source and creates one if needed, this directory is needed to store the final generated config fragment so that devtool finish can update the recipe. 3. Menuconfig command is called for users to make necessary changes. After saving the changes, diffconfig command is run to generate the fragment. Syntax: devtool menuconfig <package name> Ex: devtool menuconfig linux-yocto The config fragment is saved as devtool-fragment.cfg within oe-local-files dir. Ex: <workspace_path>/sources/linux-yocto/oe-local-files/devtool-fragment.cfg Run devtool finish to update the recipe by appending the config fragment to SRC_URI and place a copy of the fragment within the layer where the recipe resides. Ex: devtool finish linux-yocto meta [YOCTO #10416] (From OE-Core rev: 417feb559a74b367315e8658d6ba868a4f8d1340) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard.py: Create a copy of kernel source within work-shared if ↵Sai Hari Chandana Kalluri2019-07-171-1/+19
| | | | | | | | | | | | | | | | | | not present If kernel source is not already downloaded i.e staging kernel dir is empty, place a copy of the source when the user runs devtool modify linux-yocto. This way the kernel source is available for other packages that use it. [YOCTO #10416] (From OE-Core rev: bb42ab90835e8ec2f1dfbb35056c353784693266) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/standard.py: Update devtool modify to copy source from work-shared ↵Sai Hari Chandana Kalluri2019-07-171-23/+99
| | | | | | | | | | | | | | | | | | | | | if its already downloaded In the regular devtool modify flow, the kernel source is fetched by running do_fetch task. This is an overhead in time and space. This patch updates modify command to check if the kernel source is already downloaded. If so, then instead of calling do_fetch, copy the source from work-shared to devtool workspace by creating hard links else run the usual devtool modify flow and call do_fetch task. [YOCTO #10416] (From OE-Core rev: 3c3a9bae296f849dbfe03942282f44036e6fa1fb) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: upgrade: fix handling of errors parsing upgraded recipePaul Eggleton2019-07-031-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As part of upgrading a recipe we create the upgraded recipe file in the workspace and then try to parse it so we can then make further modifications. If for some reason that parsing fails then the failure was not being handled very well - the broken recipe was being left in place, breaking parsing until it was removed by hand. Fix that by adding a call to the cleanup function, and fix the following issues: * Fix the cleanup function which doesn't look like it has ever worked due to a typo in the function call * Fix double-printing the error message * Remove usage of DevtoolError in this case (DevtoolError is for simple usage errors, not this kind of issue which may be the result of a bug). We're still printing a traceback in this scenario but at least it doesn't break the build system requiring manual cleanup. I also introduced a command-line option to preserve the broken upgraded recipe file(s) for debugging purposes. (The reproducer for this is "devtool upgrade libnewt-python", however you need to check out revision b82ea144e144671d3f64c0785ba4beafe905cd4f or earlier since that recipe has now been absorbed into the libnewt recipe. The libnewt-python recipe was causing an issue with the upgrade because it actually included the libnewt recipe using ${PV} in the include statement, and of course PV was changing in the upgrade.) Fixes [YOCTO #13404]. (From OE-Core rev: c519ac360796675d7fc09a5250d21f0f5b6236fc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: warn user about multiple layer having the same base nameChen Qi2019-06-281-4/+18
| | | | | | | | | | | | | | | | | | | Currently `devtool finish RECIPE meta' will silently succeed even if there are multiple layers having the same base name of 'meta'. e.g. meta layer from oe-core and meta layer from meta-secure-core. We should at least give user a warning in such case. With the patch, we will get warning like below. WARNING: Multiple layers have the same base name 'meta', use the first one '<PROJ_DIR>/oe-core/meta'. WARNING: Consider using path instead of base name to specify layer: <PROJ_DIR>/oe-core/meta <PROJ_DIR>/meta-secure-core/meta (From OE-Core rev: 2c8740f543c38dbaef3345e40827ef48b3f75405) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* meta/lib+scripts: Convert to SPDX license headersRichard Purdie2019-05-0914-155/+14
| | | | | | | | | | | | | | | | | | | | | | | This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-07-07 13:50:42 +0000