| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
| |
Work through the initial issues I found where we need to change WORKDIR
to UNPACKDIR.
(From OE-Core rev: 86fec41b1e809d1a2fa2feadc26d29020df53d39)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The old way of keeping track of the filenames for the patches that
correspond to the commits was to add a special comment line to the end
of the commit message, e.g., "%% original patch: <filename>", using a
temporary git hook. This method had some drawbacks, e.g.:
* It caused problems if one wanted to push the commits upstream as the
comment line had to be manually removed.
* The comment line would end up in patches if someone used git
format-path rather than devtool finish to generate the patches.
* The comment line could interfere with global Git hooks used to
validate the format of the Git commit message.
* When regenerating patches with `devtool finish --force-patch-refresh`,
the process typically resulted in adding empty lines to the end of the
commit messages in the updated patches.
A better way of keeping track of the patch filenames is to use Git
notes. This way the commit messages remain unaffected, but the
information is still shown when, e.g., doing `git log`. A special Git
notes space, refs/notes/devtool, is used to not intefere with the
default Git notes. It is configured to be shown in, e.g., `git log` and
to survive rewrites (i.e., `git commit --amend` and `git rebase`).
Since there is no longer any need for a temporary Git hook, the code
that manipulated the .git/hooks directory has also been removed. To
avoid potential problems due to global Git hooks, --no-verify was added
to the `git commit` command.
To not cause troubles for those who have done `devtool modify` for a
recipe with the old solution and then do `devtool finish` with the new
solution, the code will fall back to look for the old strings in the
commit message if no Git note can be found.
While not technically motivated like above, the way to keep track of
ignored commits is also changed to use Git notes to avoid having
different methods to store similar information.
(From OE-Core rev: f5e6183b9557477bef74024a587de0bfcc2b7c0d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
This function can be used to create a commit that devtool will ignore
when creating/updating the patches.
(From OE-Core rev: 94f0838b9223b7ece7affaa707e54a5d784da25e)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a commit is marked with "%% ignore" it means it is used by devtool to
keep track of changes to the source code that are not the result of
running do_patch(). These changes need to actually be ignored when
extracting the patches as they typically make no sense as actual patches
in a recipe.
This also adds a new test for oe-selftest that verifies that there are
no patches generated from ignored commits.
(From OE-Core rev: c3d43de7e54189bf09fbe8e87ddb976e42ebf531)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Using devtool to patch CRLF based sources creates patch files which have
mixed end of lines : LF for headers and CRLF for source context and
modified lines.
Python open(..., newline=None) (default for newline arg)does detect
end-of-line in this mixed file but only outputs LF EOL data. This
result in patch files that does not apply on the original sources.
Switching to open(..., newline='') allows to detect end-of-line but keep
the original end-of-line intact. This generate correct patches for CRLF
based sources.
Fixes [YOCTO #15285]
(From OE-Core rev: 58f845499c0277a2b8069eefa235430b5f5f7661)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
/ updated commits was replaced by a dictionary where the keys are the
submodule name ("." for main repo) and the values are the
initial_rev/startcommit
* the extractPatches function now extracts patch for the main repo and
for all submodules and stores them in a hierarchical way describing the
submodule path
* store initial_rev/commit also for all submodules inside the recipe
bbappend file
* _export_patches now returns dictionaries that contains the 'patchdir'
parameter (if any). This parameter is used to add the correct
'patchdir=' parameter on the recipe
Also, recipe can extract a secondary git tree inside the workdir.
By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."
The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.
So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule
(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
If we chdir(), do the chdir back to the original directory in a finally
block so they always run.
(From OE-Core rev: cdc40292818683b6df1c814498c7589450a163fa)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
[YOCTO #14141]
Currently the gitdir is manually constructed using `git
rev-parse --show-toplevel` and appending `.git`. This is most of the time
correct but not always: `.git` can be a file with the following content:
gitdir: <some_folder>
This is the case for submodules, so when using devtool modify on a recipe
using submodules *and* patching files inside one of the submodules, do_patch
fails with the following error:
ERROR: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:patch_do_patch(d)
0003:
File: '<..>/poky/meta/classes-global/patch.bbclass', lineno: 157, function: patch_do_patch
0153: except Exception as exc:
0154: bb.utils.remove(process_tmpdir, True)
0155: bb.fatal("Importing patch '%s' with striplevel '%s'\n%s" % (parm['patchname'], parm['striplevel'], repr(exc).replace("\\n", "\n")))
0156: try:
*** 0157: resolver.Resolve()
0158: except bb.BBHandledException as e:
0159: bb.utils.remove(process_tmpdir, True)
0160: bb.fatal("Applying patch '%s' on target directory '%s'\n%s" % (parm['patchname'], patchdir, repr(e).replace("\\n", "\n")))
0161:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 769, function: Resolve
0765: def Resolve(self):
0766: olddir = os.path.abspath(os.curdir)
0767: os.chdir(self.patchset.dir)
0768: try:
*** 0769: self.patchset.Push()
0770: except Exception:
0771: import sys
0772: os.chdir(olddir)
0773: raise
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 274, function: Push
0270: else:
0271: next = 0
0272:
0273: bb.note("applying patch %s" % self.patches[next])
*** 0274: ret = self._applypatch(self.patches[next], force)
0275:
0276: self._current = next
0277: return ret
0278:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 556, function: _applypatch
0552: if os.path.lexists(hooks_dir_backup):
0553: raise Exception("Git hooks backup directory already exists: %s" % hooks_dir_backup)
0554: if os.path.lexists(hooks_dir):
0555: shutil.move(hooks_dir, hooks_dir_backup)
*** 0556: os.mkdir(hooks_dir)
0557: commithook = os.path.join(hooks_dir, 'commit-msg')
0558: applyhook = os.path.join(hooks_dir, 'applypatch-msg')
0559: with open(commithook, 'w') as f:
0560: # NOTE: the formatting here is significant; if you change it you'll also need to
Exception: NotADirectoryError: [Errno 20] Not a directory: '<..>/build/tmp/work/core2-64-poky-linux/vulkan-samples/git/devtooltmp-n87_zx1i/workdir/git/third_party/spdlog/.git/hooks'
Using `git rev-parse --absolute-git-dir` instead of `git rev-parse
--show-toplevel` ensure we get the correct gitdir
(From OE-Core rev: f74879dd95b19504ce8a8554636d2310d0336806)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The pipe library is deprecated in Python 3.11 and will be removed in
Python 3.13. pipe.quote is just an import of shlex.quote anyway.
Clean up imports while we're at it.
(From OE-Core rev: 5f33c7b99a991c380d1813da8248ba5470ca4d4e)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is for a specific case where:
- A recipe use a subpath on a git repo (e.g. git://repo.git/projects;subpath=subproject)
- The recipe contains a patch to apply
- a devtool modify is used on this recipe
With these conditions, the patch cannot be applied at all.
GitApplyTree class is used for handling patch under devtool, but
when subpath is present in SRC_URI, the resulting git tree
is dirty (every files and directories which was not in subpath are suppressed)
and so "git am" refuse to apply patches.
That would not be an issue since the GitApplyTree have a fallback
to PatchTree in case of error, but during this error management,
there is a "git reset --hard HEAD" call which suppress the subpath
operation and finally prevents the patch to be applied even with PatchTree.
When devtool is not involved, only PatchTree class is used and the
above problem is irrelevant.
To support git patching during devtool, the presence of subpath and
the dirtyness of the repo are checked. If both conditions are
met, we directly call PatchTree like it was already done
in case of error during git apply.
(From OE-Core rev: d86cac2759cf7e91f4ff240833385e28e729ab79)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.
(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.
(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)
(From OE-Core rev: 9cca53a2bcbf6809615ce5626c86c6ee481a7a76)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Since quilt upgrade to 0.67 some recipes sometimes fail in do_patch with
errors like:
ERROR: Applying patch 'GPLv2.patch' on target directory '/OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31'
CmdError('quilt --quiltrc /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
stderr: File series fully applied, ends at patch GPLv2.patch
')
* It affects only recipes with S = "${WORKDIR}", which wipe only
${S}/patches, because in other cases whole ${S} is wiped when
do_unpack is re-executed.
* It was originally added in:
https://git.openembedded.org/openembedded-core/commit/?id=5fe5e6a15f26f23f0c5b863fafad7a0d382a55e2
since then it was extended to wipe whole ${S} when ${S} != ${WORKDIR} in:
https://git.openembedded.org/openembedded-core/commit/?id=5fe5e6a15f26f23f0c5b863fafad7a0d382a55e2
https://git.openembedded.org/openembedded-core/commit/?id=eccae514b71394ffaed8fc45dea7942152a334a1
this is now causing issues to quilt-0.67 because it checks that
${S}/patches/series exists during 'quilt pop -a -f' which we call
from QuiltTree.Clean to undo patches possibly already applied
in ${S} in previous do_patch execution.
* There are couple recipes affected by this e.g. keymaps (.patch already
removed in oe-core), makedevs (.patch removal sent to ML yesterday
https://lists.openembedded.org/g/openembedded-core/message/166172),
devmem2
(https://lists.openembedded.org/g/openembedded-devel/message/97270), but
there are other recipes with S = "${WORKDIR}" where you can trigger this
e.g. by having a .patch file in DISTRO layer .bbappend (e.g. tzdata with
webOS
https://github.com/webosose/meta-webosose/blob/06e5298d9f5c47679b679081d9930f8d1c776142/meta-webos/recipes-extended/tzdata/tzdata.bbappend#L10)
This do_patch issue is caused by:
https://git.savannah.nongnu.org/cgit/quilt.git/commit/?id=8b39a960afcf45cd4f5804ae62b6b0656bdb191d
introduced in kirkstone with:
https://git.openembedded.org/openembedded-core/commit/?h=kirkstone&id=fa71afcee9ab42198c619333b77a15bd2ae02b20
The shortest sequence to reproduce this is just
bitbake keymaps -c patch
bitbake keymaps -c unpack -f
bitbake keymaps -c patch
with
https://git.openembedded.org/openembedded-core/commit/?id=17d981005a0c0c97702ad88602b7181b69bcc9eb
reverted.
And the change in quilt behavior is causing QuiltTree.Clean (quilt pop -a -f) in:
https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/patch.py?id=17d981005a0c0c97702ad88602b7181b69bcc9eb#n601
to silently fail with "No series file found" before undoing the
patches in ${S} and then quilt push failing, because all the
patches are _still_ applied in ${S}.
Removing ".pc" doesn't help, because we really
need quilt's help to undo the patches (in this case to delete COPYING
file from WORKDIR before applying the .patch which tries to add it
again), because do_unpack cannot just wipe S and start over (because S
== WORKDIR) - nor selectively removing the files listed in SRC_URI,
because COPYING file isn't listed there.
Using skip_series_check in 'quilt pop' (partially reverting the change
from upstream) does fix this as well and it's simple one line patch
(just adding skip_series_check=1 in pop.in), but might be difficult
to upstream, because it's this strange OE specific behavior that we
remove 'patches' directory and then still need quilt pop to work.
(From OE-Core rev: c9d36882044b1c633d8611a77df54cd68c9bee25)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
| |
There were few bugs in the _isInitialized() function which might trigger
git repo to be reinitialized and patches failing to apply.
(From OE-Core rev: 80500ecda4c1bc8812e6e078b6b0db5ec46624de)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If PATCHTOOL="git" has been specified but workdir is not git repo
bitbake fails to apply the patches with error message:
Command Error: 'git rev-parse --show-toplevel' exited with 0 Output:
fatal: not a git repository (or any of the parent directories): .git
Fix this by initializing the repo before patching.
This allows binary git patches to be applied.
(From OE-Core rev: 6184b56a7a0fc6f5d19fdfb81e7453667f7da940)
Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The "devtool modify" could remove message body before scissors line, so
patches re-generated from git tree were incorrectly modified.
Adding --no-scissors to "git am" invocation to prevent this behaviour.
[YOCTO #12674]
(From OE-Core rev: 13ea33fbd197b9ee3cf913d9995617115f22798f)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a function returns any stderr it will be passed to extractPatches and
used as path to patch.
For example subprocess command output can be:
| sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
| /tmp/oepatchhuqle8fj/0001-foo.patch
| /tmp/oepatchhuqle8fj/0002-bar.patch
that will result in:
| FileNotFoundError: [Errno 2] No such file or directory: 'sh:'
To fix this I separated output, made the function return stdout and
print stderr only in case of command error.
(From OE-Core rev: 482589e2cc7c3ddeefb0a0fb98d97a9cbb18c9ec)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When applying patch subject is including a " fuzz ", do_patch_qa detect
Fuzz.
After the patch is applied, the following log message appears.
Applying: meson: treat all fuzz cases as unit tests
The current Fuzz detection checks for the presence of "fuzz" in this
log message. The log in this example will be treated as Fuzz,
despite its success.
This patch change to more strictly fuzz detection.
if log message is including " fuzz " and "Hunk " in log message,
it will be treated as Fuzz.
(From OE-Core rev: a8605c66ef5afe7c3583366781dfd90fe3526398)
Signed-off-by: Naoto Yamaguchi <wata2ki@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* This reverts commit d9971f5dc8eb7de551fd6f5e058fd24770ef5d78.
* With the missing Subject line fixed in GitApplyTree.prepareCommit()
we should be able to revert, the fix which was trying to help it by
parsing GitApplyTree.patch_line_prefix ("%% original patch:") also
from Subject line, now GitApplyTree.patch_line_prefix should always
end on separate line which is then skipped when copying the lines to
resulting patch, see original commit message from Paul:
lib/oe/patch: fix handling of patches with no header
If a patch applied by a recipe has no header and we turn the recipe's
source into a git tree (when PATCHTOOL = "git" or when using devtool
extract / modify / upgrade), the commit message ends up consisting only
of the original filename marker ("%% original patch: filename.patch").
When we come to do turn the commits back into a set of patches in
extractPatches(), this first line ends up in the "Subject: " part of
the file, but we were ignoring it because the line didn't start with the
marker text. The end result was we weren't able to get the original
patch name. Strip off any "Subject [PATCH x/y]" part before looking for
the marker text to fix.
This caused "devtool modify openssl" followed by "devtool update-recipe
openssl" (without any changes in-between) to remove version-script.patch
because that patch has no header and we weren't able to determine the
original filename.
(From OE-Core rev: d9e56db415d386447a299dd633b10f1eda0dd401)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* also remove the extra blank lines which is often added to patches
when refreshed with devtool (GitApplyTree.patch_line_prefix lines
are ignored when refreshing .patch files, but newly added blank
lines aren't - the leading blank line wasneeded for patches with
just the subject line (to prevent the GitApplyTree.patch_line_prefix
line ending appended to the commit summary), but we can add it
in prepareCommit instead
(From OE-Core rev: c50c0d6144ad290168167ccef948c7b4ffc9665a)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* this was discovered with
$ devtool finish --force-patch-refresh
where it was removing some patches and replacing them with
patch in filename called "patch:"
e.g. this .patch file:
https://github.com/OSSystems/meta-browser/blob/311067d2d8a50cee5c836892606444f63f2bb3ab/dynamic-layers/rust-layer/recipes-browser/firefox/firefox/fixes/fix-camera-permission-dialg-doesnot-close.patch
confuses devtool which results to create new .patch file called "patch:"
$ devtool finish --force-patch-refresh firefox meta-browser
NOTE: Starting bitbake server...
WARNING: Host distribution "ubuntu-20.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Loading cache: 100% |###################################################################################################################################################################################################################################| Time: 0:00:00
Loaded 2480 entries from dependency cache.
Parsing recipes: 100% |#################################################################################################################################################################################################################################| Time: 0:00:00
Parsing of 1718 .bb files complete (1717 cached, 1 parsed). 2480 targets, 68 skipped, 0 masked, 0 errors.
Summary: There was 1 WARNING message shown.
INFO: Updating patch 0001-Bug-1554949-Fix-WebRTC-build-failure-with-newer-linu.patch
...
INFO: Updating patch pre-generated-old-configure.patch
INFO: Adding new patch patch:
INFO: Updating recipe firefox_68.0esr.bb
INFO: Removing file /OE/build/test-oe-build-time/poky/meta-browser/dynamic-layers/rust-layer/recipes-browser/firefox/firefox/fixes/fix-camera-permission-dialg-doesnot-close.patch
INFO: Cleaning sysroot for recipe firefox...
INFO: Leaving source tree /OE/build/test-oe-build-time/poky/build/workspace/sources/firefox as-is; if you no longer need it then please delete it manually
this looked like incorrect parsing of the git format-patch
files exported from workspace/sources (the git format-patch
version of fix-camera-permission-dialg-doesnot-close.patch
starts like this:
$ head 0008-original-patch-fix-camera-permission-dialg-doesnot-c.patch
From 37dfa11961b48024bedcfb9336f49107c9535638 Mon Sep 17 00:00:00 2001
From: Takuro Ashie <ashie@clear-code.com>
Date: Mon, 20 Aug 2018 10:16:20 +0900
Subject: [PATCH 08/34] %% original patch:
fix-camera-permission-dialg-doesnot-close.patch
so first I've modified GitApplyTree.extractPatches() to be able to
parse the original patch name correctly even in this case where subject
is wrapped, but then it still wasn't right, because we ended with
correctly named .patch file, but all we could use for Subject line
was the name of the original .patch file (instead of the Subject
from metadata commit which introduced this .patch files as some other
.patch files get when refreshed with devtool.
In the end the issue happens even sooner in GitApplyTree.prepareCommit()
where it correctly found the Subject from metadata commit, but then
didn't apply it when there weren't any other outlines from patch headers.
(From OE-Core rev: 0a8252f0cb13e3dc16f70b984f9f98b845b163de)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Add minver/maxver parameters to limit patch application by comparing PV.
(From OE-Core rev: 3fac9f884ac6bcc0280e1bb5d0f0e397bb53678f)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently there are three issues which can be enhanced:
1. Fuzz warnings cannot be configured as errors for hardening. It happened
often to me that these warnings were overseen and detected after commits
were already out.
2. The output is too verbose - particularly when more than one file is
affected. Meanwhile all users should know why patch fuzz check is performed.
So move links with background information to insane.bbclass.
3. Reduce copy & paste effort slightly by printing PN (nit: <recipe> was not
a correct suggestion e.g for native extended recipe - see example below)
To achieve patch.py drops patch-fuzz info encapsulated by a header- and footer-
string into log.do_patch. With this insane.bbclass can drop warnings/errors
depending on 'patch-fuzz' in ERROR_QA or WARN_QA. Default remains unchanged:
Spit out warnings only.
A message for two fuzzed patches and 'pact-fuzz' in ERROR_QA now looks like:
| ERROR: autoconf-native-2.69-r11 do_patch: Fuzz detected:
|
| Applying patch autoreconf-exclude.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 73 with fuzz 1 (offset -3 lines).
| Hunk #2 succeeded at 143 (offset 6 lines).
| Hunk #3 succeeded at 167 (offset 6 lines).
| Hunk #4 succeeded at 177 (offset 6 lines).
| Hunk #5 succeeded at 281 (offset 15 lines).
| Hunk #6 succeeded at 399 (offset 15 lines).
| Hunk #7 succeeded at 571 (offset 20 lines).
| Hunk #8 succeeded at 612 (offset 20 lines).
| Hunk #9 succeeded at 636 (offset 20 lines).
| Hunk #10 succeeded at 656 (offset 20 lines).
| Hunk #11 succeeded at 683 (offset 20 lines).
|
| Applying patch autoreconf-gnuconfigize.patch
| patching file bin/autoreconf.in
| Hunk #1 succeeded at 55 with fuzz 1 (offset -3 lines).
| Hunk #3 succeeded at 663 (offset 18 lines).
|
| The context lines in the patches can be updated with devtool:
|
| devtool modify autoconf-native
| devtool finish --force-patch-refresh autoconf-native <layer_path>
|
| Don't forget to review changes done by devtool!
|
| ERROR: autoconf-native-2.69-r11 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz]
(From OE-Core rev: c762c0be43a3854a43cb4b9db559b03126d50706)
Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a SRC_URI content ends with '.patch' bitbake is
trying to apply it as it's a patch file.
It causes that if we use git repository for 'patch' package
the bare clone is extracted to a directory
(i.e. build/downloads/git2/git.mirror.org.patch/) which is considered
to be a patch file, so patch.py tries to apply that directory as a patch
which ends up with a failure.
(From OE-Core rev: 1e38d74a2ca7638b1f54e2bb5617903c2683e484)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
If it's not a patch the function returns nothing.
(From OE-Core rev: cc4a75c7ebf24fb3b7b35552a090e8844985e45e)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.
Note that some show up as:
"""
meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.
"""
where the problem isn't on 1293 in package.bbclass but in some _prepend to a
package.bbclass function in a different file like mesa.inc, often from
do_package_split() calls.
(From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
We can't use subprocess.check_output() or subprocess.call() here since the one
who invokes runcmd() needs handle CmdError() exception (error out or ignore
it).
(From OE-Core rev: c3e7739987d804f7865428442479d5bece5ff2dd)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* --no-signature saves unnecessary .patch modifications when executed on
host with different git version
* --no-numbered saves unnecessary .patch modifications when number of the
applied patches is changed (the number is still in the filename so the
order how they should be applied is still preserved)
* both options exist for very long time, I've tested them with git 1.9.1
from Ubuntu 14.04 and I'm quite sure they were available even in much
older releases, so there shouldn't be any issue on relatively new sanity
tested distros
(From OE-Core rev: ad76fa92c3a5be38962aff09df070ffd9756f777)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ignoring patch context increases the chances of patches being
applied incorrectly. Depending on what code is being patched, this can go
completely unnoticed and create subtle bugs, sometimes with security implications.
Please see here for a specific example:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10450
On the other hand, we cannot simply force all patch context to match exactly:
doing this would break a lot of recipes suddenly, across all layers.
So let's try a softer approach: issue a warning, and gently update
patches over a longer span of time. When most of the warnings are eliminated,
we can start enforcing a strict patch application policy.
I do understand that this patch creates a lot of warnings all of a sudden, however
I believe the problem does need to be addressed. All of oe-core recipes have their
context already fixed.
Sample warning:
WARNING: vulkan-1.0.61.1-r0 do_patch:
Some of the context lines in patches were ignored. This can lead to incorrectly applied patches.
The context lines in the patches can be updated with devtool:
devtool modify <recipe>
devtool finish --force-patch-refresh <recipe> <layer_path>
Then the updated patches and the source tree (in devtool's workspace)
should be reviewed to make sure the patches apply in the correct place
and don't introduce duplicate lines (which can, and does happen
when some of the context is ignored).
Details:
Applying patch demos-Don-t-build-tri-or-cube.patch
patching file demos/CMakeLists.txt
Hunk #1 succeeded at 63 (offset 2 lines).
Hunk #2 succeeded at 76 with fuzz 1 (offset 2 lines).
[YOCTO #10450]
(From OE-Core rev: 5133fd46bccf14e21680f8d94e952914edccb113)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
matching
Particularly, this was causing 'devtool modify' to erroneously add those
.orig files into commits. This was getting in the way, if the goal
was to amend/update those existing patches.
(From OE-Core rev: f4f3406c3bd9599d7a19275475d52bda4c42f2ab)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
This module refers to oe.types, so it needs to actually import oe.types.
Fixes errors when parsing certain OE-Core recipes within the layer
index update script.
(From OE-Core rev: 26ff9d2835a24a84c7f2bf9c829a13ed568c9ea0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
| |
bb.data.expand(x, d) is deprecated API.
[YOCTO #10678]
(From OE-Core rev: a361babe443da635aed83be46679067457fd6a58)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add .xz to the list of extensions recognized by patch_path(), so that
compressed patches ending in .patch.xz or .diff.xz are automatically
applied.
(From OE-Core rev: f1a2c45765d14d3ca09657ad1f6b526554af2bb6)
(From OE-Core rev: f50fd7f247d5bb05bc7d1109c574a682067688da)
Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move patch_path(), src_patches() and should_apply() to oe.patch, making
them easier to call from elsewhere (particularly across the
UI/server boundary).
(From OE-Core rev: 2724511e18810cc8082c1b028e3b7c8a8b5def56)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a patch applied by a recipe has no header and we turn the recipe's
source into a git tree (when PATCHTOOL = "git" or when using devtool
extract / modify / upgrade), the commit message ends up consisting only
of the original filename marker ("%% original patch: filename.patch").
When we come to do turn the commits back into a set of patches in
extractPatches(), this first line ends up in the "Subject: " part of
the file, but we were ignoring it because the line didn't start with the
marker text. The end result was we weren't able to get the original
patch name. Strip off any "Subject [PATCH x/y]" part before looking for
the marker text to fix.
This caused "devtool modify openssl" followed by "devtool update-recipe
openssl" (without any changes in-between) to remove version-script.patch
because that patch has no header and we weren't able to determine the
original filename.
(From OE-Core rev: d9971f5dc8eb7de551fd6f5e058fd24770ef5d78)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When PATCHTOOL = "git", if we need to manually apply a patch and then
commit it (i.e. when git am doesn't work) we try to extract the author /
date / shortlog from the patch header. Make the following improvements
to that extraction process:
* If there's no explicit Subject: but the first line is followed by a
blank line, isn't an Upstream-Status: or Index: marker and isn't too
long, then assume it's good enough to be the shortlog. This avoids
having too many patches with "Upgrade to version x.y" as the shortlog
(since that is often when patches get added).
* Add --follow to the command we use to find the commit that added the
patch, so we mostly get the commit that added the patch rather than
getting stuck on upgrade commits that last moved/renamed the patch
* Populate the date from the commit that added the patch if we were able
to get the author but not the date from the patch (otherwise you get
today's date which is less useful).
(From OE-Core rev: 896cfb10ec166a677cbb3b4f8643719cabeb7663)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If you leave "From <hash>" lines in the commit message it can actually
break git rebase because it tries to interpret the line in the context
of the current repository, and if the hash is invalid then a rebase
will blow up with:
fatal: git cat-file: could not get object info
or in newer git versions:
error: unable to find <hash>
fatal: git cat-file <hash>: bad file
(I hit this when I tried to do a devtool upgrade on openssl to 1.0.2i
the first time I did "git rebase --skip")
(From OE-Core rev: 19a6b18ac23cb2d7bb89203f774b2bee7f0cb03c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When extracting patches from a git repository with PATCHTOOL = "git" we
cannot assume that all patches will be UTF-8 formatted, so as with other
places in this module, try latin-1 if utf-8 fails.
This fixes UnicodeDecodeError running devtool update-recipe or devtool
finish on the openssl recipe.
(From OE-Core rev: 579e4d54a212d04cfece2c9fc0635d7ac1644058)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With Python 3, the encoding of a file is significant; several recipes in
OE-Core have patches which are not fully utf-8 decodable e.g. man,
lrzsz, and gstreamer1.0-libav, leading to errors when using devtool's
modify, upgrade or extract subcommands on these recipes. To work around
this, try reading the patch file as utf-8 first and if that fails try
latin-1 before giving up.
(From OE-Core rev: 7f4d7a6f51569954e204f110827a8ce256bcdc68)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
| |
The syntax for octal values changed in python3, adapt to it.
(From OE-Core rev: 737a095fcde773a36e0fee1f27b74aaa88062386)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
[YOCTO #9344]
(From OE-Core rev: 574405a97f956278d31d52cfc934be2840cf2fa6)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Show the actual command that failed when raising a CmdError. Makes
figuring out what actually failed much easier.
[YOCTO #9344]
(From OE-Core rev: 8e9c03df1810daab7171733f1713ef94d3a18ab2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rather than modifying files in .git/hooks, which can be read-only
(e.g., if it is a link to a directory in /usr/share), move away the
entire .git/hooks directory temporarily.
(From OE-Core rev: a88d603b51a9ebb39210d54b667519acfbe465c3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For patches that we have to extract the header information by hand (i.e.
will not apply with "git am"), make the following improvements:
* If we can't extract author/date/subject, then try to do so from the
commit that added the patch in git (assuming the metadata is tracked
by git)
* Take only first Signed-off-by line instead of last
* Accept any case for "Signed-off-by" in case author has typed it by
hand
* Improve conditional - we can skip the other cases if one matches
Implements [YOCTO #7624].
(From OE-Core rev: 13ec296b5c35aefa2c44f64f8bd1ef54c4a0a731)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
|
|
|
|
|
| |
Makes it possible to define which paths are included in the patches.
(From OE-Core rev: 640e57b423e5a8f0e4572eac985f87139780f247)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|