summaryrefslogtreecommitdiffstats
path: root/meta-python
diff options
context:
space:
mode:
authorJiaying Song <jiaying.song.cn@windriver.com>2024-12-02 16:49:23 +0800
committerArmin Kuster <akuster808@gmail.com>2024-12-08 15:04:29 -0500
commitc5c647ba6acdf14644890d6e4e2b7c7705624143 (patch)
tree007d7260a9c2333db8568d897e82e5ef3c7f6c20 /meta-python
parentf17b6e36fc2eef7f24d08885c50732d62f2754e5 (diff)
downloadmeta-openembedded-c5c647ba6acdf14644890d6e4e2b7c7705624143.tar.gz
python3-aiohttp: fix CVE-2023-49081/CVE-2024-30251/CVE-2024-52304/CVE-2023-49082/CVE-2024-27306
CVE-2023-49081: aiohttp is an asynchronous HTTP client/server framework for asyncio and Python. Improper validation made it possible for an attacker to modify the HTTP request (e.g. to insert a new header) or create a new HTTP request if the attacker controls the HTTP version. The vulnerability only occurs if the attacker can control the HTTP version of the request. This issue has been patched in version 3.9.0. References: https://nvd.nist.gov/vuln/detail/CVE-2023-49081 Upstream patches: https://github.com/aio-libs/aiohttp/commit/1e86b777e61cf4eefc7d92fa57fa19dcc676013b CVE-2024-30251: aiohttp is an asynchronous HTTP client/server framework for asyncio and Python. In affected versions an attacker can send a specially crafted POST (multipart/form-data) request. When the aiohttp server processes it, the server will enter an infinite loop and be unable to process any further requests. An attacker can stop the application from serving requests after sending a single request. This issue has been addressed in version 3.9.4. Users are advised to upgrade. Users unable to upgrade may manually apply a patch to their systems. Please see the linked GHSA for instructions. References: https://nvd.nist.gov/vuln/detail/CVE-2024-30251 Upstream patches: https://github.com/aio-libs/aiohttp/commit/cebe526b9c34dc3a3da9140409db63014bc4cf19 https://github.com/aio-libs/aiohttp/commit/7eecdff163ccf029fbb1ddc9de4169d4aaeb6597 https://github.com/aio-libs/aiohttp/commit/f21c6f2ca512a026ce7f0f6c6311f62d6a638866 CVE-2024-52304: aiohttp is an asynchronous HTTP client/server framework for asyncio and Python. Prior to version 3.10.11, the Python parser parses newlines in chunk extensions incorrectly which can lead to request smuggling vulnerabilities under certain conditions. If a pure Python version of aiohttp is installed (i.e. without the usual C extensions) or `AIOHTTP_NO_EXTENSIONS` is enabled, then an attacker may be able to execute a request smuggling attack to bypass certain firewalls or proxy protections. Version 3.10.11 fixes the issue. References: https://nvd.nist.gov/vuln/detail/CVE-2024-52304 Upstream patches: https://github.com/aio-libs/aiohttp/commit/259edc369075de63e6f3a4eaade058c62af0df71 CVE-2023-49082: aiohttp is an asynchronous HTTP client/server framework for asyncio and Python. Improper validation makes it possible for an attacker to modify the HTTP request (e.g. insert a new header) or even create a new HTTP request if the attacker controls the HTTP method. The vulnerability occurs only if the attacker can control the HTTP method (GET, POST etc.) of the request. If the attacker can control the HTTP version of the request it will be able to modify the request (request smuggling). This issue has been patched in version 3.9.0. References: https://nvd.nist.gov/vuln/detail/CVE-2023-49082 Upstream patches: https://github.com/aio-libs/aiohttp/pull/7806/commits/a43bc1779892e7014b7723c59d08fb37a000955e CVE-2024-27306: aiohttp is an asynchronous HTTP client/server framework for asyncio and Python. A XSS vulnerability exists on index pages for static file handling. This vulnerability is fixed in 3.9.4. We have always recommended using a reverse proxy server (e.g. nginx) for serving static files. Users following the recommendation are unaffected. Other users can disable `show_index` if unable to upgrade. References: https://nvd.nist.gov/vuln/detail/CVE-2024-27306 Upstream patches: https://github.com/aio-libs/aiohttp/commit/28335525d1eac015a7e7584137678cbb6ff19397 Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
Diffstat (limited to 'meta-python')
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49081.patch96
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49082.patch105
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-27306.patch81
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-30251.patch522
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-52304.patch46
-rw-r--r--meta-python/recipes-devtools/python/python3-aiohttp_3.8.6.bb5
6 files changed, 855 insertions, 0 deletions
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49081.patch b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49081.patch
new file mode 100644
index 0000000000..503b001445
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49081.patch
@@ -0,0 +1,96 @@
1From 67bf97cd1dfa513c8b6374905ee225b4d46cdf20 Mon Sep 17 00:00:00 2001
2From: Sam Bull <git@sambull.org>
3Date: Mon, 13 Nov 2023 22:13:06 +0000
4Subject: [PATCH] Disallow arbitrary sequence types in version (#7835)
5
6Upstream-Status: Backport
7[https://github.com/aio-libs/aiohttp/commit/1e86b777e61cf4eefc7d92fa57fa19dcc676013b]
8
9CVE: CVE-2023-49081
10
11Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
12Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
13---
14 CHANGES/7835.bugfix | 1 +
15 aiohttp/client_reqrep.py | 4 ++--
16 tests/test_client_request.py | 18 +++++++++++++++---
17 3 files changed, 18 insertions(+), 5 deletions(-)
18 create mode 100644 CHANGES/7835.bugfix
19
20diff --git a/CHANGES/7835.bugfix b/CHANGES/7835.bugfix
21new file mode 100644
22index 0000000..4ce3af4
23--- /dev/null
24+++ b/CHANGES/7835.bugfix
25@@ -0,0 +1 @@
26+Fixed arbitrary sequence types being allowed to inject headers via version parameter -- by :user:`Dreamsorcerer`
27diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py
28index 987d68f..d3cd77e 100644
29--- a/aiohttp/client_reqrep.py
30+++ b/aiohttp/client_reqrep.py
31@@ -661,8 +661,8 @@ class ClientRequest:
32 self.headers[hdrs.CONNECTION] = connection
33
34 # status + headers
35- status_line = "{0} {1} HTTP/{2[0]}.{2[1]}".format(
36- self.method, path, self.version
37+ status_line = "{0} {1} HTTP/{v.major}.{v.minor}".format(
38+ self.method, path, v=self.version
39 )
40 await writer.write_headers(status_line, self.headers)
41
42diff --git a/tests/test_client_request.py b/tests/test_client_request.py
43index 9eeb933..009f1a0 100644
44--- a/tests/test_client_request.py
45+++ b/tests/test_client_request.py
46@@ -20,6 +20,7 @@ from aiohttp.client_reqrep import (
47 _merge_ssl_params,
48 )
49 from aiohttp.helpers import PY_311
50+from aiohttp.http import HttpVersion
51 from aiohttp.test_utils import make_mocked_coro
52
53
54@@ -576,18 +577,18 @@ async def test_connection_header(loop, conn) -> None:
55 req.headers.clear()
56
57 req.keep_alive.return_value = True
58- req.version = (1, 1)
59+ req.version = HttpVersion(1, 1)
60 req.headers.clear()
61 await req.send(conn)
62 assert req.headers.get("CONNECTION") is None
63
64- req.version = (1, 0)
65+ req.version = HttpVersion(1, 0)
66 req.headers.clear()
67 await req.send(conn)
68 assert req.headers.get("CONNECTION") == "keep-alive"
69
70 req.keep_alive.return_value = False
71- req.version = (1, 1)
72+ req.version = HttpVersion(1, 1)
73 req.headers.clear()
74 await req.send(conn)
75 assert req.headers.get("CONNECTION") == "close"
76@@ -1113,6 +1114,17 @@ async def test_close(loop, buf, conn) -> None:
77 await req.close()
78 resp.close()
79
80+async def test_bad_version(loop: Any, conn: Any) -> None:
81+ req = ClientRequest(
82+ "GET",
83+ URL("http://python.org"),
84+ loop=loop,
85+ headers={"Connection": "Close"},
86+ version=("1", "1\r\nInjected-Header: not allowed"),
87+ )
88+
89+ with pytest.raises(AttributeError):
90+ await req.send(conn)
91
92 async def test_custom_response_class(loop, conn) -> None:
93 class CustomResponse(ClientResponse):
94--
952.25.1
96
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49082.patch b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49082.patch
new file mode 100644
index 0000000000..cfcb980317
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2023-49082.patch
@@ -0,0 +1,105 @@
1From a2200dc43d9fe0ee19b9185b30749c204a4dfd45 Mon Sep 17 00:00:00 2001
2From: Sam Bull <git@sambull.org>
3Date: Wed, 8 Nov 2023 19:25:05 +0000
4Subject: [PATCH] Add HTTP method validation (#6533) (#7806)
5
6(cherry picked from commit 75fca0b00b4297d0a30c51ae97a65428336eb2c1)
7
8Upstream-Status: Backport
9[https://github.com/aio-libs/aiohttp/pull/7806/commits/a43bc1779892e7014b7723c59d08fb37a000955e]
10
11CVE: CVE-2023-49082
12
13Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
14Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
15---
16 CHANGES/6533.feature | 1 +
17 aiohttp/client_reqrep.py | 9 ++++++++-
18 tests/test_client_request.py | 5 +++++
19 tests/test_web_request.py | 9 +++++++--
20 4 files changed, 21 insertions(+), 3 deletions(-)
21 create mode 100644 CHANGES/6533.feature
22
23diff --git a/CHANGES/6533.feature b/CHANGES/6533.feature
24new file mode 100644
25index 0000000..36bcbeb
26--- /dev/null
27+++ b/CHANGES/6533.feature
28@@ -0,0 +1 @@
29+Add HTTP method validation.
30diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py
31index d3cd77e..a8135b2 100644
32--- a/aiohttp/client_reqrep.py
33+++ b/aiohttp/client_reqrep.py
34@@ -78,6 +78,7 @@ if TYPE_CHECKING: # pragma: no cover
35 from .tracing import Trace
36
37
38+_CONTAINS_CONTROL_CHAR_RE = re.compile(r"[^-!#$%&'*+.^_`|~0-9a-zA-Z]")
39 json_re = re.compile(r"^application/(?:[\w.+-]+?\+)?json")
40
41
42@@ -266,10 +267,16 @@ class ClientRequest:
43 proxy_headers: Optional[LooseHeaders] = None,
44 traces: Optional[List["Trace"]] = None,
45 ):
46-
47 if loop is None:
48 loop = asyncio.get_event_loop()
49
50+ match = _CONTAINS_CONTROL_CHAR_RE.search(method)
51+ if match:
52+ raise ValueError(
53+ f"Method cannot contain non-token characters {method!r} "
54+ "(found at least {match.group()!r})"
55+ )
56+
57 assert isinstance(url, URL), url
58 assert isinstance(proxy, (URL, type(None))), proxy
59 # FIXME: session is None in tests only, need to fix tests
60diff --git a/tests/test_client_request.py b/tests/test_client_request.py
61index 009f1a0..d0f208b 100644
62--- a/tests/test_client_request.py
63+++ b/tests/test_client_request.py
64@@ -89,6 +89,11 @@ def test_method3(make_request) -> None:
65 assert req.method == "HEAD"
66
67
68+def test_method_invalid(make_request) -> None:
69+ with pytest.raises(ValueError, match="Method cannot contain non-token characters"):
70+ make_request("METHOD WITH\nWHITESPACES", "http://python.org/")
71+
72+
73 def test_version_1_0(make_request) -> None:
74 req = make_request("get", "http://python.org/", version="1.0")
75 assert req.version == (1, 0)
76diff --git a/tests/test_web_request.py b/tests/test_web_request.py
77index c6aeaf8..2bb0cd5 100644
78--- a/tests/test_web_request.py
79+++ b/tests/test_web_request.py
80@@ -43,7 +43,10 @@ def test_base_ctor() -> None:
81
82 assert "GET" == req.method
83 assert HttpVersion(1, 1) == req.version
84- assert req.host == socket.getfqdn()
85+ # MacOS may return CamelCased host name, need .lower()
86+ # FQDN can be wider than host, e.g.
87+ # 'fv-az397-495' in 'fv-az397-495.internal.cloudapp.net'
88+ assert req.host.lower() in socket.getfqdn().lower()
89 assert "/path/to?a=1&b=2" == req.path_qs
90 assert "/path/to" == req.path
91 assert "a=1&b=2" == req.query_string
92@@ -66,7 +69,9 @@ def test_ctor() -> None:
93 assert "GET" == req.method
94 assert HttpVersion(1, 1) == req.version
95 # MacOS may return CamelCased host name, need .lower()
96- assert req.host.lower() == socket.getfqdn().lower()
97+ # FQDN can be wider than host, e.g.
98+ # 'fv-az397-495' in 'fv-az397-495.internal.cloudapp.net'
99+ assert req.host.lower() in socket.getfqdn().lower()
100 assert "/path/to?a=1&b=2" == req.path_qs
101 assert "/path/to" == req.path
102 assert "a=1&b=2" == req.query_string
103--
1042.25.1
105
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-27306.patch b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-27306.patch
new file mode 100644
index 0000000000..f87ef92679
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-27306.patch
@@ -0,0 +1,81 @@
1From d05042f1a35ec0adb797c056024d457ac1fd7088 Mon Sep 17 00:00:00 2001
2From: Sam Bull <git@sambull.org>
3Date: Thu, 11 Apr 2024 15:54:45 +0100
4Subject: [PATCH] Escape filenames and paths in HTML when generating index
5 pages (#8317) (#8319)
6
7Upstream-Status: Backport
8[https://github.com/aio-libs/aiohttp/commit/28335525d1eac015a7e7584137678cbb6ff19397]
9
10CVE: CVE-2024-27306
11
12Co-authored-by: J. Nick Koston <nick@koston.org>
13(cherry picked from commit ffbc43233209df302863712b511a11bdb6001b0f)
14Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
15---
16 CHANGES/8317.bugfix.rst | 1 +
17 aiohttp/web_urldispatcher.py | 11 ++++++-----
18 2 files changed, 7 insertions(+), 5 deletions(-)
19 create mode 100644 CHANGES/8317.bugfix.rst
20
21diff --git a/CHANGES/8317.bugfix.rst b/CHANGES/8317.bugfix.rst
22new file mode 100644
23index 0000000..b24ef2a
24--- /dev/null
25+++ b/CHANGES/8317.bugfix.rst
26@@ -0,0 +1 @@
27+Escaped filenames in static view -- by :user:`bdraco`.
28diff --git a/aiohttp/web_urldispatcher.py b/aiohttp/web_urldispatcher.py
29index e8a8023..791ab94 100644
30--- a/aiohttp/web_urldispatcher.py
31+++ b/aiohttp/web_urldispatcher.py
32@@ -1,7 +1,9 @@
33 import abc
34 import asyncio
35 import base64
36+import functools
37 import hashlib
38+import html
39 import inspect
40 import keyword
41 import os
42@@ -87,6 +89,7 @@ PATH_SEP: Final[str] = re.escape("/")
43 _ExpectHandler = Callable[[Request], Awaitable[None]]
44 _Resolve = Tuple[Optional["UrlMappingMatchInfo"], Set[str]]
45
46+html_escape = functools.partial(html.escape, quote=True)
47
48 class _InfoDict(TypedDict, total=False):
49 path: str
50@@ -706,7 +709,7 @@ class StaticResource(PrefixResource):
51 assert filepath.is_dir()
52
53 relative_path_to_dir = filepath.relative_to(self._directory).as_posix()
54- index_of = f"Index of /{relative_path_to_dir}"
55+ index_of = f"Index of /{html_escape(relative_path_to_dir)}"
56 h1 = f"<h1>{index_of}</h1>"
57
58 index_list = []
59@@ -714,7 +717,7 @@ class StaticResource(PrefixResource):
60 for _file in sorted(dir_index):
61 # show file url as relative to static path
62 rel_path = _file.relative_to(self._directory).as_posix()
63- file_url = self._prefix + "/" + rel_path
64+ quoted_file_url = _quote_path(f"{self._prefix}/{rel_path}")
65
66 # if file is a directory, add '/' to the end of the name
67 if _file.is_dir():
68@@ -723,9 +726,7 @@ class StaticResource(PrefixResource):
69 file_name = _file.name
70
71 index_list.append(
72- '<li><a href="{url}">{name}</a></li>'.format(
73- url=file_url, name=file_name
74- )
75+ f'<li><a href="{quoted_file_url}">{html_escape(file_name)}</a></li>'
76 )
77 ul = "<ul>\n{}\n</ul>".format("\n".join(index_list))
78 body = f"<body>\n{h1}\n{ul}\n</body>"
79--
802.25.1
81
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-30251.patch b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-30251.patch
new file mode 100644
index 0000000000..20226eb754
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-30251.patch
@@ -0,0 +1,522 @@
1From 44108afc0c0460d154216cab9aaa1d8f57edc3cc Mon Sep 17 00:00:00 2001
2From: Sam Bull <git@sambull.org>
3Date: Sun, 7 Apr 2024 13:19:31 +0100
4Subject: [PATCH] Fix handling of multipart/form-data (#8280) (#8302)
5
6https://datatracker.ietf.org/doc/html/rfc7578
7(cherry picked from commit 7d0be3fee540a3d4161ac7dc76422f1f5ea60104)
8
9The following commits are also included:
107eecdff1 [PR #8332/482e6cdf backport][3.9] Add set_content_disposition test (#8333)
11f21c6f2c [PR #8335/5a6949da backport][3.9] Add Content-Disposition automatically (#8336)
12
13Upstream-Status: Backport
14[https://github.com/aio-libs/aiohttp/commit/cebe526b9c34dc3a3da9140409db63014bc4cf19
15https://github.com/aio-libs/aiohttp/commit/7eecdff163ccf029fbb1ddc9de4169d4aaeb6597
16https://github.com/aio-libs/aiohttp/commit/f21c6f2ca512a026ce7f0f6c6311f62d6a638866]
17
18CVE: CVE-2024-30251
19
20Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
21---
22 CHANGES/8280.bugfix.rst | 1 +
23 CHANGES/8280.deprecation.rst | 2 +
24 CHANGES/8332.bugfix.rst | 1 +
25 CHANGES/8335.bugfix.rst | 1 +
26 aiohttp/formdata.py | 12 +++-
27 aiohttp/multipart.py | 128 ++++++++++++++++++++++++-----------
28 tests/test_multipart.py | 87 ++++++++++++++++++++----
29 tests/test_web_functional.py | 27 ++------
30 8 files changed, 182 insertions(+), 77 deletions(-)
31 create mode 100644 CHANGES/8280.bugfix.rst
32 create mode 100644 CHANGES/8280.deprecation.rst
33 create mode 100644 CHANGES/8332.bugfix.rst
34 create mode 100644 CHANGES/8335.bugfix.rst
35
36diff --git a/CHANGES/8280.bugfix.rst b/CHANGES/8280.bugfix.rst
37new file mode 100644
38index 0000000..3aebe36
39--- /dev/null
40+++ b/CHANGES/8280.bugfix.rst
41@@ -0,0 +1 @@
42+Fixed ``multipart/form-data`` compliance with :rfc:`7578` -- by :user:`Dreamsorcerer`.
43diff --git a/CHANGES/8280.deprecation.rst b/CHANGES/8280.deprecation.rst
44new file mode 100644
45index 0000000..302dbb2
46--- /dev/null
47+++ b/CHANGES/8280.deprecation.rst
48@@ -0,0 +1,2 @@
49+Deprecated ``content_transfer_encoding`` parameter in :py:meth:`FormData.add_field()
50+<aiohttp.FormData.add_field>` -- by :user:`Dreamsorcerer`.
51diff --git a/CHANGES/8332.bugfix.rst b/CHANGES/8332.bugfix.rst
52new file mode 100644
53index 0000000..70cad26
54--- /dev/null
55+++ b/CHANGES/8332.bugfix.rst
56@@ -0,0 +1 @@
57+Fixed regression with adding Content-Disposition to form-data part after appending to writer -- by :user:`Dreamsorcerer`/:user:`Olegt0rr`.
58diff --git a/CHANGES/8335.bugfix.rst b/CHANGES/8335.bugfix.rst
59new file mode 100644
60index 0000000..cd93b86
61--- /dev/null
62+++ b/CHANGES/8335.bugfix.rst
63@@ -0,0 +1 @@
64+Added default Content-Disposition in multipart/form-data responses -- by :user:`Dreamsorcerer`.
65diff --git a/aiohttp/formdata.py b/aiohttp/formdata.py
66index e7cd24c..2b75b3d 100644
67--- a/aiohttp/formdata.py
68+++ b/aiohttp/formdata.py
69@@ -1,4 +1,5 @@
70 import io
71+import warnings
72 from typing import Any, Iterable, List, Optional
73 from urllib.parse import urlencode
74
75@@ -53,7 +54,12 @@ class FormData:
76 if isinstance(value, io.IOBase):
77 self._is_multipart = True
78 elif isinstance(value, (bytes, bytearray, memoryview)):
79+ msg = (
80+ "In v4, passing bytes will no longer create a file field. "
81+ "Please explicitly use the filename parameter or pass a BytesIO object."
82+ )
83 if filename is None and content_transfer_encoding is None:
84+ warnings.warn(msg, DeprecationWarning)
85 filename = name
86
87 type_options: MultiDict[str] = MultiDict({"name": name})
88@@ -81,7 +87,11 @@ class FormData:
89 "content_transfer_encoding must be an instance"
90 " of str. Got: %s" % content_transfer_encoding
91 )
92- headers[hdrs.CONTENT_TRANSFER_ENCODING] = content_transfer_encoding
93+ msg = (
94+ "content_transfer_encoding is deprecated. "
95+ "To maintain compatibility with v4 please pass a BytesPayload."
96+ )
97+ warnings.warn(msg, DeprecationWarning)
98 self._is_multipart = True
99
100 self._fields.append((type_options, headers, value))
101diff --git a/aiohttp/multipart.py b/aiohttp/multipart.py
102index 73801f4..9cd49bb 100644
103--- a/aiohttp/multipart.py
104+++ b/aiohttp/multipart.py
105@@ -255,13 +255,22 @@ class BodyPartReader:
106 chunk_size = 8192
107
108 def __init__(
109- self, boundary: bytes, headers: "CIMultiDictProxy[str]", content: StreamReader
110+ self,
111+ boundary: bytes,
112+ headers: "CIMultiDictProxy[str]",
113+ content: StreamReader,
114+ *,
115+ subtype: str = "mixed",
116+ default_charset: Optional[str] = None,
117 ) -> None:
118 self.headers = headers
119 self._boundary = boundary
120 self._content = content
121+ self._default_charset = default_charset
122 self._at_eof = False
123- length = self.headers.get(CONTENT_LENGTH, None)
124+ self._is_form_data = subtype == "form-data"
125+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.8
126+ length = None if self._is_form_data else self.headers.get(CONTENT_LENGTH, None)
127 self._length = int(length) if length is not None else None
128 self._read_bytes = 0
129 # TODO: typeing.Deque is not supported by Python 3.5
130@@ -329,6 +338,8 @@ class BodyPartReader:
131 assert self._length is not None, "Content-Length required for chunked read"
132 chunk_size = min(size, self._length - self._read_bytes)
133 chunk = await self._content.read(chunk_size)
134+ if self._content.at_eof():
135+ self._at_eof = True
136 return chunk
137
138 async def _read_chunk_from_stream(self, size: int) -> bytes:
139@@ -444,7 +455,8 @@ class BodyPartReader:
140 """
141 if CONTENT_TRANSFER_ENCODING in self.headers:
142 data = self._decode_content_transfer(data)
143- if CONTENT_ENCODING in self.headers:
144+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.8
145+ if not self._is_form_data and CONTENT_ENCODING in self.headers:
146 return self._decode_content(data)
147 return data
148
149@@ -478,7 +490,7 @@ class BodyPartReader:
150 """Returns charset parameter from Content-Type header or default."""
151 ctype = self.headers.get(CONTENT_TYPE, "")
152 mimetype = parse_mimetype(ctype)
153- return mimetype.parameters.get("charset", default)
154+ return mimetype.parameters.get("charset", self._default_charset or default)
155
156 @reify
157 def name(self) -> Optional[str]:
158@@ -533,9 +545,17 @@ class MultipartReader:
159 part_reader_cls = BodyPartReader
160
161 def __init__(self, headers: Mapping[str, str], content: StreamReader) -> None:
162+ self._mimetype = parse_mimetype(headers[CONTENT_TYPE])
163+ assert self._mimetype.type == "multipart", "multipart/* content type expected"
164+ if "boundary" not in self._mimetype.parameters:
165+ raise ValueError(
166+ "boundary missed for Content-Type: %s" % headers[CONTENT_TYPE]
167+ )
168+
169 self.headers = headers
170 self._boundary = ("--" + self._get_boundary()).encode()
171 self._content = content
172+ self._default_charset: Optional[str] = None
173 self._last_part: Optional[Union["MultipartReader", BodyPartReader]] = None
174 self._at_eof = False
175 self._at_bof = True
176@@ -587,7 +607,24 @@ class MultipartReader:
177 await self._read_boundary()
178 if self._at_eof: # we just read the last boundary, nothing to do there
179 return None
180- self._last_part = await self.fetch_next_part()
181+
182+ part = await self.fetch_next_part()
183+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.6
184+ if (
185+ self._last_part is None
186+ and self._mimetype.subtype == "form-data"
187+ and isinstance(part, BodyPartReader)
188+ ):
189+ _, params = parse_content_disposition(part.headers.get(CONTENT_DISPOSITION))
190+ if params.get("name") == "_charset_":
191+ # Longest encoding in https://encoding.spec.whatwg.org/encodings.json
192+ # is 19 characters, so 32 should be more than enough for any valid encoding.
193+ charset = await part.read_chunk(32)
194+ if len(charset) > 31:
195+ raise RuntimeError("Invalid default charset")
196+ self._default_charset = charset.strip().decode()
197+ part = await self.fetch_next_part()
198+ self._last_part = part
199 return self._last_part
200
201 async def release(self) -> None:
202@@ -623,19 +660,16 @@ class MultipartReader:
203 return type(self)(headers, self._content)
204 return self.multipart_reader_cls(headers, self._content)
205 else:
206- return self.part_reader_cls(self._boundary, headers, self._content)
207-
208- def _get_boundary(self) -> str:
209- mimetype = parse_mimetype(self.headers[CONTENT_TYPE])
210-
211- assert mimetype.type == "multipart", "multipart/* content type expected"
212-
213- if "boundary" not in mimetype.parameters:
214- raise ValueError(
215- "boundary missed for Content-Type: %s" % self.headers[CONTENT_TYPE]
216+ return self.part_reader_cls(
217+ self._boundary,
218+ headers,
219+ self._content,
220+ subtype=self._mimetype.subtype,
221+ default_charset=self._default_charset,
222 )
223
224- boundary = mimetype.parameters["boundary"]
225+ def _get_boundary(self) -> str:
226+ boundary = self._mimetype.parameters["boundary"]
227 if len(boundary) > 70:
228 raise ValueError("boundary %r is too long (70 chars max)" % boundary)
229
230@@ -726,6 +760,7 @@ class MultipartWriter(Payload):
231 super().__init__(None, content_type=ctype)
232
233 self._parts: List[_Part] = []
234+ self._is_form_data = subtype == "form-data"
235
236 def __enter__(self) -> "MultipartWriter":
237 return self
238@@ -803,32 +838,38 @@ class MultipartWriter(Payload):
239
240 def append_payload(self, payload: Payload) -> Payload:
241 """Adds a new body part to multipart writer."""
242- # compression
243- encoding: Optional[str] = payload.headers.get(
244- CONTENT_ENCODING,
245- "",
246- ).lower()
247- if encoding and encoding not in ("deflate", "gzip", "identity"):
248- raise RuntimeError(f"unknown content encoding: {encoding}")
249- if encoding == "identity":
250- encoding = None
251-
252- # te encoding
253- te_encoding: Optional[str] = payload.headers.get(
254- CONTENT_TRANSFER_ENCODING,
255- "",
256- ).lower()
257- if te_encoding not in ("", "base64", "quoted-printable", "binary"):
258- raise RuntimeError(
259- "unknown content transfer encoding: {}" "".format(te_encoding)
260+ encoding: Optional[str] = None
261+ te_encoding: Optional[str] = None
262+ if self._is_form_data:
263+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.7
264+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.8
265+ assert (
266+ not {CONTENT_ENCODING, CONTENT_LENGTH, CONTENT_TRANSFER_ENCODING}
267+ & payload.headers.keys()
268 )
269- if te_encoding == "binary":
270- te_encoding = None
271-
272- # size
273- size = payload.size
274- if size is not None and not (encoding or te_encoding):
275- payload.headers[CONTENT_LENGTH] = str(size)
276+ # Set default Content-Disposition in case user doesn't create one
277+ if CONTENT_DISPOSITION not in payload.headers:
278+ name = f"section-{len(self._parts)}"
279+ payload.set_content_disposition("form-data", name=name)
280+ else:
281+ # compression
282+ encoding = payload.headers.get(CONTENT_ENCODING, "").lower()
283+ if encoding and encoding not in ("deflate", "gzip", "identity"):
284+ raise RuntimeError(f"unknown content encoding: {encoding}")
285+ if encoding == "identity":
286+ encoding = None
287+
288+ # te encoding
289+ te_encoding = payload.headers.get(CONTENT_TRANSFER_ENCODING, "").lower()
290+ if te_encoding not in ("", "base64", "quoted-printable", "binary"):
291+ raise RuntimeError(f"unknown content transfer encoding: {te_encoding}")
292+ if te_encoding == "binary":
293+ te_encoding = None
294+
295+ # size
296+ size = payload.size
297+ if size is not None and not (encoding or te_encoding):
298+ payload.headers[CONTENT_LENGTH] = str(size)
299
300 self._parts.append((payload, encoding, te_encoding)) # type: ignore[arg-type]
301 return payload
302@@ -886,6 +927,11 @@ class MultipartWriter(Payload):
303 async def write(self, writer: Any, close_boundary: bool = True) -> None:
304 """Write body."""
305 for part, encoding, te_encoding in self._parts:
306+ if self._is_form_data:
307+ # https://datatracker.ietf.org/doc/html/rfc7578#section-4.2
308+ assert CONTENT_DISPOSITION in part.headers
309+ assert "name=" in part.headers[CONTENT_DISPOSITION]
310+
311 await writer.write(b"--" + self._boundary + b"\r\n")
312 await writer.write(part._binary_headers)
313
314diff --git a/tests/test_multipart.py b/tests/test_multipart.py
315index cc3f5ff..1d036fb 100644
316--- a/tests/test_multipart.py
317+++ b/tests/test_multipart.py
318@@ -942,6 +942,58 @@ class TestMultipartReader:
319 assert first.at_eof()
320 assert not second.at_eof()
321
322+ async def test_read_form_default_encoding(self) -> None:
323+ with Stream(
324+ b"--:\r\n"
325+ b'Content-Disposition: form-data; name="_charset_"\r\n\r\n'
326+ b"ascii"
327+ b"\r\n"
328+ b"--:\r\n"
329+ b'Content-Disposition: form-data; name="field1"\r\n\r\n'
330+ b"foo"
331+ b"\r\n"
332+ b"--:\r\n"
333+ b"Content-Type: text/plain;charset=UTF-8\r\n"
334+ b'Content-Disposition: form-data; name="field2"\r\n\r\n'
335+ b"foo"
336+ b"\r\n"
337+ b"--:\r\n"
338+ b'Content-Disposition: form-data; name="field3"\r\n\r\n'
339+ b"foo"
340+ b"\r\n"
341+ ) as stream:
342+ reader = aiohttp.MultipartReader(
343+ {CONTENT_TYPE: 'multipart/form-data;boundary=":"'},
344+ stream,
345+ )
346+ field1 = await reader.next()
347+ assert field1.name == "field1"
348+ assert field1.get_charset("default") == "ascii"
349+ field2 = await reader.next()
350+ assert field2.name == "field2"
351+ assert field2.get_charset("default") == "UTF-8"
352+ field3 = await reader.next()
353+ assert field3.name == "field3"
354+ assert field3.get_charset("default") == "ascii"
355+
356+ async def test_read_form_invalid_default_encoding(self) -> None:
357+ with Stream(
358+ b"--:\r\n"
359+ b'Content-Disposition: form-data; name="_charset_"\r\n\r\n'
360+ b"this-value-is-too-long-to-be-a-charset"
361+ b"\r\n"
362+ b"--:\r\n"
363+ b'Content-Disposition: form-data; name="field1"\r\n\r\n'
364+ b"foo"
365+ b"\r\n"
366+ ) as stream:
367+ reader = aiohttp.MultipartReader(
368+ {CONTENT_TYPE: 'multipart/form-data;boundary=":"'},
369+ stream,
370+ )
371+ with pytest.raises(RuntimeError, match="Invalid default charset"):
372+ await reader.next()
373+
374
375 async def test_writer(writer) -> None:
376 assert writer.size == 7
377@@ -1228,6 +1280,25 @@ class TestMultipartWriter:
378 part = writer._parts[0][0]
379 assert part.headers[CONTENT_TYPE] == "test/passed"
380
381+ def test_set_content_disposition_after_append(self):
382+ writer = aiohttp.MultipartWriter("form-data")
383+ part = writer.append("some-data")
384+ part.set_content_disposition("form-data", name="method")
385+ assert 'name="method"' in part.headers[CONTENT_DISPOSITION]
386+
387+ def test_automatic_content_disposition(self):
388+ writer = aiohttp.MultipartWriter("form-data")
389+ writer.append_json(())
390+ part = payload.StringPayload("foo")
391+ part.set_content_disposition("form-data", name="second")
392+ writer.append_payload(part)
393+ writer.append("foo")
394+
395+ disps = tuple(p[0].headers[CONTENT_DISPOSITION] for p in writer._parts)
396+ assert 'name="section-0"' in disps[0]
397+ assert 'name="second"' in disps[1]
398+ assert 'name="section-2"' in disps[2]
399+
400 def test_with(self) -> None:
401 with aiohttp.MultipartWriter(boundary=":") as writer:
402 writer.append("foo")
403@@ -1278,7 +1349,6 @@ class TestMultipartWriter:
404 CONTENT_TYPE: "text/python",
405 },
406 )
407- content_length = part.size
408 await writer.write(stream)
409
410 assert part.headers[CONTENT_TYPE] == "text/python"
411@@ -1289,9 +1359,7 @@ class TestMultipartWriter:
412 assert headers == (
413 b"--:\r\n"
414 b"Content-Type: text/python\r\n"
415- b'Content-Disposition: attachments; filename="bug.py"\r\n'
416- b"Content-Length: %s"
417- b"" % (str(content_length).encode(),)
418+ b'Content-Disposition: attachments; filename="bug.py"'
419 )
420
421 async def test_set_content_disposition_override(self, buf, stream):
422@@ -1305,7 +1373,6 @@ class TestMultipartWriter:
423 CONTENT_TYPE: "text/python",
424 },
425 )
426- content_length = part.size
427 await writer.write(stream)
428
429 assert part.headers[CONTENT_TYPE] == "text/python"
430@@ -1316,9 +1383,7 @@ class TestMultipartWriter:
431 assert headers == (
432 b"--:\r\n"
433 b"Content-Type: text/python\r\n"
434- b'Content-Disposition: attachments; filename="bug.py"\r\n'
435- b"Content-Length: %s"
436- b"" % (str(content_length).encode(),)
437+ b'Content-Disposition: attachments; filename="bug.py"'
438 )
439
440 async def test_reset_content_disposition_header(self, buf, stream):
441@@ -1330,8 +1395,6 @@ class TestMultipartWriter:
442 headers={CONTENT_TYPE: "text/plain"},
443 )
444
445- content_length = part.size
446-
447 assert CONTENT_DISPOSITION in part.headers
448
449 part.set_content_disposition("attachments", filename="bug.py")
450@@ -1344,9 +1407,7 @@ class TestMultipartWriter:
451 b"--:\r\n"
452 b"Content-Type: text/plain\r\n"
453 b"Content-Disposition:"
454- b' attachments; filename="bug.py"\r\n'
455- b"Content-Length: %s"
456- b"" % (str(content_length).encode(),)
457+ b' attachments; filename="bug.py"'
458 )
459
460
461diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py
462index 5fdfb23..61739e9 100644
463--- a/tests/test_web_functional.py
464+++ b/tests/test_web_functional.py
465@@ -34,7 +34,8 @@ def fname(here):
466
467 def new_dummy_form():
468 form = FormData()
469- form.add_field("name", b"123", content_transfer_encoding="base64")
470+ with pytest.warns(DeprecationWarning, match="BytesPayload"):
471+ form.add_field("name", b"123", content_transfer_encoding="base64")
472 return form
473
474
475@@ -429,25 +430,6 @@ async def test_release_post_data(aiohttp_client) -> None:
476 await resp.release()
477
478
479-async def test_POST_DATA_with_content_transfer_encoding(aiohttp_client) -> None:
480- async def handler(request):
481- data = await request.post()
482- assert b"123" == data["name"]
483- return web.Response()
484-
485- app = web.Application()
486- app.router.add_post("/", handler)
487- client = await aiohttp_client(app)
488-
489- form = FormData()
490- form.add_field("name", b"123", content_transfer_encoding="base64")
491-
492- resp = await client.post("/", data=form)
493- assert 200 == resp.status
494-
495- await resp.release()
496-
497-
498 async def test_post_form_with_duplicate_keys(aiohttp_client) -> None:
499 async def handler(request):
500 data = await request.post()
501@@ -505,7 +487,8 @@ async def test_100_continue(aiohttp_client) -> None:
502 return web.Response()
503
504 form = FormData()
505- form.add_field("name", b"123", content_transfer_encoding="base64")
506+ with pytest.warns(DeprecationWarning, match="BytesPayload"):
507+ form.add_field("name", b"123", content_transfer_encoding="base64")
508
509 app = web.Application()
510 app.router.add_post("/", handler)
511@@ -683,7 +666,7 @@ async def test_upload_file(aiohttp_client) -> None:
512 app.router.add_post("/", handler)
513 client = await aiohttp_client(app)
514
515- resp = await client.post("/", data={"file": data})
516+ resp = await client.post("/", data={"file": io.BytesIO(data)})
517 assert 200 == resp.status
518
519 await resp.release()
520--
5212.25.1
522
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-52304.patch b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-52304.patch
new file mode 100644
index 0000000000..a76968c6ca
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-aiohttp/CVE-2024-52304.patch
@@ -0,0 +1,46 @@
1From 27b9925ad3ac716a6db3a3d1214b3fe2a260c5c8 Mon Sep 17 00:00:00 2001
2From: "J. Nick Koston" <nick@koston.org>
3Date: Wed, 13 Nov 2024 08:50:36 -0600
4Subject: [PATCH] Fix incorrect parsing of chunk extensions with the pure
5 Python parser (#9853)
6
7Upstream-Status: Backport
8[https://github.com/aio-libs/aiohttp/commit/259edc369075de63e6f3a4eaade058c62af0df71]
9
10CVE: CVE-2024-52304
11
12Signed-off-by: Jiaying Song <jiaying.song.cn@windriver.com>
13---
14 CHANGES/9851.bugfix.rst | 1 +
15 aiohttp/http_parser.py | 7 +++++++
16 2 files changed, 8 insertions(+)
17 create mode 100644 CHANGES/9851.bugfix.rst
18
19diff --git a/CHANGES/9851.bugfix.rst b/CHANGES/9851.bugfix.rst
20new file mode 100644
21index 0000000..02541a9
22--- /dev/null
23+++ b/CHANGES/9851.bugfix.rst
24@@ -0,0 +1 @@
25+Fixed incorrect parsing of chunk extensions with the pure Python parser -- by :user:`bdraco`.
26diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py
27index 91e01f4..1ee1269 100644
28--- a/aiohttp/http_parser.py
29+++ b/aiohttp/http_parser.py
30@@ -820,6 +820,13 @@ class HttpPayloadParser:
31 i = chunk.find(CHUNK_EXT, 0, pos)
32 if i >= 0:
33 size_b = chunk[:i] # strip chunk-extensions
34+ # Verify no LF in the chunk-extension
35+ if b"\n" in (ext := chunk[i:pos]):
36+ exc = BadHttpMessage(
37+ f"Unexpected LF in chunk-extension: {ext!r}"
38+ )
39+ set_exception(self.payload, exc)
40+ raise exc
41 else:
42 size_b = chunk[:pos]
43
44--
452.25.1
46
diff --git a/meta-python/recipes-devtools/python/python3-aiohttp_3.8.6.bb b/meta-python/recipes-devtools/python/python3-aiohttp_3.8.6.bb
index c805e17d86..479c2f2064 100644
--- a/meta-python/recipes-devtools/python/python3-aiohttp_3.8.6.bb
+++ b/meta-python/recipes-devtools/python/python3-aiohttp_3.8.6.bb
@@ -5,6 +5,11 @@ LICENSE = "Apache-2.0"
5LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=748073912af33aa59430d3702aa32d41" 5LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=748073912af33aa59430d3702aa32d41"
6 6
7SRC_URI += "file://CVE-2024-23334.patch \ 7SRC_URI += "file://CVE-2024-23334.patch \
8 file://CVE-2023-49081.patch \
9 file://CVE-2024-30251.patch \
10 file://CVE-2024-52304.patch \
11 file://CVE-2023-49082.patch \
12 file://CVE-2024-27306.patch \
8 " 13 "
9 14
10SRC_URI[sha256sum] = "b0cf2a4501bff9330a8a5248b4ce951851e415bdcce9dc158e76cfd55e15085c" 15SRC_URI[sha256sum] = "b0cf2a4501bff9330a8a5248b4ce951851e415bdcce9dc158e76cfd55e15085c"