CVE-2024-49767 - Werkzeug / Flask / Quart

Werkzeug is a Web Server Gateway Interface (WSGI) library used to develop python web applications or frameworks. Applications using werkzeug.formparser.MultiPartParser to parse multipart/form-data requests (e.g. all flask and quart applications) were vulnerable to resource exhaustion (denial of service) attacks. A specifically crafted form submission request could cause the parser to allocate and block 3 to 8 times the upload size in main memory. There was no upper limit; a single upload at 1 Gbit/s could exhaust 32 GB of RAM in less than 60 seconds.

• CVE-2024-49767
• GHSA-q34m-jh98-gwm2
• CVSSv4: 6.9 (CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:N/VI:N/VA:L/SC:N/SI:N/SA:N)

Original Report¶

Note: The final report published by the maintainers was heavily edited and lacked essential details, which is why I decided publish the original report here.

The multipart parser implementation found in werkzeug.formparser distinguishes between file uploads and text fields by looking for a filename option in the Content-Disposition multipart segment header. If present, the multipart segment is treated as file uploads and buffered into a SpooledTemporaryFile. If the filename option is missing, the segment is treated as a text field and buffered into an in-memory list instead. This list is allowed to grow indefinitely. Once complete, the buffered chunks are concatenated and decoded which causes a peak memory consumption of 3 to 8 times the original uncompressed upload size. With clever use of specific unicode glyphs this factor can probably be increased even further.

This allows an attacker to craft multipart/form-data requests that completely exhaust the servers main memory, causing excessive swapping or out-of-memory situations. A 32GB server can be exhausted over a 1 Gbit/s line in less than a minute, with just a single request. Transfer-Encoding: chunked can be used to bypass early safeguards that rely on a Content-Length header. Sending data slowly over several requests in parallel should avoid timeouts but still block huge amounts of memory for a long time before individual requests or the entire server process is eventually killed by the kernels OOM killer.

Other multipart libraries or web frameworks have configurable limits for in-memory (and on-disk) data structures and will either error out if header sections or text fields grow larger than allowed, or treat those large text fields as file uploads (with an empty filename) and buffer to disk as a safeguard. In werkzeug there is no limit for on-disk buffering and the max_form_memory_size limit is only enforced on the header section of a multipart segment, but ignored for the in-memory list used to buffer text fields. The max_content_length limit on the other hand is too strict, as it would also limit regular file uploads. Both limits are not set in Werkzeug, Flask or Quart by default, leaving applications vulnerable to this attack by default.

Proof of concept

import flask
app = flask.Flask(__name__)
@app.post('/')
def foo():
    flask.request.form
    return 'DONE'

from quart import Quart, request
app = Quart(__name__)

@app.route('/', methods=["POST"])
async def greet():
    await request.form
    return 'DONE'

app.run()

$ curl http://localhost:5000 -F 'big=</dev/urandom'
# WARNING: This will fill up memory and start swapping almost instantly. Use --limit-rate 1M for a slow death

Impact

Vulnerable are all applications that use werkzeug.formparser.MultiPartParser to parse multipart/form-data requests (e.g. all flask and quart applications) and expose an endpoint that processes form submissions. Those applications can be crashed or slowed down by exhausting system memory, possibly also affecting other services running on the same system (e.g. database servers).

Mitigation

Limit maximum upload size in the WSGI/ASGI server or proxy in front of the vulnerable application to a value that would fits into available memory twice per worker thread.

Timeline¶

03.09.2024 (+0 days) First contact. Sent a detailed report (similar to the one above) to the projects security contact email.

10.09.2024 (+7 days) No response. Contacted their security email again and asked for confirmation. Got a quick response this time. They asked to open a GitHub security advisory, which I did. Note that the published CVE and GHSA text is not the original report.

02.10.2024 (+29 days) No response for three weeks, so asked again for an estimate. The responsible maintainer suggested that I should provide a patch, mentioned two config setting which were already mentioned in the initial report as known to be ineffective or unsuitable, then continued to argue that insecure defaults or resource exhaustion should not count as a security issue in the first place. A bit of back and forth followed, mostly about the impact and severity of the issue. They then tried to reproduce the issue with max_content_length = 1000, a value so low that it would also reject most legitimate requests. This also triggered an unrelated logic bug and broke all form submissions, even smaller ones. The provided PoC, which uses default values for those settings and worked as expected, was ignored.

10.10.2024 (+37 days) Mentioned that Starlette had a very similar issue and already released a fix. The details are now public and easily transferable to werkzeug. No response from the maintainer.

25.10.2024 (+52 days) The same maintainer tried again to reproduce the issue, failed for the exact same reason that was already discussed 3 weeks ago, and closed the report without waiting for feedback. Explained (again) why a limit below 64k breaks all form submissions, not only malicious ones, and how to properly reproduce the issue. No response.

25.10.2024 (+52 days) Reached out on their the public developer discord and asked for a second opinion, of cause without disclosing any details. A couple of hours later, the same maintainer deleted the public discord thread, accepted and published the report with a heavily edited description, pushed an incomplete fix and triggered a security patch release for werkzeug. All this happened within a couple of hours without asking for feedback or waiting for confirmation. The fix partly repaired the ineffective max_content_length parameter, but did not change the insecure defaults. The new report text lacked important details and downplayed the impact. It was published under my name without my consent.

27.10.2024 (+54 days) Opened a public issue mentioning that all Flask and Quart (or other Werkzeug) applications are still vulnerable by default, and suggesting to introduce reasonable defaults for critical config settings. This was marked with the label docs instead of security.

31.10.2024 (+58 days) Werkzeug 3.1.0 was released with secure defaults.

01.10.2024 (+59 days) Noticed and reported that the fix for quart (0.19.7) was not effective.

23.12.2024 (+111 days) Quart was finally fixed, but the report was closed instead of published and no CVE was issued. The original CVE has not been updated either and still contains inaccurate ranges for affected versions.

Lessons learned¶

Communication during this incident left a lot to be desired and the timeline speaks for itself. I will continue reporting issues to Pallets projects responsibly of cause, but no longer spend time arguing with maintainers who clearly do not want to deal with those issues. Next time there will be a single report on a neutral platform (i.e., not GHSA) and a reasonable deadline.