Skip to content

CVE-2021-41125 HTTP authentication credential leak to target websites in scrapy

MediumCVSS 6.5 EPSS 1.3%
Published
Updated
Exploit status
No public exploit known
Technology
UbuntuDebian

Description

Scrapy is a high-level web crawling and scraping framework for Python. If you use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new http_auth_domain spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the w3lib.http.basic_auth_header function to convert your credentials into a value that you can assign to the Authorization header of your request, instead of defining your credentials globally using HttpAuthMiddleware.

Upgrade path

9 packages: 7 fixed, 1 with no fix yet.

PackageAffectedFixed inStatus
ubuntu/python-scrapy(Ubuntu 16.04) = 1.0.0-1; = 1.0.3-1; All versions No fix listed No fix yet
debian/python-scrapy(Debian 12) < 2.5.1-1 2.5.1-1 Fixed
debian/python-scrapy(Debian 13) < 2.5.1-1 2.5.1-1 Fixed
debian/python-scrapy(Debian 14) < 2.5.1-1 2.5.1-1 Fixed
pypi scrapy(PyPI) < 1.8.1; = 0.7; = 0.8; = 0.9 +79 more 1.8.1, 2.5.1 Fixed
scrapy < 1.8.1; = < 1.8.1; = >= 2.0.0, < 2.5.1; >= 2.0.0, < 2.5.1 1.8.1, 2.5.1 Fixed
ubuntu/python-scrapy(Ubuntu Pro 18.04) < 1.5.0-1ubuntu0.1~esm1; = 1.3.0-1~exp2; = 1.4.0-1; = 1.5.0-1 1.5.0-1ubuntu0.1~esm1 Fixed
ubuntu/python-scrapy(Ubuntu Pro 20.04) < 1.7.3-1ubuntu0.1~esm1; = 1.7.3-1 1.7.3-1ubuntu0.1~esm1 Fixed
debian linux = 9.0 No fix listed Versions listed

Affected products

ProductAffected versionsFixed in
pypi scrapy(PyPI) < 1.8.1 1.8.1
pypi scrapy(PyPI) = 0.10.4.2364 No fix listed
pypi scrapy(PyPI) = 0.12.0.2550 No fix listed
pypi scrapy(PyPI) = 0.14.1 No fix listed
pypi scrapy(PyPI) = 0.14.2 No fix listed
pypi scrapy(PyPI) = 0.14.3 No fix listed
pypi scrapy(PyPI) = 0.14.4 No fix listed
pypi scrapy(PyPI) = 0.16.0 No fix listed
pypi scrapy(PyPI) = 0.16.1 No fix listed
pypi scrapy(PyPI) = 0.16.2 No fix listed
pypi scrapy(PyPI) = 0.16.3 No fix listed
pypi scrapy(PyPI) = 0.16.4 No fix listed
pypi scrapy(PyPI) = 0.16.5 No fix listed
pypi scrapy(PyPI) = 0.18.0 No fix listed
pypi scrapy(PyPI) = 0.18.1 No fix listed
pypi scrapy(PyPI) = 0.18.2 No fix listed
pypi scrapy(PyPI) = 0.18.3 No fix listed
pypi scrapy(PyPI) = 0.18.4 No fix listed
pypi scrapy(PyPI) = 0.20.0 No fix listed
pypi scrapy(PyPI) = 0.20.1 No fix listed
pypi scrapy(PyPI) = 0.20.2 No fix listed
pypi scrapy(PyPI) = 0.22.0 No fix listed
pypi scrapy(PyPI) = 0.22.1 No fix listed
pypi scrapy(PyPI) = 0.22.2 No fix listed
pypi scrapy(PyPI) = 0.24.0 No fix listed
Show 75 more
ProductAffected versionsFixed in
pypi scrapy(PyPI) = 0.24.1 No fix listed
pypi scrapy(PyPI) = 0.24.2 No fix listed
pypi scrapy(PyPI) = 0.24.3 No fix listed
pypi scrapy(PyPI) = 0.24.4 No fix listed
pypi scrapy(PyPI) = 0.24.5 No fix listed
pypi scrapy(PyPI) = 0.24.6 No fix listed
pypi scrapy(PyPI) = 0.7 No fix listed
pypi scrapy(PyPI) = 0.8 No fix listed
pypi scrapy(PyPI) = 0.9 No fix listed
pypi scrapy(PyPI) = 1.0.0 No fix listed
pypi scrapy(PyPI) = 1.0.0rc1 No fix listed
pypi scrapy(PyPI) = 1.0.0rc2 No fix listed
pypi scrapy(PyPI) = 1.0.0rc3 No fix listed
pypi scrapy(PyPI) = 1.0.1 No fix listed
pypi scrapy(PyPI) = 1.0.2 No fix listed
pypi scrapy(PyPI) = 1.0.3 No fix listed
pypi scrapy(PyPI) = 1.0.4 No fix listed
pypi scrapy(PyPI) = 1.0.5 No fix listed
pypi scrapy(PyPI) = 1.0.6 No fix listed
pypi scrapy(PyPI) = 1.0.7 No fix listed
pypi scrapy(PyPI) = 1.1.0 No fix listed
pypi scrapy(PyPI) = 1.1.0rc1 No fix listed
pypi scrapy(PyPI) = 1.1.0rc2 No fix listed
pypi scrapy(PyPI) = 1.1.0rc3 No fix listed
pypi scrapy(PyPI) = 1.1.0rc4 No fix listed
pypi scrapy(PyPI) = 1.1.1 No fix listed
pypi scrapy(PyPI) = 1.1.2 No fix listed
pypi scrapy(PyPI) = 1.1.3 No fix listed
pypi scrapy(PyPI) = 1.1.4 No fix listed
pypi scrapy(PyPI) = 1.2.0 No fix listed
pypi scrapy(PyPI) = 1.2.1 No fix listed
pypi scrapy(PyPI) = 1.2.2 No fix listed
pypi scrapy(PyPI) = 1.2.3 No fix listed
pypi scrapy(PyPI) = 1.3.0 No fix listed
pypi scrapy(PyPI) = 1.3.1 No fix listed
pypi scrapy(PyPI) = 1.3.2 No fix listed
pypi scrapy(PyPI) = 1.3.3 No fix listed
pypi scrapy(PyPI) = 1.4.0 No fix listed
pypi scrapy(PyPI) = 1.5.0 No fix listed
pypi scrapy(PyPI) = 1.5.1 No fix listed
pypi scrapy(PyPI) = 1.5.2 No fix listed
pypi scrapy(PyPI) = 1.6.0 No fix listed
pypi scrapy(PyPI) = 1.7.0 No fix listed
pypi scrapy(PyPI) = 1.7.1 No fix listed
pypi scrapy(PyPI) = 1.7.2 No fix listed
pypi scrapy(PyPI) = 1.7.3 No fix listed
pypi scrapy(PyPI) = 1.7.4 No fix listed
pypi scrapy(PyPI) = 1.8.0 No fix listed
pypi scrapy(PyPI) >= 2.0.0, < 2.5.1 2.5.1
pypi scrapy(PyPI) = 2.0.0 No fix listed
pypi scrapy(PyPI) = 2.0.1 No fix listed
pypi scrapy(PyPI) = 2.1.0 No fix listed
pypi scrapy(PyPI) = 2.2.0 No fix listed
pypi scrapy(PyPI) = 2.2.1 No fix listed
pypi scrapy(PyPI) = 2.3.0 No fix listed
pypi scrapy(PyPI) = 2.4.0 No fix listed
pypi scrapy(PyPI) = 2.4.1 No fix listed
pypi scrapy(PyPI) = 2.5.0 No fix listed
debian/python-scrapy(Debian:12) < 2.5.1-1 2.5.1-1
debian/python-scrapy(Debian:13) < 2.5.1-1 2.5.1-1
debian/python-scrapy(Debian:14) < 2.5.1-1 2.5.1-1
scrapy < 1.8.1 1.8.1
scrapy >= 2.0.0, < 2.5.1 2.5.1
debian linux = 9.0 No fix listed
scrapy = < 1.8.1 No fix listed
scrapy = >= 2.0.0, < 2.5.1 No fix listed
ubuntu/python-scrapy(Ubuntu:16.04) All versions No fix listed
ubuntu/python-scrapy(Ubuntu:16.04) = 1.0.0-1 No fix listed
ubuntu/python-scrapy(Ubuntu:16.04) = 1.0.3-1 No fix listed
ubuntu/python-scrapy(Ubuntu:Pro:18.04) < 1.5.0-1ubuntu0.1~esm1 1.5.0-1ubuntu0.1~esm1
ubuntu/python-scrapy(Ubuntu:Pro:18.04) = 1.3.0-1~exp2 No fix listed
ubuntu/python-scrapy(Ubuntu:Pro:18.04) = 1.4.0-1 No fix listed
ubuntu/python-scrapy(Ubuntu:Pro:18.04) = 1.5.0-1 No fix listed
ubuntu/python-scrapy(Ubuntu:Pro:20.04) < 1.7.3-1ubuntu0.1~esm1 1.7.3-1ubuntu0.1~esm1
ubuntu/python-scrapy(Ubuntu:Pro:20.04) = 1.7.3-1 No fix listed

References