-
Notifications
You must be signed in to change notification settings - Fork 344
Open
Labels
Description
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
from scholarly import ProxyGenerator
pg = ProxyGenerator()
http_proxy_url = "http://anystr"
success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)The above code outputs:
Traceback (most recent call last):
File "/root/tracefuzz/test_scholarly.py", line 5, in <module>
success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)
File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 124, in SingleProxy
proxy_works = self._use_proxy(http=http, https=https)
File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 215, in _use_proxy
self._new_session(proxies=proxies)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 491, in _new_session
self._session = httpx.Client(**init_kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'proxies'Expected behavior
success should be True.
Screenshots
In this Python version, httpx.Client receives proxy (URL|str|Proxy) rather than proxies
Desktop (please complete the following information):
- Proxy service: SingleProxy
- python version: [3.13.4]
- OS: [Ubuntu 22.04.5 LTS]
- Version [1.7.11]
Do you plan on contributing?
No, but I think it's a piece of cake to fix this issue, so the developer can make it.
# scholarly/_proxy_generator.py
# ...
if self._proxy_works:
init_kwargs["proxies"] = proxies #.get("http", None)
"""==================================
Add code like this
=================================="""
if sys.version_info >= (3, 13):
init_kwargs["proxy"] = proxies.get("http")
init_kwargs.pop("proxies", None)
"""==================================
=================================="""
self._proxies = proxies
if self.proxy_mode is ProxyMode.SCRAPERAPI:
# SSL Certificate verification must be disabled for
# ScraperAPI requests to work.
# https://www.scraperapi.com/documentation/
init_kwargs["verify"] = False
self._session = httpx.Client(**init_kwargs)
self._webdriver = None