Skip to content

SingleProxy: Failed to create httpx.Client in Python 3.13 #577

@EgodPrime

Description

@EgodPrime

Describe the bug
A clear and concise description of what the bug is.

To Reproduce

from scholarly import ProxyGenerator

pg = ProxyGenerator()
http_proxy_url = "http://anystr"
success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)

The above code outputs:

Traceback (most recent call last):
  File "/root/tracefuzz/test_scholarly.py", line 5, in <module>
    success = pg.SingleProxy(http=http_proxy_url,https=http_proxy_url)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 124, in SingleProxy
    proxy_works = self._use_proxy(http=http, https=https)
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 215, in _use_proxy
    self._new_session(proxies=proxies)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/root/tracefuzz/.venv/lib/python3.13/site-packages/scholarly/_proxy_generator.py", line 491, in _new_session
    self._session = httpx.Client(**init_kwargs)
                    ~~~~~~~~~~~~^^^^^^^^^^^^^^^
TypeError: Client.__init__() got an unexpected keyword argument 'proxies'

Expected behavior
success should be True.

Screenshots

Image

In this Python version, httpx.Client receives proxy (URL|str|Proxy) rather than proxies

Desktop (please complete the following information):

  • Proxy service: SingleProxy
  • python version: [3.13.4]
  • OS: [Ubuntu 22.04.5 LTS]
  • Version [1.7.11]

Do you plan on contributing?
No, but I think it's a piece of cake to fix this issue, so the developer can make it.

# scholarly/_proxy_generator.py
# ...
if self._proxy_works:
            init_kwargs["proxies"] = proxies #.get("http", None)
            """==================================
                Add code like this
            =================================="""
            if sys.version_info >= (3, 13):
                init_kwargs["proxy"] = proxies.get("http") 
                init_kwargs.pop("proxies", None) 
             """==================================
             =================================="""
            self._proxies = proxies
            if self.proxy_mode is ProxyMode.SCRAPERAPI:
                # SSL Certificate verification must be disabled for
                # ScraperAPI requests to work.
                # https://www.scraperapi.com/documentation/
                init_kwargs["verify"] = False
        self._session = httpx.Client(**init_kwargs)
        self._webdriver = None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions