Skip to content

Proxy scraper and checker - Asynchronous Python tool for parsing, validating and testing proxies with high concurrency. Features Rich terminal UI, regex IPv4 validation, and httpbin testing.

License

Notifications You must be signed in to change notification settings

kranoley/Proxy-Scraper-Parser-And-Checker

Repository files navigation

Proxy Tool

Advanced Proxy Checker & Scraper

Python Async License Rich

Modern proxy collection and validation tool with beautiful CLI interface


Features

  • Beautiful Interface
  • Async Checking
  • 4 Protocols - SOCKS5, SOCKS4, HTTP, HTTPS
  • Live Statistics
  • Organized Output
  • Auto Detection
  • Configurable
  • Auto Save

Installation

Prerequisites

The current version of the software is developed and tested using Python 3.14.2.


Quick Setup

# Clone or Download the Repository

git clone https://github.com/kranoley/Proxy-Scraper-Parser-And-Checker.git

cd Proxy-Scraper-Parser-And-Checker

pip install -r requirements.txt

python main.py

⚙️ Configuration

The program-settings.config file

The behaviour of the program is controlled from a single file named "program-settings.config".

To add custom sources simply paste them using your favourite text editor into their respective sections. The export path, and other program settings can also be configured here.

If you want to comment out a line, simply inset a hashtag at the start of the line.

You don't need sources for every proxy type, but naturally, at least one should have an entry.

Increasing "Concurrent Checks" will improve the speed of the program, at the cost of higher CPU utilisation. The default value of 1000 is a sensible, but if you have a more (or less) powerful system you might want to change it.

Example program-settings.config

[socks5]
https://raw.githubusercontent.com/dpangestuw/Free-Proxy/refs/heads/main/socks5_proxies.txt

[socks4]
https://raw.githubusercontent.com/dpangestuw/Free-Proxy/refs/heads/main/socks4_proxies.txt

[http]
https://raw.githubusercontent.com/dpangestuw/Free-Proxy/refs/heads/main/http_proxies.txt

[https]
https://raw.githubusercontent.com/proxifly/free-proxy-list/refs/heads/main/proxies/protocols/https/data.txt

[test-urls]
http://httpbin.org/ip

[export-path]
# Leave as 'proxies' to export files to the same directory the program runs from.
proxies

[options]
Timeout: 5
Fetch Timeout: 5
Concurrent Checks: 1000

UI Preview

mg1 mg2 mg3

About

Proxy scraper and checker - Asynchronous Python tool for parsing, validating and testing proxies with high concurrency. Features Rich terminal UI, regex IPv4 validation, and httpbin testing.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Contributors 3

  •  
  •  
  •