Update: ICMP / ping must be enabled for some speed tests, otherwise latency checking will fail. Please check that ICMP / ping is enabled with your provider before running a speed test.
IP addresses are often metaphorically described as addresses in the real world; a unique IP is akin to a specific house found on Google Maps. Unlike actual physical addresses, there are millions of IP addresses, and thousands of new ones are generated every day. Most of these IP addresses are actually proxies, which is what you’ve come here to explore.
If I were to continue this house metaphor, a bulk scanner is to proxies as the Google Street View Car is to homes. Someone, or something, goes around and checks in on addresses to make sure they exist, operate, and what sort of condition they’re in.
What kind of scanner are you?
The proxy scanning community can be split up into a few categories:
This is by far the biggest share of users, and most likely the category you fall under. I say that because you’re here, at an article that includes “how to bulk scan” in the title.
These users want to bulk scan and test proxies, often massive lists of free proxies, to see if they work. They also want to know their ports and speed, how blocked or blacklisted those proxies are at a variety of sites, and where they’re located.
The free proxies are available through a number of diﬀerent online sources. You can usually find massive proxy lists assembled by VPN providers, individuals in the community, and other interested parties.
However, these free proxies become defunct or blacklisted very quickly. New, updated lists are often updated on a daily basis. Thus the need to scan them.
Private IP Testers
Many people in the proxy world don’t use free proxies, they buy them in bulk. For the most part, these paid proxies are high-quality and don’t have issues. However, if you purchase, say, 1,000 proxies at a time and use them heavily, you might find certain proxies blocked by certain sites over time.
A bulk scan and test comes in handy so the renters of said proxies can get a grip on which proxies are malfunctioning. A good proxy provider will also be able to do this and should provide you with a new proxy immediately, assuming you didn’t abuse the one that’s broken.
There’s money to be made in everything these days, and that includes proxy scanning. Companies or employed individuals scan proxies in much the same way they scrape data — after all, proxies, their ports and speed and location, are just another type of data. These lists can be sold privately as well as posted for public use.
Professional scanners can also be a system or network administrators who use scanning methods to monitor host or service uptime and service upgrade schedules. Another huge part of professional scanning lies in assessing the security risks of a network, which is basically fake-hacking your own network to further improve its security.
You might actually be a bulk scanner on their way to becoming a professional scanner. That’s just fine. Everyone has to start somewhere.
Before we get into the actual applications that will help you to bulk scan proxies, I need to point out a few pertinent disclaimers.
The proxy world can be a dangerous one, and as such any legitimate article on them needs to be upfront about the legal and illegal issues.
Proxies are often negatively associated in media because they are used by covert hacking groups to steal and crunch massive amounts of data. Also, free proxies tend to be an open can of worms. Many companies only make them free to scrape your data or identity, which can be used in a myriad of negative ways.
In short, a lot of illegal actions can be performed with proxies.
On the bulk scanning front you don’t have quite as much to worry about. Typically, as a proxy scanner, you are just looking for proxies that work. What you decide to do with the proxies that work is up to you — but know what is legal and what isn’t.
The one aspect of proxy scanning that can get tricky is dealing with your Internet Service Provider (ISP). If you’re scanning a huge number of proxies and ports, your ISP is likely to have an issue.
This is partly because scanning for so many proxies and ports is often coupled with harmful behavior (like a DDoS attack), but it’s also a serious drain on ISP services. Many ISPs today oﬀer high data speeds, but most users don’t actually use those resources to the fullest. If you run a massive proxy scan, you’re likely to tie up your ISP in a way it’s not going to be happy about.
With that out of the way, we can continue to the diﬀerent applications that will help you to bulk scan proxies.
Free Proxy Scanning Tools
For the most part, you will need to download application software to do bulk proxy scans. There are some browser-based services that allow you to scan for single proxies at a time, like Proxy Checker, but this is not at all practical for bulk measures.
Luckily, we live in a world that supports open-source creation. This can be especially true in the facets of the technology industry, and shines in the proxy bulk scan area.
NMap (“Network Mapper”)
NMap was originally founded in 1997, and stands as the longest-running free resource for proxy scanning methods. You can download the NMap software, which comes in a command-line format (also called NMap), or a GUI interface format called ZenMap. Installation instructions are available for both of these versions, and the NMap suite runs on Windows and any Linux based platforms.
To put it shortly, NMap is not only a robust tool for scanning proxies, it has a dedicated community of users and builders that consistently improve on its methods. The service was originally created for the professional scanning community — those network admins detailed above — and is still primarily used for that purpose.
The NMap software supports a number of proxy identifiers, including:
- Host discovery. This is your basic proxy scan that checks to see if an IP is “alive” or “dead” (if it will respond or not).
- Port scanning. You can find which ports are open on certain IP addresses.
- Version and OS detection. This is a sophisticated tool that can show you what application is being used on a proxy, and what OS the proxy is running on.
These are the basic uses of NMap, but, as the site notes, with a high level of technical expertise, the options are pretty endless. NMap defaults to SYN proxy scanning, which is the most common and fastest type of scan, but it has a huge list of other methods if SYN will not work, or is not the scan type you want to use.
Angry IP Scanner – Free
Another free proxy scanner, Angry IP Scanner (also known as ipscan) is also open source and works across multiple platforms. Available for Windows, Mac, and Linux, Angry IP is more of a specifically designed tool for the general bulk proxy scanner, though it has professional applications as well. The service has reached over 18 million downloads to date.
Angry IP Scanner can scan proxies in any IP range and with any port, and doesn’t actually require an installation, but is rather a downloadable program you can use immediately. It’s fast and eﬀective for most proxy scanning needs, and uses a separate thread for every IP address scanned.
Angry IP Scanner supports a number of proxy identifiers, including:
- IP addresses and ports, whether or not they are active.
- Can get hostnames and MAC address.
- NetBIOS information, including computer name, workgroup name, and logged in Windows user
- You can set a favorite range of IP addresses in the software
- Web server detection
- Results can be saved in a number of file types, including CSV, TXT, XML or IP-Port.
Also, because Angry IP Scanner is open source, a number of plugins have been developed for it that greatly increase the information you can gather. If you know how to write Java code you can writ your own plugin, too.
Yet Another Proxy Hunter (YAPH) – Free
As the name implies, this is just another ol’ proxy hunter. “Proxy hunter” has been the name of many original free proxy scanners, and YAPH is the most current version.
YAPH is a lot more specific than the previous two examples — it only scans public proxies that are SOCKS4, SOCKS5, or HTTP proxies with CONNECT enabled. In short, YAPH can only scan and ping proxies that can be tunneled. This has its benefits if you’re looking to make a direct connection with proxies, but it’s also normally a way to get in trouble.
To help avoid that trouble, YAPH works in conjunction with both NMaps’s decoy system and ProxyChains‘s stealth system in order to not get caught. I’m not going to give any more details than that.
If you are curious, check out the Github documentation on YAPH , and use at your own risk.
ProxyFire – Free
ProxyFire is (yet) another free proxy scanner. It is downloadable as the ProxyFire Master Suite Free, and has many abilities that make it a viable software application for bulk scanners. It uses the SYN scanning method.
ProxyFire supports a huge amount of very specific proxy settings and identifiers, including:
- SSL/Https, SMTP/EMail, Gateway, Socks4/5, Dangerous IP Test
- Scan for Port, Country, dangerous IP, Duplicate Filter
- Planetlab/CoDeeN proxies Filter
- RBL, hostname Filter
- Diﬀerent type of forums leeching(support LOGIN)
- Non-forum proxy sites
- Diﬀerent search engines leeching
- IP to Country lookup
- Leech proxies from FILE, URL, FTP, EMAIL(pop3)
- Proxies port connecting Test
- SSL/HTTPS URLs
Updated, Offical ProxyFire.net has closed.
The free version allows a max thread check of 1,000, and a max search thread of 5. The paid version is the ProxyFire Master Suite Professional, and costs $89 USD. It includes technical support and updates for a year in addition to a higher range of maximum threads.
Paid Proxy Scan Applications
We can include the paid version of ProxyFire in this list, though I won’t enumerate its abilities again. In general, with so many free proxy scan applications there isn’t a huge need to pay for one. However, I will list one application below.
GSA Proxy Scraper is a proxy Scraper that built-in port scanner to Harvest proxies online and This proxy Scraper can help you test and filter proxies on AutoPilot!
ScrapeBox Proxy Harvester
Maybe you’ve heard of ScrapeBox, maybe not. It’s sort of a proxy paradise tool in that it can do most of the things you’ll want to do with proxies. Not only is it a data scraping machine, but the dedicated Proxy Harvester in ScrapeBox makes it easy to search large batches of proxies in small amounts of time.
The full list of ScrapeBox Proxy Harvester features include:
- Multi-Threaded Connections
- Filter by Country
- Filter by Port
- Filter by Speed
- Add Custom Sources
- Classify Sources
- Custom Testing URL
- Automator Support
ScrapeBox has a one time fee of $97, but if you visit Scrapebox.com/bhw, you can get it for a discounted price. The benefits of ScrapeBox are that the information displayed is well organized and easy to manage, and it includes a host of other tools that will take your proxies farther than the other applications listed here.
The Actual How-To
So far I’ve explained the purpose of proxy scanners, the warnings that go along with them, and a number of premium tools to help you do the scanning. I’m aware that the actual how-to of proxy scanning may not be clear as of yet.
The reality is that each program is slightly diﬀerent. In general, you will want to start by finding a free public proxy list that has been updated recently. You can search for these on Google.
Remember, because you are doing your own scan of the proxies, you’ll weed out the ones that don’t work. Some of the tools above also have options to find safe public proxies, but that’s another subject entirely.
After finding your list, copy and paste them into the application software window that accepts proxies. Depending on the tool it might ask for a file import — if this is the case, make sure to download your free proxy list as a file type that is accepted by the application.
After doing this you’ll see that huge list in the window of your proxy scanner. Depending on the list and your tool’s options, you can then set a number of parameters, like what port to scan, a number of parallels sockets, and how long the connection timeout should be.
After that, simply click the “scan” button. Depending on your search list and internet connection speed, you should have a list of checked proxies in a few minutes. Now you’ve got to decide what to do with them!
- Why SEO Proxies & How to find Proxies to use With SEO software
- Use Chrome Headless and Dedicated Proxies to Scrape Any Website
- How to Automatically Switch Proxies From a List of IPs
Do not want to waste time on these free proxy scanner or paid proxy scraper? Plan to use paid proxies for your marketing needs? easily read this post to choose the best proxy provider!