Are you interested in using a public IP pool? Then come in now to learn more about what they are. You are going to be learning what they are, how they are built, and examples of them in the market.
I am sure you must have heard all the bad things said about public proxies. If you haven’t, I will advise you to read this article from Trustedproxies to know why you shouldn’t use public proxies.
Despite all of the disadvantages of public proxies, just searching for the term “free proxy lists” (Top List website Here) will spring up hundreds of pages dedicated to providing free proxies that are public and open to everyone.
This points to only one fact – public proxies are still in use. If you are one of the users of public proxies, let me introduce you to public IP pools. With a public IP pool, you will spend less time searching for proxies to use from time to time as free proxies are known to stop working after a while as you will have a good number of them ready to be used.
But what are these public/Free IP Proxy pools?
What is a Free Public Proxy Pool?
A Public proxy pool is a managed list of free proxies that are constantly refreshed to remove bad IPs and have a system that decides the set of proxies to assign to a particular user. Some of them even have support for IP rotation. This minimizes the problem associated with open/public proxies in the aspects of a lack of reliability.
However, you just have to accept the fact that public proxy pools only make it easy to get new proxies. In most use cases, using it is a bad idea because of how sites see public proxies. Some sites outrightly deny requests sent through open proxies as they are a den for hackers, spammers, and spammers.
However, there are still some scenarios where public IP pools can be helpful. While most complex sites have very strict systems in place to prevent their usage, there are still many sites that the free proxies can work on.
Usually, they are used for projects that are not speed-sensitive and not critical. Someone not ready to spend money on a forum automation project might use a public proxy pool to send replies and drop comments. They can also use it to scrape some websites, among other things. Most importantly, make sure they are not revealing your real IP Address as some free proxies leaks their users real IP Addresses.
How to Build a Free Proxy Pool
If you know how to write code to automate tasks, then building a public pool won’t be a problem for you. Depending on what you want, you might decide to make your pool to work from the terminal or have a web interface in the form of a restful API. The algorithm for creating a public proxy Pool is described below.
You need a list of websites that provides free proxy lists. Make sure you are after proxies that are highly anonymous and do not reveal your IP Address in any way. Some of the free proxy sites are Proxyrack, Free Proxy List, name, Free Proxy Lists, and many others. The more sites you find, the larger your proxy pool is.
- You need to code web scrapers that scrape the proxy ports and IPs from the public/free proxy list sites, test them to make sure they are functioning then add them to a database.
- You then need to create a routine function to check the database at specific time intervals and test the proxies using an API such as http://httpbin.org/ip to see if the IP Addresses are still available for use. For the ones not working, you need to remove them from the database. This step is very important as free proxies usually stops working after sometimes as the owners of the IP may deactivate or sites block them.
- With the above, you should have a proxy pool to use for your project. The remaining task is how to pull out the proxies from the database. You can create an API for querying the database and providing you working proxies. You can make it in such a way that the proxy provided is rotated so that the system does not keep providing the same proxy to avoid blocks.
You can follow the steps above to create your own public proxy pool in any programming language of your choice. For those that code in Python, check out this IP-Proxy-Pool project on GitHub built using Flask web framework and Redis database.
- Build a Python Keyword Scraper with Beautifulsoup
- Scrapy Vs. Beautifulsoup Vs. Selenium for Web Scraping
Example of Public IP Pools
If you are not a coder, then you have no option but to make use of public proxy pools developed by others. The truth is, I won’t recommend any public IP pool provider for you as I won’t want to be responsible for any adverse effects that might arise from using them.
However, I can only suggest a few of them for you to choose from. Proxies from public proxy pools are very slow and might not be good for most use cases. Some are a den for man in the middle attacks. Most, if not all, are not even reliable as they become easily not available – and get blocked easily too. Here are three public proxy pools below.
Proxy Orbit is a rotating proxy API that returns a new proxy for every API request. They claim that their pool is constantly being refreshed and bad IPs taken out of the pool. You can use proxies from Proxy Orbit for some web scraping tasks provided the site in question does not have a very strict anti-spam system to prevent proxy usage. They support a host of complex sites, including Twitter, and you can specify the site of interest in the API request.
GetProxyList has a free plan with a daily request limit you can use for small tasks. GetProxyList works just like Proxy Orbit; it is a rotating proxy pool that provides random IPs when you send an API. For each of the proxies returned, you get to get information about the proxy, which includes the last time the proxy was tested, the country it originates from, its protocol, and other data in a JSON format.
Do you want to send less than 50 requests in a day? Then Pubproxy has got you covered as it has a free plan that allows up to 50 requests daily. You can decide to get the proxy in either a TXT format or JSON when sending the API request. There are other parameters you can set, such as the speed of the proxy you want, country, port, and a few others.
Tired of slow speed for a free proxy pool? Get paid choice to scale your requests,
Proxy Scraper Software
Yes, you may found that public IP pools are similar to proxies scrape tools and software. such as zennoproxy checker,
The main difference is, those proxy pools, mainly offer the API to give others use, provide API function that easily to extend services.
So, If you're not good at on programming or use the proxy list for one-time, I would suggest you efficiently use Proxy scrape software. Also some Blackhat SEO softwares, Additional those proxy scraping functionalities to run more threads, such as scrapebox and GSA Proxy Scraper,
Conclusion
As a way of concluding this article, I must say that you will be wasting your time thinking you can use a public proxy pool for any task at a reasonable scale as most of them have few IPs in their pool.
If you bring in their lack of reliability into the mix, you would want to make use of paid proxies that aren’t public. I am not saying public IP pools do not work. What I am saying is that they are not suitable for most serious Internet marketing activities.
Free Proxy sources,
