Are you looking for a tutorial on how to develop a proxy server using the NodeJS platform? Then you are on the right page as the article below will provide you with a step-by-step guide on how to get that done easily.
Proxies are known to give Internet users security and protections against surveillance and IP-based restrictions on the Internet by masking your real IP address with a different one.
That is the basics of a proxy server but some do have advanced features such as IP rotation, the ability to make themselves undetectable, and even support for HTTP header modification, among others.
The technology might look alien to some people and they feel it is not something they can replicate. In reality, except if you are trying to build one to use in a production-level environment, a proxy server can be developed easily.
This is because the working model can be relocated in most programming languages without coding hundreds of pages. In this article, we would be working on using the NodeJS platform to develop a proxy server that can act as an intermediary between client applications and web servers on the Internet. It might interest you to know that NodeJS is a good platform for the development of proxy servers.
It is important you know before we continue that the proxy server you will develop with this guide is not up to par with the proxy servers being used for actual usage as this one will not incorporate security measures, is not multithreaded, and does not even handle exceptions. You can just see it as a proof of concept.
An Overview of Proxy Servers
Before going into the coding aspect, it is important you understand the concept behind proxy servers as this will help you to understand the workings better. To many, they think proxies are just a tool for masking IP addresses. Well, while that is the popularly known use case of proxies by regular Internet users, proxies are much more than that.
The best definition for proxy servers is computer software with hardware that acts as an intermediary between client computers and web servers on the Internet. Proxies must not necessarily mask one’s IP address and that can be seen in the case of transparent proxies that have been set up for surveillance by workplaces or for filtering content.
If you can develop working software that requests get routed through to web servers on the Internet, then you have a proxy. Many of the proxy servers on the Internet use specialized proxy server software and do not bother reinventing the wheel.
You will also not need to reinvent the wheel as there are software you can use to create and manage proxies easily. However, it does not hurt if you have a working knowledge of how proxy servers are developed.
How Do Proxy Servers Work?
The proxy server we will be developing will have a simple working mechanism. It does have a predefined target since it is a proof of concept. All we want to prove is that your web requests are hitting the proxy server.
So, when you send a web request with the proxy enabled, what happens is that the requests go through the proxy server which then prints something onto the console before returning the response to you in the browser environment.
It uses the web socket library to achieve that. One thing you need to know is that the code is being run and managed on the same PC and as such, the IP address will be the same.
If you want a different IP address, you will need to host your proxy server script on a different computer so that it can really be an intermediary as proxies are known.
For the script being used in this article, the IP address remains the same and that is why you should treat it more as a proof of concept. Remember, the most important thing is that your requests are hitting the proxy server script before going to the target which is the basic working of a proxy server.
Coding Guide for HTTP Proxy Using NodeJS
For a rudimentary proxy server, you will not need to utilize many libraries and frameworks. In fact, for the one we will be developing, all you need is NodeJS and Express framework. You require the http and https libraries but these come bundled when you install NodeJS.
Installation of Requirements
NodeJS
NodeJS is the most important tool you need to have installed before thinking of developing a proxy server using Javascript. For the newbies, NodeJS is the environment in which Javascript code run and executes outside of the browser environment.
It is the game changer and what brought Javascript close to the likes of other full-fledged languages considering it was initially developed to run in browser environments. Below is how to install NodeJS.
Step 2: Run the installer and follow the steps. The Installer will guide you all through.
Step 3: Agree to the terms and wait for the installation to complete.
Step 4: For some computers, you will need to restart them for the installation to work. For others, you can start using NodeJS straightaway.
Step 5: Enter the code below in the command prompt to confirm whether the installation was successful.
node -v
The command above will print out the nodeJS version on the console if it is installed or throw an exception if the installation wasn’t successful. Try out the code below to also test the nodeJS package installation.
npm -v
ExpressJS
ExpressJS is a web framework for developing web applications. If you want to develop a real proxy server, you need to have it hosted on a different computer or server and as such, it will be in the form of a web application. Below are the steps required to install the ExpressJS web framework. This should be done only after nodeJS has been installed.
Step 1: Create a new folder and make it your working directory.
Step 2: Create the package.json file using the init command
Step 3: Install the ExpressJS framework using the npm command below.
Sample Code for nodeJS HTTP Proxy
Below is a sample code for a HTTP proxy server. The code is quite basic and just a proof of concept. The code is culled from StackOverflow.
/* eslint-disable @typescript-eslint/no-var-requires */
// https://stackoverflow.com/a/63602976/470749
const express = require('express');
const app = express();
const https = require('https');
const http = require('http');
// const { response } = require('express');
const targetUrl = process.env.TARGET_URL || 'https://jsonplaceholder.typicode.com'; // Run localtunnel like `lt -s rscraper -p 8080 --print-requests`; then visit https://yourname.loca.lt/todos/1 .
const proxyServerPort = process.env.PROXY_SERVER_PORT || 8080;
// eslint-disable-next-line max-lines-per-function
app.use('/', function (clientRequest, clientResponse) {
const parsedHost = targetUrl.split('/').splice(2).splice(0, 1).join('/');
let parsedPort;
let parsedSSL;
if (targetUrl.startsWith('https://')) {
parsedPort = 443;
parsedSSL = https;
} else if (targetUrl.startsWith('http://')) {
parsedPort = 80;
parsedSSL = http;
}
const options = {
hostname: parsedHost,
port: parsedPort,
path: clientRequest.url,
method: clientRequest.method,
headers: {
'User-Agent': clientRequest.headers['user-agent'],
},
};
const serverRequest = parsedSSL.request(options, function (serverResponse) {
let body = '';
if (String(serverResponse.headers['content-type']).indexOf('text/html') !== -1) {
serverResponse.on('data', function (chunk) {
body += chunk;
});
serverResponse.on('end', function () {
// Make changes to HTML files when they're done being read.
// body = body.replace(`example`, `Cat!`);
clientResponse.writeHead(serverResponse.statusCode, serverResponse.headers);
clientResponse.end(body);
});
} else {
serverResponse.pipe(clientResponse, {
end: true,
});
clientResponse.contentType(serverResponse.headers['content-type']);
}
});
serverRequest.end();
});
app.listen(proxyServerPort);
console.log(`Proxy server listening on port ${proxyServerPort}`);
FAQs
Q. Should You Develop a Proxy Server Yourself?
You might ask if you should develop a proxy server for yourself or use an already developed one. The right answer to that is — to use a proxy server from a trusted provider. This is because you will be reinventing the wheel and at the same time, spending more money.
It is also important you know that you will need some level of skills and maintenance to be able to set up proxies which will not be worth it at the end of the day. Except you have a good reason to develop a proxy server for your personal use, you should do that yourself as there are trusted providers with high-quality proxies.
Q. What is the Best Way to Develop Proxy Servers?
If you must develop proxy servers yourself, your best birth is using infrastructures and proxy software that have been tested and trusted over the years. While you can code a proxy script yourself, it will not be effective enough in a production environment.
There are proxy software such as squid proxy server that you can use to make it easy for you to set up proxies rather than coding a proxy software from scratch. The advantages of using already-made proxy software include better performance, excellent security, and reliability. You also save coding time and money in the process.
Conclusion
Looking at the above, you can see that we do not advise our readers to reinvent the wheel. You could learn how to develop a proxy server in other to learn it principles and know how they work.
However, it is cheaper and more efficient for you if you make use of proxies from trusted providers as that saves you time and money while providing you better performance.
