Do you want to install Python’s BeautifulSoup on your computer? Come in now and discover how to use the pip install BeautifulSoup command to install BeautifulSoup on Windows, macOS, and Linux.
BeautifulSoup is no doubt the most popular extraction tool when web scraping in Python. While it is more commonly used together with Requests, an HTTP library, it has also seen usage in Selenium and even Scrapy. This is because of its learning curve and how easy it makes parsing and extraction of data from a web page document.
It might interest you to know that BeautifulSoup is not part of the Python Standard Library. This means that you will need to install it as a third-party tool in other to make use of it. In this article, I will show you how to install Python using the pip install command.
How to Install BeautifulSoup on Your Computer
The first step to installing BeautifulSoup is to install Python on your computer. I recommend you download the latest version of Python 3 from the official Python website. Here is the link to the download page. Python is available for Windows, macOS, Linux/UNIX, and many other Operating Systems.
If you have downloaded and installed it, or you even have one installed, you can use the command below in the command prompt or terminal to verify the version of Python installed.
python3 —version >> Python 3.11.0
You can see the output with (>>) at the beginning. If you don’t see this, an error will definitely come up suggesting the installation wasn’t successful.
With Python installed, you can then move to the next step, which is upgrading your pip version and then installing BeautifulSoup. The way you go about this is determined by your computer Operating System.
-
Installing BeautifulSoup for Windows
It is recommended you upgrade your pip version to the latest before installing. Use the code below to install the latest version of pip.
pip3 install —upgrade pip
If the code above ran successfully, it is now time to install BeautifulSoup on your system. You can use the below pip command to get that done too.
pip install beautifulsoup4
As you can see, I didn’t enter beautifulsoup but beautifulsoup4.
-
Installing BeautifulSoup for Linux and macOS
As with Windows, you are recommended to upgrade pip to the latest version to prevent any issues. In most cases, though, it will install with no issues but will log a warning on the screen. Below is the pip command for upgrading your pip version in macOS and Linux.
sudo pip3 install —upgrade pip
The command above will upgrade pip to the latest version for you. With pip upgraded, you can then use the pip command below to install BeautifulSoup.
sudo pip install beautifulsoup4
How to Import and Use BeautifulSoup
Now that you have BeautifulSoup imported, it is time to make use of it, and you use it the same way whether you are using Windows, macOS, or even Linux. If you are used to importing other modules, you might want to import it with “import beautifulsoup4”. But this will throw an exception. Instead, you can import it as you see below.
import requests from bs4 import BeautifulSoup page_content = requests.get(“YOUR_TARGET_URL”).content soup = BeautifulSoup(page_content, “html.parser”) a_elements = soup.find_all(“a”) for a in a_elements: print(a)
The above code will visit any URL you provided as the YOUR_TARGET_URL replacement and download the page content. It will then feed it into BeaitufulSoup and find all of the anchor elements (<a>); then, using the for-loop statement, print out all of the elements on the console. If you check the code above, you will see that I added “html.parser” as the second argument for BeautifulSoup.
This is used to specify the parser to use, in my own case, html.parser. This is not compulsory, as BeautifulSoup will choose for you if you don’t specify it. However, it will print a warning message on the screen for you.
Conclusion
The BeautifulSoup library is incredibly useful for web scraping as one can use it to extract data easily with easy-to-learn APIs. BeautifulSoup is mostly used alongside requests for easy scraping of data from the web, but a good number of developers are using it together with Selenium and other web scraping tools too.
