Are you looking to speed up the execution of scripts, web applications, and even machine learning systems with cache in Python? Then come in now and learn the basics of cache and how it is implemented in Python to boost performance and reduce the time taken to execute a task.
What do you do when there is a certain bottleneck in your application that takes time to execute, the return value is the same for most times, and this is accessed frequently? For some, looking for ways to speed it up is the fix. However, this is not the only fix and can even be an overkill or not even possible.
There is a solution for this in programming known as caching. With this, you store frequently access data or results of a function so accessing it subsequently costs you less in terms of time and CPU load.
Python does have support for this and is quite useful in areas such as web applications, web scraping, machine learning, and some other CPU-intensive tasks. In this guide, I will discuss caching in Python and how to implement it effectively. Before doing that, let’s take a look at the concept and when it is appropriate to implement.
Concept of Cache and Caching in Programming
Caching was introduced as a result of the bottlenecks introduced as a result of accessing data or doing CPU-intensive tasks that introduce waiting time and affect user experience. Caching is the process of storing data in a temporary storage area outside of its source and accessing it from there when needed.
The area the data is stored temporarily is called a cache and helps speed up data access time, reduce system load, and improve user experience. A perfect example is static files used in the frontend such as the popular Bootstrap.
Over 24% of websites globally use Bootstrap and this has to be retrieved for every request a user sends to access a page that has it loaded. With the aid of caching, browsers do not request Bootstrap all the time. Instead, a copy of Bootstrap is saved in the browser so it saves bandwidth and time it would use for this for downloading something else. This helps considerably with the user experience.
Usually, there are 3 conditions that must be met before caching can be useful and its implementation will make sense. Without meeting either of the requirements, it really wouldn’t make sense to implement it.
- Frequently Accessed Data that Does Not Change Often
This is the number one requirement for setting a caching system. You wouldn’t want to spend resources caching data that would have changed in the next minute as the next minute it would be retrieved, the returned data would be incomplete or incorrect.
Take, for instance, in a betting arbitrage system where betting odds change every seconds, your arbitrage finder tool can’t afford to cache odd values — else, any decision made based on that would be incorrect.
Likewise, it makes no sense to store data that does not get retrieved regularly. In terms of access frequency, the more frequently a particular data point is accessed, the more suitable it is for caching. Take, for instance in the Bootstrap example I gave, for every single time a user accesses a page on a website that uses Bootstrap, it is requested.
This frequency is too much for you to always retrieve it from the original source all the time. Keep it cached in the browser and speed of access is not only faster but less load is placed on the website.
- Speed of Retrieving Data from the Cache is Lower
Yes, speed is one of the determinant of whether you should cache a data point or not as it has a direct correlation with performance. If you have a CPU-intensive function that takes 0.5 seconds to compute and return a value but the return value based on a specific argument does not change often and a lot of users access this function in a short period of time, it makes use to cache it.
Instead of spending 50 minutes to attend to 100 users, you can potentially spend just 2 minutes cumulatively for all the 100 users if the returned value is cached.
- Cost of Retrieving Data from the Cache is Cheaper
The second requirement mentioned above is the caching system has to cost lower in terms of time. There are other costs which are in terms of the monetary cost of retrieval per request or even the load on the entire system. Generally, saving the data in a cache memory should help you spend less than retrieving it from the original source.
In most cases, if implemented correctly, caching should help you save money. This is because the demand on the CPU resource and database access will be less compared to when every request will lead to either carrying out the CPU-intensive task or hitting the database.
How to Implement Caching in Python
The Python programming language support caching. There are multiple options available to you as far as caching is concerned. In this guide, I will focus on two methods of caching in Python. The first method is by using a Python dictionary data structure and the second method is by using the lru_cache decorator in the Python standard library. Let's start with the dictionary method.
Using Python Dictionary for Caching
The method here is simple. You create a Python dictionary that stores values returned by functions. When next you need to calculate such value, you check if the argument is present as a key in the dictionary and if it is, it means it has been calculated in the past and returns the value. The dictionary data structure is an O(n) time complexity structure which means the time taken to retrieve an item is the same regardless of the size of the dictionary.
In the below example, is a function that parses a user agent string into its individual details. This is done by using an API. I will use this to demonstrate caching.
import time
import requests
def user_agent_parser_normal(user_agent_string):
url = 'https://api.apicagent.com'
response = requests.post(url=url, data=user_agent_string, headers={'content-type': 'application/json'})
return response.json()
if __name__ == "__main__":
ua_string = '{"ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"}'
start_time = time.time()
print(user_agent_parser_normal(ua_string))
end_time = time.time()
time_taken = end_time - start_time
print(f'time taken: {time_taken} seconds')
As you can see above, the request module in Python is used to access the API endpoint that does the parsing. If you run the below code, you will get something like the result below.
It took about 1.16 seconds to get this. If you introduce a loop so this is fetched 10 times, the time taken multiples. Let's see it in practice.
if __name__ == "__main__":
ua_string = '{"ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"}'
start_time = time.time()
for i in range(10):
print(user_agent_parser_normal(ua_string))
end_time = time.time()
time_taken = end_time - start_time
print(f'time taken: {time_taken} seconds')
The time taken for this one above is 12.15 as seen below.
The speed differs depending on the Internet speed in your region. But even a value of 0.4 seconds is a lot when multiplied by the number of requests. If you pay per request, the financial cost is also something to worry about.
We can create a caching system that stores the results for user agents and retrieves them without necessarily sending another API request if the same user agent is used. Below is how to convert the code above to utilize caching with the dictionary data structure.
import time
import requests
def memoize_with_dict(func):
cache = {}
# Inner wrapper function to store the data in the cache
def wrapper(*args):
if args in cache:
return cache[args]
else:
result = func(*args)
cache[args] = result
return result
return wrapper
@memoize_with_dict
def user_agent_parser_normal(user_agent_string):
url = 'https://api.apicagent.com'
response = requests.post(url=url, data=user_agent_string, headers={'content-type': 'application/json'})
return response.json()
if __name__ == "__main__":
ua_string = '{"ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"}'
start_time = time.time()
for i in range(10):
print(user_agent_parser_normal(ua_string))
end_time = time.time()
time_taken = end_time - start_time
print(f'time taken: {time_taken} seconds')
In the code above, a decorator function was created that keeps track of results in the Python dictionary. When you send requests, it checks if the value is present in the cache dictionary. If it is, it returns it. Else, it goes ahead to call the function, retrieves the values, and stores them in the cache dictionary for access later. If you run the code above, even with the 10 iterations, below is the result.
As you can see, the time taken is the same as when you send a single request. This is because it does not send additional requests since the user agent string result has also been fetched and cached. While this works in some way, it is not just simple but there is a complex usage that needs something better.
Using LRU Cache Decorator in Python
The Least Recently Used (LRU) caching method keeps a list of cached data and removes the least recently used item in the cache if the maximum size of the cache is reached. You can use the lru_cache decorator from the functools to implement this in Python.
It takes maxsize and typed as arguments. The max size is for defining the maximum size of data entry you want to cache after which the least recently used item is removed from the cache. Let's see how to implement the same code above.
from functools import lru_cache
import time
import requests
@lru_cache(maxsize=100)
def user_agent_parser_normal(user_agent_string):
url = 'https://api.apicagent.com'
response = requests.post(url=url, data=user_agent_string, headers={'content-type': 'application/json'})
return response.json()
if __name__ == "__main__":
ua_string = '{"ua": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"}'
start_time = time.time()
for i in range(10):
print(user_agent_parser_normal(ua_string))
end_time = time.time()
time_taken = end_time - start_time
print(f'time taken: {time_taken} seconds')
As you can see above, we pass 100 as the max size. If you don’t set it, the cache can grow too large with unexpected behavior. The result of the above is almost the same with our own implementation and it seems even faster too. Here, I can’t tell whether the slight performance boost is as a result of network speed fluctuation or the lru_cache performs better — but I suspect the lru_cache performs better.
Conclusion
As a way of concluding this, you need to know that caching as done above is basic and only explains the idea of caching to you which can be implemented in your small applications and scripts. For web applications and other large applications, you will need to use specialized caching systems with their specialized caching memory built for efficient storing and retrieval of cached data.
However, the idea is still still the same — store frequently access data that does not change often in other to improve performance and reduce cost.
