Python requests. 403 Forbidden is an error which happens in Python when you do not have a user-agent.
In this article I am going to explain in detail why the error is occurring and how you can go about solving it using multiple solutions which may solve the issue in your particular case.
Explaining The Error : Python requests. 403 Forbidden
First, we should understand that this error is the result of the absence of a user-agent
Second of all, let us try and reproduce the error.
Let us run the Python code bellow.
#
url = 'https://reddit.com/'
result = requests.get(url)
print(result.content.decode())
#
The resulting error is the following
#
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>
#
The important line which represents the error is.
#
<head><title>403 Forbidden</title></head>
#
You can clearly see that the webpage is rejecting GET requests which do not have a User-Agent.
Bellow are great solutions which worked for me and will help you to successfully troubleshoot and solve the error.
Solution 1 : Add user-agent with Headers
First of all, you need to understand that it is normal for pages to rejects GET requests that do not have a User-Agent.
The solution is simple, just specify a user-agent with your Get request using headers like in the modified code bellow.
Basically this code represents the code you had before plus the fix which is adding the user-agent in headers.
Your code starts with the following lines.
#
import requests
url = 'https://reddit.com/'
#
Followed by the fix, also known as headers.
#
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)}
result = requests.get(url, headers=headers)
print(result.content.decode())
#
This fix should be enough to solve the error for most developers, if it did not do the job for you, then keep reading, we have more solutions for you.
Solution 2 : Identify the user-agent
This is more like a complementary solution of the solution above, or a fix of a fix if you know what I mean.
Maybe you do not know where to get the user-agent, this is going to solve your problem.
To find the headers, you should navigate to:
Network, in the developer console. Then click on headers then request headers.
This is an example of a user-agent
#
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36'
#
Solution 3 : Add referer to Headers
Sometimes, none of the above will work, but do not give up, you can try another solution.
In some cases you need to add more elements to headers, like referer for example.
You can modify the code of the first solution, so it can look like this.
#
headers = {
'User-Agent': '...',
'referer': 'https://...'
}
#
You can get the refer by using the developer console like we did before.
If the solutions above helped you, consider supporting us on Kofi, any help is appreciated, writing and working on finding and testing these fixes takes a lot of time.
Summing-up
This is the end of our article, I hope the solutions I presented for the error Python requests 403 Forbidden worked for you, Learning Python is a fun journey, do not let the erros discourage you.
Keep coding and cheers.
If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/