Popular Posts

Wednesday, May 3, 2023

Python - Web crawling exercises

Hello Python developer!

How are you? Hope you've been good and had wonderful time with Python.

There are 10 Python requests module exercises.

Please read and type and go for better understanding of requests module.


Preparation : Python requests module installation

$ pip install requests 


Exercise 1. importing requests module

import requests as rq


Exercise 2. Getting web page data with URL

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

rq.get(url)
rq.post(url) 


Exercise 3. Getting Web server response code

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)
print(res)


Exercise 4. Getting Web server status code(Available page)

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)
print(res.status_code)


Exercise 5. Getting Web server status code(Unavailable web page)

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today//errors"

res = rq.get(url)
print(res)
print(res.status_code)


Exercise 6. Conditional cases by web server status coderror code

import requests as rq

def url_check(url):
    res = rq.get(url)
   
    print(res)
   
    sc = res.status_code
   
    if sc == 200:
        print("%s Request succeeded." %(url))
    elif sc == 404:
        print("%s page was not found." %(url))
    else :
        print("%s had unknown error" %(url))

url_check("https://github.com/Cevastian/Python-Developer-start-today")
url_check("https://github.com/Cevastian/Python-Developer-start-today//a")


Exercise 7. Getting web page header data

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)
print(res.headers)


Exercise 8. Getting web page set cookies 

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

headers = res.headers
print(headers['Set-Cookie'])


Exercise 9. Getting all web page header data

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

headers = res.headers

for header in headers:
    print(headers[header])


Exercise 10. Getting cookies by cookies attribute

import requests as rq

url = "https://github.com/Cevastian/Python-Developer-start-today"

res = rq.get(url)

print(res)

cookies = res.cookies
print(cookies)


Hope these requests code exercises are useful for you.


See you next topics and exercises! Have a nice day!





Python - Web crawling exercises

Hello Python developer! How are you? Hope you've been good and had wonderful time with Python. There are 10 Python requests module exerc...