Requests: HTTP for Humans

인터넷 주요 통신 방식인 HTTP를 다루는 데에 사용할 수 있는 라이브러리

restful API와 json 등을 잘 지원하고 사용하기 쉽고 편리하다.

Requests is ready for today's web.

Keep-Alive & Connection Pooling
International Domains and URLs
Sessions with Cookie Persistence
Browser-style SSL Verification
Automatic Content Decoding
Basic/Digest Authentication
Elegant Key/Value Cookies
Automatic Decompression
Unicode Response Bodies
HTTP(S) Proxy Support
Multipart File Uploads
Streaming Downloads
Connection Timeouts
Chunked Requests
.netrc Support
Thread-safety

기존 코드 with urllib2

python 2.7.x 기준 코드

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import urllib2

gh_url = 'https://api.github.com'

req = urllib2.Request(gh_url)

password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')

auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)

urllib2.install_opener(opener)

handler = urllib2.urlopen(req)

print handler.getcode()
print handler.headers.getheader('content-type')

# ------
# 200
# 'application/json'

같은 코드를 requests로

import requests

r = requests.get('https://api.github.com', auth=('user', 'pass'))

print(r.status_code)
print(r.headers['content-type'])

# ------
# 200
# 'application/json'

설치방법

pip or easy_install

$ pip3 install requests
$ pip3.6 install requests

or

$ easy_install requests

Make a Request

Requests를 이용해서 요청을 만드는 것은 매우 쉽습니다.

먼저 the Requests module을 import 해야 합니다.

>>> import requests

웹페이지를 호출하는 것을 시도해봅니다.

GET 방식

예제 : GitHub’s public timeline 요청

>>> r = requests.get('https://api.github.com/events')

Response 객체가 리턴됩니다. 이를 통해 응답 정보를 조회할 수 있습니다.

POST 방식

Post 호출도 명확하게 HTTP 요청 Type에 따라 메서도가 지원됩니다.

>>> r = requests.post("http://httpbin.org/post")

다른 HTTP 요청 방식

>>> r = requests.put("http://httpbin.org/put")

>>> r = requests.delete("http://httpbin.org/delete")

>>> r = requests.head("http://httpbin.org/get")

>>> r = requests.options("http://httpbin.org/get")

파라미터와 함께 호출

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.get("http://httpbin.org/get", params=payload)

Response Content

응답 객체로부터 컨텐츠 정보를 얻을 수 있습니다.

>>> import requests

>>> r = requests.get('https://api.github.com/events')

>>> r.text
u'[{"repository":{"open_issues":0,"url":"https://github.com/...

Binary Response Content

텍스트 형태가 아닌 컨텐츠는 .content를 통해 얻을 수 있습니다.

>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...

이미지 요청 예제

>>> from PIL import Image
>>> from StringIO import StringIO

>>> i = Image.open(StringIO(r.content))

JSON Response Content

>>> import requests
>>> r = requests.get('https://api.github.com/events')

>>> r.json()
[{u'repository': {u'open_issues': 0, u'url': 'https://github.com/...

JSON decoding fails, r.json raises an exception
if the response gets a 401 (Unauthorized), attempting r.json raises ValueError: No JSON object could be decoded

Custom Headers

>>> import json
>>> url = 'https://api.github.com/some/endpoint'
>>> payload = {'some': 'data'}
>>> headers = {'content-type': 'application/json'}

>>> r = requests.post(url, data=json.dumps(payload), headers=headers)

좀 더 복잡한 POST 요청

>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("http://httpbin.org/post", data=payload)
>>> print(r.text)
{
 ...
 "form": {
 "key2": "value2",
 "key1": "value1"
 },
 ...
}

POST a Multipart-Encoded File

>>> url = 'http://httpbin.org/post'
>>> files = {'file': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)
>>> r.text
{
 ...
 "files": {
 "file": "<censored...binary...data>"
 },
 ...
}

Response Status Codes

>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200

>>> r.status_code == requests.codes.ok
True

>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404

>>> bad_r.raise_for_status()
Traceback (most recent call last):
  File "requests/models.py", line 832, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error

# 응답코드가 200일 경우
>>> r.raise_for_status()
None

Response Headers

>>> r.headers
{
 'content-encoding': 'gzip',
 'transfer-encoding': 'chunked',
 'connection': 'close',
 'server': 'nginx/1.0.4',
 'x-runtime': '148ms',
 'etag': '"e1ca502697e5c9317743dc078f67693f"',
 'content-type': 'application/json'
}

>>> r.headers['Content-Type']
'application/json'

# 사전 방식으로 조회 가능, 대소문자 구분 없음.
>>> r.headers.get('content-type')
'application/json'

Cookies

>>> url = 'http://example.com/some/cookie/setting/url'
>>> r = requests.get(url)

>>> r.cookies['example_cookie_name']
'example_cookie_value'

>>> url = 'http://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')

>>> r = requests.get(url, cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'

Redirection and History

>>> r = requests.get('http://github.com')
>>> r.url
'https://github.com/'

>>> r.status_code
200

>>> r.history
[<Response [301]>]

Timeouts

>>> requests.get('http://github.com', timeout=0.001)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

인간을 위한 HTTP (Requests: HTTP for Humans)