0

I am very new to Python, I am trying to extract data from a site. For that I am stuck on the first step of Login into the site only.

This is what I have tried:

#Importing Libs
import urllib3
from bs4 import BeautifulSoup
import requests
import http

jar = http.cookiejar.CookieJar(policy=None)

http = urllib3.PoolManager()

#Setting account details
acc_pwd = {'user_username':'userABC',
           'user_password':'ABC123'}
#enter URL           
quote_page = 'example.com'
response = http.request('GET', quote_page)
soup = BeautifulSoup(response.data)

print ("Data %s" % soup)

r = requests.get(quote_page, cookies=jar)
r = requests.post(quote_page, cookies=jar, data=acc_pwd)

print ("##############")
print ("RData %s" % r.text)

It takes me back to login page only. Not sure if i am entering the details properly or not.

Brown Bear
  • 19,655
  • 10
  • 58
  • 76
Rdivine
  • 1
  • 1
  • 1
  • This might help you. https://stackoverflow.com/questions/23102833/how-to-scrape-a-website-which-requires-login-using-python-and-beautifulsoup – Anuj Masand Sep 19 '17 at 07:14

1 Answers1

0

this generally works for me:

from bs4 import BeautifulSoup
import requests
from requests import Request, Session
from requests_ntlm import HttpNtlmAuth 

base_url = ''
r = requests.get(base_url, auth=HttpNtlmAuth('domain\\username', 'password'))
smundlay
  • 155
  • 7