1

I want to get a mobile number from a website with a Python script. The website is showing those mobile numbers like this, if you aren't logged in:

0123 ...

If I'm logged in it looks like this:

123 456

This is the HTML Code from the Website when I'm logged in:

<span id="phone" class="text-bold">0123456</span>

My Code is: 

from bs4 import BeautifulSoup
import requests

def crawler_2():
    url = 'www.example.com'         
    source_code = requests.get(url) 
    plain_text = source_code.text
    soup = BeautifulSoup(plain_text)

    for link in soup.find_all('span', {'class': 'text-bold'}):
        number = link.string
        print (number)

crawler_2()

I'm logged in with my browser , but still cant get the complete Number. In the console the output is the shortened version of the number like I'm not logged in.

Am I on the wrong track or do I need Mechanize or something?

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
Pachenko
  • 11
  • 1
  • 1
    Currently your program cannot login into the site no matter if you are logged in with your browser. Try using saved cookies of the site in your program. – arshovon Feb 21 '17 at 12:00
  • There are a few questions about BeautifulSoup and websites that require login - as you say you could use mechanize e.g. http://stackoverflow.com/questions/23102833/how-to-scrape-a-website-which-requires-login-using-python-and-beautifulsoup or make a session: http://stackoverflow.com/questions/33078805/login-to-website-for-scraping-with-python or http://stackoverflow.com/questions/31233421/login-to-a-website-using-python – doctorlove Feb 21 '17 at 12:15

0 Answers0