Not able to scrape "link-text" using python

Asked Aug 12 '20 at 10:49

Active Aug 12 '20 at 10:52

Viewed 24 times

Not able to scrape the link-text using python, there is no output provided.

import requests
from bs4 import BeautifulSoup

 URL='https://pro.imdb.com/companies? 
 ref_=hm_nv_co_pr#companyType=production&sort=ranking&pos=0&page=1'
 headers={"User Agent":'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like 
 Gecko) Chrome/84.0.4147.105 Safari/537.36'}


 def Extract_prod_companies():
   page=requests.get(URL,headers=headers)
   soup=BeautifulSoup(page.text,'html5lib')

   for tag in soup.find_all('a',{"class":"display-company"}):
      print(tag.text)    
 Extract_prod_companies()

edited Aug 12 '20 at 10:52

Danny Staple

7,101
4
43
56

asked Aug 12 '20 at 10:49

Nikita Sharma

Hello, can you paste an example of the tag you are searching for (with its attributes and text) in your question? – Danny Staple Aug 12 '20 at 10:53
[How to scrape a website which requires login using python and beautifulsoup?](https://stackoverflow.com/q/23102833) maybe. – 001 Aug 12 '20 at 10:55
@DannyStaple Kreativ Artists – Nikita Sharma Aug 12 '20 at 10:56
Can you put that in backticks '`', in the answer, so it's formatted please? It didn't seem to come out correctly there. – Danny Staple Aug 12 '20 at 10:58
The url probably needs authentication before listing the results you want. Whn I (or bs4) go to the page I just see a landing page to buy IMDB pro. – Simon Tulling Aug 12 '20 at 10:59

Not able to scrape "link-text" using python

0 Answers0