0

I've seen bits and pieces that help me move forward on this, but I'm trying to tie it all together and I'm not at that point.

In short, I'm trying to connect to a TiVo by logging in, so I can get an xml file of the recordings on the device. I've tried finding ways to include the user and password in the URL, but that doesn't seem to work. When I type the URL into Chrome, I get a requester for username and password:

enter image description here

I'm used to seeing that when logging on to my ISP's router or my firewall. I'm sure this is a commonplace thing, but I don't understand enough about HTTP requests to know how to handle it. I also know it's a problem because this page wants me to login, but the SSL certificate is self-signed.

Once I get past the authentication request, I get the data I need. I see this question about ignoring certificate issues that talks about dealing with a certification check when using urllib2. The first and selected answer does not help me, since it says urllib2 does not check for certification authenticity. The answer with over 120 upvotes does more to address this concern about the certificate.

I also see this question about logins, that, too, addresses part of my problem. I don't understand if that answer is talking about using the same kind of login I need help with. I also don't see how to combine the answers of the two questions, since they use two different ways to retrieve the data from the URL.

I've also found a solution using wget:

 wget --no-check-certificate --http-user=tivo --http-password=your-MAK-here
  -O nowplaying.xml "https://192.168.2.103/TiVoConnect?
 Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes"

I tried wget in a debugging mode to see what it did. I was hoping that, somehow, the username and password were just embedded in the URL, but now I see that they're not and are still passed on, after initial contact, in response to a validation request from the target system.

I need to be able to use something, in Python, that handles the kind of login I'm describing. But I also need that same method to ignore the fact that the certificate is self signed. I see a solution to each one, but not to dealing with both together.

So how do I handle this type of login? (Is it the same as the login issue on the 2nd question) And how do I, at the same time, ignore the certificate issue?

Tango
  • 649
  • 1
  • 12
  • 29

1 Answers1

0

I have tried the following with a local Bitbucket instance with a self-signed cert.

import httplib
import ssl
from requests import Session
from bs4 import BeautifulSoup as bs


server = 'https://bitbucket.xxx.xxx'
#connection = httplib.HTTPSConnection(server, '443', timeout=60, context=ssl.SSLContext(ssl.PROTOCOL_TLSv1))

with Session() as s:
    site = s.get('%s/login' % server, timeout=60, verify=False)
    bs_content = bs(site.content, "html.parser")
    #token = bs_content.find("input", {"name":"csrf_token"})["value"]
    login_data = {"username": "xxx", "password": "xxx"}
    s.post("%s/login" % server, login_data)
    home_page = s.get(server)
    print(home_page.content)

Reference: https://linuxhint.com/logging_into_websites_python/

You could also use something like the Robot Framework - although it may be an overkill for what you are trying to achieve. https://robotframework.org/#introduction

rakesh_singh
  • 86
  • 7
  • I'm getting the warnings about the cert (which is no problem - they're just warnings, not errors). For output, I get:

    Authorization Required

    When you use "%s/login", is that to handle the particular page on the site that is named login or is that to tell Session to handle the login data?     – Tango Aug 15 '19 at 16:31
  • /login handles the authentication only. Once authenticated, you can use s.get(url) to get information from another page (url) – rakesh_singh Aug 16 '19 at 14:30